Jul 22 16:07:18.241021 ip-10-0-142-224 systemd[1]: Starting Kubernetes Kubelet...
Jul 22 16:07:20.281214 ip-10-0-142-224 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 16:07:20.281214 ip-10-0-142-224 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 22 16:07:20.281214 ip-10-0-142-224 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 16:07:20.281214 ip-10-0-142-224 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 22 16:07:20.281214 ip-10-0-142-224 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 16:07:20.281214 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.146643    2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 22 16:07:20.281214 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153299    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 16:07:20.281214 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153313    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 16:07:20.281214 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153317    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 16:07:20.281214 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153320    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 16:07:20.281214 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153323    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 16:07:20.281214 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153326    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 16:07:20.281214 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153329    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 16:07:20.281214 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153332    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 16:07:20.220795 ip-10-0-142-224 systemd[1]: Started Kubernetes Kubelet.
Jul 22 16:07:20.282828 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153335    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 16:07:20.282828 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153338    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 16:07:20.282828 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153341    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 16:07:20.282828 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153343    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 16:07:20.282828 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153346    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 16:07:20.282828 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153357    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 16:07:20.282828 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153360    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 16:07:20.282828 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153362    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 16:07:20.282828 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153365    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 16:07:20.282828 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153367    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 16:07:20.282828 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153371    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 16:07:20.282828 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153374    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 16:07:20.282828 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153376    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 16:07:20.282828 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153379    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 16:07:20.282828 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153382    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 16:07:20.282828 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153384    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 16:07:20.282828 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153387    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 16:07:20.282828 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153389    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 16:07:20.282828 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153393    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 16:07:20.282828 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153395    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 16:07:20.283823 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153398    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 16:07:20.283823 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153400    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 16:07:20.283823 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153403    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 16:07:20.283823 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153406    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 16:07:20.283823 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153409    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 16:07:20.283823 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153411    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 16:07:20.283823 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153413    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 16:07:20.283823 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153416    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 16:07:20.283823 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153419    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 16:07:20.283823 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153422    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 16:07:20.283823 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153424    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 16:07:20.283823 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153427    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 16:07:20.283823 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153430    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 16:07:20.283823 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153433    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 16:07:20.283823 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153436    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 16:07:20.283823 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153440    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 16:07:20.283823 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153442    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 16:07:20.283823 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153445    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 16:07:20.283823 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153448    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 16:07:20.283823 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153451    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 16:07:20.285108 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153454    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 16:07:20.285108 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153456    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 16:07:20.285108 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153459    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 16:07:20.285108 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153462    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 16:07:20.285108 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153465    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 16:07:20.285108 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153468    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 16:07:20.285108 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153470    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 16:07:20.285108 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153473    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 16:07:20.285108 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153476    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 16:07:20.285108 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153478    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 16:07:20.285108 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153481    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 16:07:20.285108 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153484    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 16:07:20.285108 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153486    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 16:07:20.285108 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153489    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 16:07:20.285108 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153492    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 16:07:20.285108 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153495    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 16:07:20.285108 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153498    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 16:07:20.285108 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153500    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 16:07:20.285108 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153503    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 16:07:20.285108 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153505    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 16:07:20.286200 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153509    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 16:07:20.286200 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153514    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 16:07:20.286200 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153517    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 16:07:20.286200 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153522    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 16:07:20.286200 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153525    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 16:07:20.286200 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153528    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 16:07:20.286200 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153531    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 16:07:20.286200 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153534    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 16:07:20.286200 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153536    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 16:07:20.286200 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153539    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 16:07:20.286200 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153542    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 16:07:20.286200 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153545    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 16:07:20.286200 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153547    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 16:07:20.286200 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153550    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 16:07:20.286200 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153553    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 16:07:20.286200 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153556    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 16:07:20.286200 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153558    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 16:07:20.286200 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153561    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 16:07:20.286200 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153564    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 16:07:20.287266 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153566    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 16:07:20.287266 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153569    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 16:07:20.287266 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153571    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 16:07:20.287266 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153574    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 16:07:20.287266 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153578    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 16:07:20.287266 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153581    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 16:07:20.287266 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153584    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 16:07:20.287266 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153587    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 16:07:20.287266 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153590    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 16:07:20.287266 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153592    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 16:07:20.287266 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153595    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 16:07:20.287266 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.153598    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 16:07:20.287266 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154323    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 16:07:20.287266 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154330    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 16:07:20.287266 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154334    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 16:07:20.287266 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154336    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 16:07:20.287266 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154339    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 16:07:20.287266 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154342    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 16:07:20.287266 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154345    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 16:07:20.288344 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154347    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 16:07:20.288344 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154350    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 16:07:20.288344 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154353    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 16:07:20.288344 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154355    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 16:07:20.288344 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154358    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 16:07:20.288344 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154361    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 16:07:20.288344 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154364    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 16:07:20.288344 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154366    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 16:07:20.288344 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154369    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 16:07:20.288344 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154372    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 16:07:20.288344 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154374    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 16:07:20.288344 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154377    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 16:07:20.288344 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154379    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 16:07:20.288344 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154382    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 16:07:20.288344 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154386    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 16:07:20.288344 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154388    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 16:07:20.288344 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154391    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 16:07:20.288344 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154394    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 16:07:20.288344 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154396    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 16:07:20.288344 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154399    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 16:07:20.289410 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154401    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 16:07:20.289410 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154404    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 16:07:20.289410 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154406    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 16:07:20.289410 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154409    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 16:07:20.289410 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154411    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 16:07:20.289410 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154414    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 16:07:20.289410 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154416    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 16:07:20.289410 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154419    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 16:07:20.289410 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154421    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 16:07:20.289410 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154426    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 16:07:20.289410 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154429    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 16:07:20.289410 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154432    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 16:07:20.289410 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154435    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 16:07:20.289410 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154438    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 16:07:20.289410 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154442    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 16:07:20.289410 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154446    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 16:07:20.289410 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154450    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 16:07:20.289410 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154452    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 16:07:20.289410 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154455    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 16:07:20.290480 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154458    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 16:07:20.290480 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154461    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 16:07:20.290480 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154464    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 16:07:20.290480 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154467    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 16:07:20.290480 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154469    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 16:07:20.290480 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154472    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 16:07:20.290480 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154474    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 16:07:20.290480 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154477    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 16:07:20.290480 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154479    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 16:07:20.290480 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154482    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 16:07:20.290480 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154485    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 16:07:20.290480 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154487    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 16:07:20.290480 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154490    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 16:07:20.290480 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154492    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 16:07:20.290480 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154495    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 16:07:20.290480 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154497    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 16:07:20.290480 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154500    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 16:07:20.290480 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154505    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 16:07:20.290480 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154508    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 16:07:20.290480 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154511    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 16:07:20.291542 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154514    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 16:07:20.291542 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154516    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 16:07:20.291542 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154519    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 16:07:20.291542 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154521    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 16:07:20.291542 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154524    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 16:07:20.291542 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154527    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 16:07:20.291542 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154529    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 16:07:20.291542 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154532    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 16:07:20.291542 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154534    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 16:07:20.291542 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154537    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 16:07:20.291542 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154539    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 16:07:20.291542 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154543    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 16:07:20.291542 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154546    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 16:07:20.291542 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154549    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 16:07:20.291542 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154552    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 16:07:20.291542 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154555    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 16:07:20.291542 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154557    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 16:07:20.291542 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154560    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 16:07:20.291542 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154562    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 16:07:20.291542 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154565    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 16:07:20.292628 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154567    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 16:07:20.292628 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154570    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 16:07:20.292628 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154572    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 16:07:20.292628 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154575    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 16:07:20.292628 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154578    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 16:07:20.292628 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154580    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 16:07:20.292628 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154583    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 16:07:20.292628 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154585    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 16:07:20.292628 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154588    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 16:07:20.292628 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154591    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 16:07:20.292628 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154595    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 16:07:20.292628 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154599    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 16:07:20.292628 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.154602    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 16:07:20.292628 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155045    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 16:07:20.292628 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155052    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 16:07:20.292628 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155055    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 16:07:20.292628 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155058    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 16:07:20.292628 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155061    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 16:07:20.292628 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155064    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 16:07:20.293672 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155066    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 16:07:20.293672 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155069    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 16:07:20.293672 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155072    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 16:07:20.293672 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155075    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 16:07:20.293672 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155077    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 16:07:20.293672 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155080    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 16:07:20.293672 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155083    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 16:07:20.293672 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155086    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 16:07:20.293672 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155089    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 16:07:20.293672 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155092    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 16:07:20.293672 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155095    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 16:07:20.293672 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155097    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 16:07:20.293672 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155100    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 16:07:20.293672 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155107    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 16:07:20.293672 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155110    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 16:07:20.293672 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155113    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 16:07:20.293672 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155115    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 16:07:20.293672 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155118    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 16:07:20.293672 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155120    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 16:07:20.293672 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155123    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 16:07:20.294749 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155126    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 16:07:20.294749 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155128    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 16:07:20.294749 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155132    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 16:07:20.294749 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155135    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 16:07:20.294749 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155138    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 16:07:20.294749 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155141    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 16:07:20.294749 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155144    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 16:07:20.294749 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155146    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 16:07:20.294749 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155149    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 16:07:20.294749 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155152    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 16:07:20.294749 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155155    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 16:07:20.294749 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155158    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 16:07:20.294749 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155160    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 16:07:20.294749 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155163    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 16:07:20.294749 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155165    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 16:07:20.294749 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155169    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 16:07:20.294749 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155173    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 16:07:20.294749 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155176    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 16:07:20.294749 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155179    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 16:07:20.295607 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155182    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 16:07:20.295607 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155186    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 16:07:20.295607 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155189    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 16:07:20.295607 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155192    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 16:07:20.295607 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155195    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 16:07:20.295607 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155198    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 16:07:20.295607 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155202    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 16:07:20.295607 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155204    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 16:07:20.295607 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155207    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 16:07:20.295607 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155209    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 16:07:20.295607 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155211    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 16:07:20.295607 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155214    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 16:07:20.295607 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155216    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 16:07:20.295607 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155219    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 16:07:20.295607 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155221    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 16:07:20.295607 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155225    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 16:07:20.295607 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155227    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 16:07:20.295607 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155230    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 16:07:20.295607 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155233    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 16:07:20.295607 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155235    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 16:07:20.296695 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155238    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 16:07:20.296695 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155240    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 16:07:20.296695 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155242    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 16:07:20.296695 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155245    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 16:07:20.296695 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155248    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 16:07:20.296695 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155251    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 16:07:20.296695 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155254    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 16:07:20.296695 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155256    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 16:07:20.296695 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155259    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 16:07:20.296695 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155261    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 16:07:20.296695 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155264    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 16:07:20.296695 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155266    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 16:07:20.296695 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155269    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 16:07:20.296695 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155272    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 16:07:20.296695 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155275    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 16:07:20.296695 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155277    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 16:07:20.296695 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155280    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 16:07:20.296695 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155282    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 16:07:20.296695 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155286    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 16:07:20.296695 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155288    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 16:07:20.297550 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155291    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 16:07:20.297550 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155294    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 16:07:20.297550 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155297    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 16:07:20.297550 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155300    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 16:07:20.297550 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155302    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 16:07:20.297550 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155305    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 16:07:20.297550 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155307    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 16:07:20.297550 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155311    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 16:07:20.297550 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155313    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 16:07:20.297550 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155316    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 16:07:20.297550 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155319    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 16:07:20.297550 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155321    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 16:07:20.297550 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155324    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 16:07:20.297550 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.155326    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 16:07:20.297550 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155399    2563 flags.go:64] FLAG: --address="0.0.0.0"
Jul 22 16:07:20.297550 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155406    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 22 16:07:20.297550 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155413    2563 flags.go:64] FLAG: --anonymous-auth="true"
Jul 22 16:07:20.297550 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155417    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 22 16:07:20.297550 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155421    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 22 16:07:20.297550 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155424    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 22 16:07:20.297550 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155428    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 22 16:07:20.298622 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155432    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 22 16:07:20.298622 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155436    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 22 16:07:20.298622 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155439    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 22 16:07:20.298622 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155442    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 22 16:07:20.298622 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155446    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 22 16:07:20.298622 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155449    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 22 16:07:20.298622 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155452    2563 flags.go:64] FLAG: --cgroup-root=""
Jul 22 16:07:20.298622 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155455    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 22 16:07:20.298622 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155458    2563 flags.go:64] FLAG: --client-ca-file=""
Jul 22 16:07:20.298622 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155463    2563 flags.go:64] FLAG: --cloud-provider="external"
Jul 22 16:07:20.298622 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155466    2563 flags.go:64] FLAG: --cluster-dns="[]"
Jul 22 16:07:20.298622 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155470    2563 flags.go:64] FLAG: --cluster-domain=""
Jul 22 16:07:20.298622 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155473    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 22 16:07:20.298622 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155476    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 22 16:07:20.298622 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155479    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 22 16:07:20.298622 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155482    2563 flags.go:64] FLAG: --container-log-max-files="5"
Jul 22 16:07:20.298622 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155486    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 22 16:07:20.298622 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155489    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 22 16:07:20.298622 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155493    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 22 16:07:20.298622 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155496    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 22 16:07:20.298622 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155500    2563 flags.go:64] FLAG: --contention-profiling="false"
Jul 22 16:07:20.298622 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155503    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 22 16:07:20.298622 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155506    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 22 16:07:20.298622 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155509    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 22 16:07:20.302216 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155512    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 22 16:07:20.302216 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155520    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 22 16:07:20.302216 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155523    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 22 16:07:20.302216 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155526    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 22 16:07:20.302216 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155529    2563 flags.go:64] FLAG: --enable-load-reader="false"
Jul 22 16:07:20.302216 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155531    2563 flags.go:64] FLAG: --enable-server="true"
Jul 22 16:07:20.302216 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155535    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 22 16:07:20.302216 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155539    2563 flags.go:64] FLAG: --event-burst="100"
Jul 22 16:07:20.302216 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155542    2563 flags.go:64] FLAG: --event-qps="50"
Jul 22 16:07:20.302216 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155545    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 22 16:07:20.302216 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155548    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 22 16:07:20.302216 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155551    2563 flags.go:64] FLAG: --eviction-hard=""
Jul 22 16:07:20.302216 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155555    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 22 16:07:20.302216 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155558    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 22 16:07:20.302216 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155562    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 22 16:07:20.302216 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155564    2563 flags.go:64] FLAG: --eviction-soft=""
Jul 22 16:07:20.302216 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155568    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 22 16:07:20.302216 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155572    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 22 16:07:20.302216 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155575    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 22 16:07:20.302216 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155578    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 22 16:07:20.302216 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155581    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 22 16:07:20.302216 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155584    2563 flags.go:64] FLAG: --fail-swap-on="true"
Jul 22 16:07:20.302216 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155587    2563 flags.go:64] FLAG: --feature-gates=""
Jul 22 16:07:20.302216 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155590    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 22 16:07:20.302216 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155593    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 22 16:07:20.303515 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155596    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 22 16:07:20.303515 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155601    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 22 16:07:20.303515 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155604    2563 flags.go:64] FLAG: --healthz-port="10248"
Jul 22 16:07:20.303515 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155607    2563 flags.go:64] FLAG: --help="false"
Jul 22 16:07:20.303515 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155610    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-142-224.ec2.internal"
Jul 22 16:07:20.303515 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155613    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 22 16:07:20.303515 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155616    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 22 16:07:20.303515 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155619    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 22 16:07:20.303515 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155622    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 22 16:07:20.303515 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155625    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 22 16:07:20.303515 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155628    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 22 16:07:20.303515 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155631    2563 flags.go:64] FLAG: --image-service-endpoint=""
Jul 22 16:07:20.303515 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155634    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 22 16:07:20.303515 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155637    2563 flags.go:64] FLAG: --kube-api-burst="100"
Jul 22 16:07:20.303515 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155640    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 22 16:07:20.303515 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155643    2563 flags.go:64] FLAG: --kube-api-qps="50"
Jul 22 16:07:20.303515 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155645    2563 flags.go:64] FLAG: --kube-reserved=""
Jul 22 16:07:20.303515 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155648    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 22 16:07:20.303515 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155651    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 22 16:07:20.303515 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155654    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 22 16:07:20.303515 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155657    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 22 16:07:20.303515 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155660    2563 flags.go:64] FLAG: --lock-file=""
Jul 22 16:07:20.303515 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155664    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 22 16:07:20.303515 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155667    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 22 16:07:20.308803 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155672    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 22 16:07:20.308803 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155677    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 22 16:07:20.308803 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155680    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 22 16:07:20.308803 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155683    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 22 16:07:20.308803 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155685    2563 flags.go:64] FLAG: --logging-format="text"
Jul 22 16:07:20.308803 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155688    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 22 16:07:20.308803 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155692    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 22 16:07:20.308803 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155694    2563 flags.go:64] FLAG: --manifest-url=""
Jul 22 16:07:20.308803 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155697    2563 flags.go:64] FLAG: --manifest-url-header=""
Jul 22 16:07:20.308803 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155702    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 22 16:07:20.308803 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155706    2563 flags.go:64] FLAG: --max-open-files="1000000"
Jul 22 16:07:20.308803 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155710    2563 flags.go:64] FLAG: --max-pods="110"
Jul 22 16:07:20.308803 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155713    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 22 16:07:20.308803 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155716    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 22 16:07:20.308803 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155719    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 22 16:07:20.308803 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155722    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 22 16:07:20.308803 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155725    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 22 16:07:20.308803 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155728    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 22 16:07:20.308803 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155731    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 22 16:07:20.308803 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155738    2563 flags.go:64] FLAG: --node-status-max-images="50"
Jul 22 16:07:20.308803 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155741    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 22 16:07:20.308803 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155744    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 22 16:07:20.308803 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155747    2563 flags.go:64] FLAG: --pod-cidr=""
Jul 22 16:07:20.309540 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155749    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 22 16:07:20.309540 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155755    2563 flags.go:64] FLAG: --pod-manifest-path=""
Jul 22 16:07:20.309540 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155758    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 22 16:07:20.309540 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155761    2563 flags.go:64] FLAG: --pods-per-core="0"
Jul 22 16:07:20.309540 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155764    2563 flags.go:64] FLAG: --port="10250"
Jul 22 16:07:20.309540 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155767    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 22 16:07:20.309540 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155770    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b263d5cc4473ee09"
Jul 22 16:07:20.309540 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155773    2563 flags.go:64] FLAG: --qos-reserved=""
Jul 22 16:07:20.309540 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155777    2563 flags.go:64] FLAG: --read-only-port="10255"
Jul 22 16:07:20.309540 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155781    2563 flags.go:64] FLAG: --register-node="true"
Jul 22 16:07:20.309540 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155784    2563 flags.go:64] FLAG: --register-with-taints=""
Jul 22 16:07:20.309540 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155788    2563 flags.go:64] FLAG: --registry-burst="10"
Jul 22 16:07:20.309540 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155791    2563 flags.go:64] FLAG: --registry-qps="5"
Jul 22 16:07:20.309540 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155794    2563 flags.go:64] FLAG: --reserved-cpus=""
Jul 22 16:07:20.309540 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155797    2563 flags.go:64] FLAG: --reserved-memory=""
Jul 22 16:07:20.309540 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155800    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 22 16:07:20.309540 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155803    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 22 16:07:20.309540 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155806    2563 flags.go:64] FLAG: --rotate-certificates="false"
Jul 22 16:07:20.309540 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155809    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 22 16:07:20.309540 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155813    2563 flags.go:64] FLAG: --runonce="false"
Jul 22 16:07:20.309540 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155816    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 22 16:07:20.309540 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155819    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 22 16:07:20.309540 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155823    2563 flags.go:64] FLAG: --seccomp-default="false"
Jul 22 16:07:20.309540 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155825    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 22 16:07:20.309540 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155828    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 22 16:07:20.310325 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155831    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 22 16:07:20.310325 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155835    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 22 16:07:20.310325 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155838    2563 flags.go:64] FLAG: --storage-driver-password="root"
Jul 22 16:07:20.310325 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155841    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 22 16:07:20.310325 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155843    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 22 16:07:20.310325 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155846    2563 flags.go:64] FLAG: --storage-driver-user="root"
Jul 22 16:07:20.310325 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155866    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 22 16:07:20.310325 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155870    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 22 16:07:20.310325 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155873    2563 flags.go:64] FLAG: --system-cgroups=""
Jul 22 16:07:20.310325 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155875    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 22 16:07:20.310325 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155881    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 22 16:07:20.310325 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155884    2563 flags.go:64] FLAG: --tls-cert-file=""
Jul 22 16:07:20.310325 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155887    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 22 16:07:20.310325 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155891    2563 flags.go:64] FLAG: --tls-min-version=""
Jul 22 16:07:20.310325 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155894    2563 flags.go:64] FLAG: --tls-private-key-file=""
Jul 22 16:07:20.310325 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155897    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 22 16:07:20.310325 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155901    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 22 16:07:20.310325 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155904    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 22 16:07:20.310325 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155907    2563 flags.go:64] FLAG: --v="2"
Jul 22 16:07:20.310325 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155911    2563 flags.go:64] FLAG: --version="false"
Jul 22 16:07:20.310325 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155916    2563 flags.go:64] FLAG: --vmodule=""
Jul 22 16:07:20.310325 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155919    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 22 16:07:20.310325 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.155923    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 22 16:07:20.310325 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156054    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 16:07:20.468487 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156059    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 16:07:20.468487 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156062    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 16:07:20.468487 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156067    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 16:07:20.468487 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156070    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 16:07:20.468487 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156072    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 16:07:20.468487 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156075    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 16:07:20.468487 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156078    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 16:07:20.468487 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156081    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 16:07:20.468487 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156083    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 16:07:20.468487 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156086    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 16:07:20.468487 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156089    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 16:07:20.468487 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156091    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 16:07:20.468487 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156094    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 16:07:20.468487 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156097    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 16:07:20.468487 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156099    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 16:07:20.468487 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156102    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 16:07:20.468487 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156105    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 16:07:20.468487 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156107    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 16:07:20.468487 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156110    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 16:07:20.468487 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156112    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 16:07:20.485745 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156115    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 16:07:20.485745 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156117    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 16:07:20.485745 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156121    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 16:07:20.485745 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156125    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 16:07:20.485745 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156129    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 16:07:20.485745 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156133    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 16:07:20.485745 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156136    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 16:07:20.485745 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156139    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 16:07:20.485745 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156142    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 16:07:20.485745 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156145    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 16:07:20.485745 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156148    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 16:07:20.485745 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156151    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 16:07:20.485745 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156153    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 16:07:20.485745 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156156    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 16:07:20.485745 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156160    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 16:07:20.485745 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156163    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 16:07:20.485745 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156166    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 16:07:20.485745 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156168    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 16:07:20.485745 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156171    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 16:07:20.486841 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156173    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 16:07:20.486841 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156176    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 16:07:20.486841 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156178    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 16:07:20.486841 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156181    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 16:07:20.486841 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156184    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 16:07:20.486841 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156187    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 16:07:20.486841 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156190    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 16:07:20.486841 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156194    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 16:07:20.486841 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156197    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 16:07:20.486841 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156201    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 16:07:20.486841 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156205    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 16:07:20.486841 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156208    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 16:07:20.486841 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156211    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 16:07:20.486841 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156214    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 16:07:20.486841 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156217    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 16:07:20.486841 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156219    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 16:07:20.486841 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156222    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 16:07:20.486841 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156226    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 16:07:20.486841 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156229    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 16:07:20.487913 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156232    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 16:07:20.487913 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156235    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 16:07:20.487913 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156238    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 16:07:20.487913 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156241    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 16:07:20.487913 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156244    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 16:07:20.487913 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156246    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 16:07:20.487913 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156249    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 16:07:20.487913 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156251    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 16:07:20.487913 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156255    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 16:07:20.487913 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156257    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 16:07:20.487913 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156260    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 16:07:20.487913 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156263    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 16:07:20.487913 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156265    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 16:07:20.487913 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156268    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 16:07:20.487913 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156271    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 16:07:20.487913 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156274    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 16:07:20.487913 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156277    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 16:07:20.487913 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156279    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 16:07:20.487913 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156282    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 16:07:20.487913 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156285    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 16:07:20.489208 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156287    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 16:07:20.489208 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156290    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 16:07:20.489208 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156292    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 16:07:20.489208 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156295    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 16:07:20.489208 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156297    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 16:07:20.489208 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156300    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 16:07:20.489208 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156303    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 16:07:20.489208 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156305    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 16:07:20.489208 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156308    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 16:07:20.489208 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156310    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 16:07:20.489208 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156314    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 16:07:20.489208 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156316    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 16:07:20.489208 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156319    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 16:07:20.489208 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156322    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 16:07:20.489208 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156324    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 16:07:20.489208 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156327    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 16:07:20.489208 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156329    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 16:07:20.489208 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156332    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 16:07:20.489208 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156334    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 16:07:20.489208 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.156337    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 16:07:20.490065 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.157733    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 16:07:20.490065 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.168062    2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 22 16:07:20.490065 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.168080    2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 22 16:07:20.490065 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.168102    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 22 16:07:20.490065 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.168111    2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 22 16:07:20.490065 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168186    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 16:07:20.490065 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168192    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 16:07:20.490065 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168195    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 16:07:20.490065 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168199    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 16:07:20.490065 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168202    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 16:07:20.490065 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168204    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 16:07:20.490065 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168207    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 16:07:20.490065 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168210    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 16:07:20.490065 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168213    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 16:07:20.491137 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168215    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 16:07:20.491137 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168218    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 16:07:20.491137 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168220    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 16:07:20.491137 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168223    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 16:07:20.491137 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168226    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 16:07:20.491137 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168229    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 16:07:20.491137 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168232    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 16:07:20.491137 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168235    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 16:07:20.491137 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168239    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 16:07:20.491137 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168243    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 16:07:20.491137 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168246    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 16:07:20.491137 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168249    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 16:07:20.491137 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168252    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 16:07:20.491137 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168255    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 16:07:20.491137 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168257    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 16:07:20.491137 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168260    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 16:07:20.491137 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168262    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 16:07:20.491137 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168265    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 16:07:20.491137 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168268    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 16:07:20.492008 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168270    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 16:07:20.492008 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168279    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 16:07:20.492008 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168282    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 16:07:20.492008 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168284    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 16:07:20.492008 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168288    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 16:07:20.492008 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168291    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 16:07:20.492008 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168293    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 16:07:20.492008 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168296    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 16:07:20.492008 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168301    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 16:07:20.492008 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168304    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 16:07:20.492008 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168307    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 16:07:20.492008 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168309    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 16:07:20.492008 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168312    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 16:07:20.492008 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168314    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 16:07:20.492008 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168317    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 16:07:20.492008 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168320    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 16:07:20.492008 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168322    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 16:07:20.492008 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168325    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 16:07:20.492008 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168328    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 16:07:20.492008 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168331    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 16:07:20.493097 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168334    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 16:07:20.493097 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168336    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 16:07:20.493097 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168339    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 16:07:20.493097 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168342    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 16:07:20.493097 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168344    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 16:07:20.493097 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168347    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 16:07:20.493097 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168349    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 16:07:20.493097 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168352    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 16:07:20.493097 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168355    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 16:07:20.493097 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168357    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 16:07:20.493097 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168360    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 16:07:20.493097 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168362    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 16:07:20.493097 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168365    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 16:07:20.493097 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168367    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 16:07:20.493097 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168370    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 16:07:20.493097 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168375    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 16:07:20.493097 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168378    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 16:07:20.493097 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168381    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 16:07:20.493097 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168383    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 16:07:20.494360 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168386    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 16:07:20.494360 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168390    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 16:07:20.494360 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168392    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 16:07:20.494360 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168395    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 16:07:20.494360 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168398    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 16:07:20.494360 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168400    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 16:07:20.494360 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168403    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 16:07:20.494360 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168408    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 16:07:20.494360 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168411    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 16:07:20.494360 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168414    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 16:07:20.494360 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168417    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 16:07:20.494360 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168420    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 16:07:20.494360 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168424    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 16:07:20.494360 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168426    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 16:07:20.494360 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168429    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 16:07:20.494360 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168431    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 16:07:20.494360 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168434    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 16:07:20.494360 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168436    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 16:07:20.494360 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168439    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 16:07:20.494360 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168441    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 16:07:20.495424 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168444    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 16:07:20.495424 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168446    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 16:07:20.495424 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168449    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 16:07:20.495424 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168451    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 16:07:20.495424 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168454    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 16:07:20.495424 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168457    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 16:07:20.495424 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168459    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 16:07:20.495424 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168462    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 16:07:20.495424 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168466    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 16:07:20.495424 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168469    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 16:07:20.495424 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168471    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 16:07:20.495424 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168473    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 16:07:20.495424 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.168478    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 16:07:20.495424 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168596    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 16:07:20.495424 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168602    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 16:07:20.496282 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168605    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 16:07:20.496282 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168608    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 16:07:20.496282 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168611    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 16:07:20.496282 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168614    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 16:07:20.496282 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168616    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 16:07:20.496282 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168619    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 16:07:20.496282 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168622    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 16:07:20.496282 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168624    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 16:07:20.496282 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168627    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 16:07:20.496282 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168629    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 16:07:20.496282 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168632    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 16:07:20.496282 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168635    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 16:07:20.496282 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168637    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 16:07:20.496282 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168640    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 16:07:20.496282 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168642    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 16:07:20.496282 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168645    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 16:07:20.496282 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168648    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 16:07:20.496282 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168650    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 16:07:20.496282 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168653    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 16:07:20.496282 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168655    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 16:07:20.497367 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168658    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 16:07:20.497367 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168661    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 16:07:20.497367 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168663    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 16:07:20.497367 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168666    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 16:07:20.497367 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168668    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 16:07:20.497367 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168673    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 16:07:20.497367 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168676    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 16:07:20.497367 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168679    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 16:07:20.497367 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168682    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 16:07:20.497367 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168684    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 16:07:20.497367 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168686    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 16:07:20.497367 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168691    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 16:07:20.497367 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168694    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 16:07:20.497367 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168696    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 16:07:20.497367 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168699    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 16:07:20.497367 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168701    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 16:07:20.497367 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168704    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 16:07:20.497367 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168707    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 16:07:20.497367 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168709    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 16:07:20.497367 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168712    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 16:07:20.498421 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168715    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 16:07:20.498421 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168717    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 16:07:20.498421 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168720    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 16:07:20.498421 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168722    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 16:07:20.498421 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168725    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 16:07:20.498421 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168727    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 16:07:20.498421 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168730    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 16:07:20.498421 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168732    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 16:07:20.498421 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168735    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 16:07:20.498421 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168738    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 16:07:20.498421 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168740    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 16:07:20.498421 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168743    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 16:07:20.498421 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168745    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 16:07:20.498421 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168748    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 16:07:20.498421 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168751    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 16:07:20.498421 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168754    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 16:07:20.498421 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168757    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 16:07:20.498421 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168760    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 16:07:20.498421 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168763    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 16:07:20.498421 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168766    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 16:07:20.499498 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168769    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 16:07:20.499498 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168771    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 16:07:20.499498 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168774    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 16:07:20.499498 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168778    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 16:07:20.499498 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168780    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 16:07:20.499498 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168783    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 16:07:20.499498 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168786    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 16:07:20.499498 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168788    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 16:07:20.499498 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168791    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 16:07:20.499498 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168793    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 16:07:20.499498 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168796    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 16:07:20.499498 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168800    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 16:07:20.499498 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168804    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 16:07:20.499498 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168807    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 16:07:20.499498 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168810    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 16:07:20.499498 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168813    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 16:07:20.499498 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168816    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 16:07:20.499498 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168818    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 16:07:20.499498 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168821    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 16:07:20.500363 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168823    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 16:07:20.500363 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168826    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 16:07:20.500363 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168828    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 16:07:20.500363 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168831    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 16:07:20.500363 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168833    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 16:07:20.500363 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168836    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 16:07:20.500363 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168838    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 16:07:20.500363 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168841    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 16:07:20.500363 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168844    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 16:07:20.500363 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168846    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 16:07:20.500363 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168866    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 16:07:20.500363 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168869    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 16:07:20.500363 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168872    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 16:07:20.500363 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168875    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 16:07:20.500363 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168878    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 16:07:20.500363 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168881    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 16:07:20.500363 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168886    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 16:07:20.500363 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:20.168889    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 16:07:20.501308 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.168894    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 16:07:20.501308 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.169019    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 22 16:07:20.501308 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.171249    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 22 16:07:20.501308 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.172571    2563 server.go:1019] "Starting client certificate rotation"
Jul 22 16:07:20.501308 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.172668    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 22 16:07:20.501308 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.172693    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 22 16:07:20.501308 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.175002    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 22 16:07:20.501308 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.176425    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 22 16:07:20.501308 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.189450    2563 log.go:25] "Validated CRI v1 runtime API"
Jul 22 16:07:20.501308 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.191567    2563 log.go:25] "Validated CRI v1 image API"
Jul 22 16:07:20.501308 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.192407    2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 22 16:07:20.501308 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.194879    2563 fs.go:135] Filesystem UUIDs: map[26dbd9df-9177-4694-a122-3be4b26d6c11:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 98f55d39-80ff-4d73-9ce5-ea431208da98:/dev/nvme0n1p3]
Jul 22 16:07:20.504587 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.194900    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 22 16:07:20.504798 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.199215    2563 manager.go:216] Machine: {Timestamp:2026-07-22 16:07:20.198669707 +0000 UTC m=+0.329139700 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102252 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2cbf141ee11d56f6ff8487c9ba4061 SystemUUID:ec2cbf14-1ee1-1d56-f6ff-8487c9ba4061 BootID:91896d97-0e08-4341-b950-6deea4a4bac9 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2f:72:35:a4:e3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2f:72:35:a4:e3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fe:a5:a2:22:6f:1d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 22 16:07:20.504798 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.199320    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 22 16:07:20.504798 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.199441    2563 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 22 16:07:20.504798 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.202866    2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 22 16:07:20.504798 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.202898    2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-224.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 22 16:07:20.504798 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.203614    2563 topology_manager.go:138] "Creating topology manager with none policy"
Jul 22 16:07:20.504798 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.203624    2563 container_manager_linux.go:309] "Creating device plugin manager"
Jul 22 16:07:20.504798 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.203632    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.203642    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.204896    2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.205317    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.207833    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.208004    2563 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.208034    2563 kubelet.go:486] "Attempting to sync node with API server"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.208051    2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.208066    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.208075    2563 kubelet.go:398] "Adding apiserver pod source"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.208085    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.208479    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.208496    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.210587    2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.212456    2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.212486    2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.213507    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.213527    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.213539    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.213547    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.213558    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.213565    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.213578    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.213587    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.213600    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.213606    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.213614    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.213620    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.214375    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.214383    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.219605    2563 server.go:1268] "Started kubelet"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.219694    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.219838    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.220203    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:20.220715    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-224.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:20.220771    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.221160    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-224.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.224503    2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.225676    2563 server.go:310] "Adding debug handlers to kubelet server"
Jul 22 16:07:20.511669 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:20.227046    2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 22 16:07:20.513403 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:20.227882    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-224.ec2.internal.18c4a85620fbaeb7  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-224.ec2.internal,UID:ip-10-0-142-224.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-224.ec2.internal,},FirstTimestamp:2026-07-22 16:07:20.219578039 +0000 UTC m=+0.350048032,LastTimestamp:2026-07-22 16:07:20.219578039 +0000 UTC m=+0.350048032,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-224.ec2.internal,}"
Jul 22 16:07:20.513403 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.231050    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 22 16:07:20.513403 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.231068    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 22 16:07:20.513403 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:20.231283    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-224.ec2.internal\" not found"
Jul 22 16:07:20.513403 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.231669    2563 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 22 16:07:20.513403 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.231685    2563 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 22 16:07:20.513403 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.231669    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 22 16:07:20.513403 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.231787    2563 reconstruct.go:97] "Volume reconstruction finished"
Jul 22 16:07:20.513403 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.231795    2563 reconciler.go:29] "Reconciler: start to sync state"
Jul 22 16:07:20.513403 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.231932    2563 factory.go:55] Registering systemd factory
Jul 22 16:07:20.513403 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.231947    2563 factory.go:223] Registration of the systemd container factory successfully
Jul 22 16:07:20.513403 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.232804    2563 factory.go:153] Registering CRI-O factory
Jul 22 16:07:20.513403 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.232818    2563 factory.go:223] Registration of the crio container factory successfully
Jul 22 16:07:20.513403 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.232886    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 22 16:07:20.513403 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.232909    2563 factory.go:103] Registering Raw factory
Jul 22 16:07:20.513403 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.232926    2563 manager.go:1196] Started watching for new ooms in manager
Jul 22 16:07:20.513403 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.233417    2563 manager.go:349] Starting recovery of all containers
Jul 22 16:07:20.513403 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.235710    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-x9h8m"
Jul 22 16:07:20.513403 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.242119    2563 manager.go:354] Recovery completed
Jul 22 16:07:20.513403 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:20.243227    2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-224.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 22 16:07:20.513403 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:20.243341    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 22 16:07:20.513403 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.246296    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-x9h8m"
Jul 22 16:07:20.513403 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.246765    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 16:07:20.513403 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.248702    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-224.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 16:07:20.513403 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.248725    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-224.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 16:07:20.513403 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.248737    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-224.ec2.internal" event="NodeHasSufficientPID"
Jul 22 16:07:20.513403 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.249959    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 22 16:07:20.513403 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.249968    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 22 16:07:20.513403 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.249987    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 22 16:07:20.513403 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.250021    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 16:07:20.513403 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.250746    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-224.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 16:07:20.513403 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.250773    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-224.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 16:07:20.515118 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.250784    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-224.ec2.internal" event="NodeHasSufficientPID"
Jul 22 16:07:20.515118 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.251210    2563 policy_none.go:49] "None policy: Start"
Jul 22 16:07:20.515118 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.251228    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 22 16:07:20.515118 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.251244    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 22 16:07:20.515118 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.252466    2563 policy_none.go:47] "Start"
Jul 22 16:07:20.515118 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.310694    2563 manager.go:337] "Starting Device Plugin manager"
Jul 22 16:07:20.515118 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:20.310915    2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 22 16:07:20.515118 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.310932    2563 server.go:87] "Starting device plugin registration server"
Jul 22 16:07:20.515118 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.311217    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 22 16:07:20.515118 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.311230    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 22 16:07:20.515118 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.311345    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 22 16:07:20.515118 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.311468    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 22 16:07:20.515118 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.311521    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 22 16:07:20.515118 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:20.312734    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 22 16:07:20.515118 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:20.312792    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-224.ec2.internal\" not found"
Jul 22 16:07:20.515118 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.412060    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 16:07:20.515118 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.413823    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-224.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 16:07:20.515118 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.413868    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-224.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 16:07:20.515118 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.413888    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-224.ec2.internal" event="NodeHasSufficientPID"
Jul 22 16:07:20.515118 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.413922    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-224.ec2.internal"
Jul 22 16:07:20.515118 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.425067    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-224.ec2.internal"
Jul 22 16:07:20.515118 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:20.425091    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-224.ec2.internal\": node \"ip-10-0-142-224.ec2.internal\" not found"
Jul 22 16:07:20.515118 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.440688    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 22 16:07:20.515118 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.441832    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 22 16:07:20.515118 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.441894    2563 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 22 16:07:20.515118 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.441917    2563 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 22 16:07:20.515118 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:20.441951    2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 22 16:07:20.515118 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.445608    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 16:07:20.515118 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:20.449092    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-224.ec2.internal\" not found"
Jul 22 16:07:20.654717 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.542297    2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-224.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-224.ec2.internal"]
Jul 22 16:07:20.654717 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.542370    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 16:07:20.654717 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.543393    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-224.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 16:07:20.654717 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.543423    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-224.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 16:07:20.654717 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.543433    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-224.ec2.internal" event="NodeHasSufficientPID"
Jul 22 16:07:20.654717 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.544763    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 16:07:20.654717 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.544876    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-224.ec2.internal"
Jul 22 16:07:20.654717 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.544904    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 16:07:20.654717 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.545538    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-224.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 16:07:20.654717 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.545550    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-224.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 16:07:20.654717 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.545563    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-224.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 16:07:20.654717 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.545569    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-224.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 16:07:20.654717 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.545574    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-224.ec2.internal" event="NodeHasSufficientPID"
Jul 22 16:07:20.654717 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.545578    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-224.ec2.internal" event="NodeHasSufficientPID"
Jul 22 16:07:20.654717 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.546952    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-224.ec2.internal"
Jul 22 16:07:20.654717 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.546979    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 16:07:20.654717 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.547646    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-224.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 16:07:20.654717 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.547671    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-224.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 16:07:20.654717 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.547690    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-224.ec2.internal" event="NodeHasSufficientPID"
Jul 22 16:07:20.654717 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:20.549818    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-224.ec2.internal\" not found"
Jul 22 16:07:20.654717 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:20.591672    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-224.ec2.internal\" not found" node="ip-10-0-142-224.ec2.internal"
Jul 22 16:07:20.654717 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:20.596478    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-224.ec2.internal\" not found" node="ip-10-0-142-224.ec2.internal"
Jul 22 16:07:20.654717 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.633755    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3c7981c2b5b06a134fa86d61e672a2e9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-224.ec2.internal\" (UID: \"3c7981c2b5b06a134fa86d61e672a2e9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-224.ec2.internal"
Jul 22 16:07:20.654717 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.633792    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c7981c2b5b06a134fa86d61e672a2e9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-224.ec2.internal\" (UID: \"3c7981c2b5b06a134fa86d61e672a2e9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-224.ec2.internal"
Jul 22 16:07:20.654717 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.633810    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7b422f2f9cb0c4f90f53104ef61aedd3-config\") pod \"kube-apiserver-proxy-ip-10-0-142-224.ec2.internal\" (UID: \"7b422f2f9cb0c4f90f53104ef61aedd3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-224.ec2.internal"
Jul 22 16:07:20.654717 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:20.650698    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-224.ec2.internal\" not found"
Jul 22 16:07:20.859804 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.734937    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3c7981c2b5b06a134fa86d61e672a2e9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-224.ec2.internal\" (UID: \"3c7981c2b5b06a134fa86d61e672a2e9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-224.ec2.internal"
Jul 22 16:07:20.859804 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.734962    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c7981c2b5b06a134fa86d61e672a2e9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-224.ec2.internal\" (UID: \"3c7981c2b5b06a134fa86d61e672a2e9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-224.ec2.internal"
Jul 22 16:07:20.859804 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.734983    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7b422f2f9cb0c4f90f53104ef61aedd3-config\") pod \"kube-apiserver-proxy-ip-10-0-142-224.ec2.internal\" (UID: \"7b422f2f9cb0c4f90f53104ef61aedd3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-224.ec2.internal"
Jul 22 16:07:20.859804 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.735031    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c7981c2b5b06a134fa86d61e672a2e9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-224.ec2.internal\" (UID: \"3c7981c2b5b06a134fa86d61e672a2e9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-224.ec2.internal"
Jul 22 16:07:20.859804 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.735047    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3c7981c2b5b06a134fa86d61e672a2e9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-224.ec2.internal\" (UID: \"3c7981c2b5b06a134fa86d61e672a2e9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-224.ec2.internal"
Jul 22 16:07:20.859804 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.735076    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7b422f2f9cb0c4f90f53104ef61aedd3-config\") pod \"kube-apiserver-proxy-ip-10-0-142-224.ec2.internal\" (UID: \"7b422f2f9cb0c4f90f53104ef61aedd3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-224.ec2.internal"
Jul 22 16:07:20.859804 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:20.751574    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-224.ec2.internal\" not found"
Jul 22 16:07:20.859804 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:20.852092    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-224.ec2.internal\" not found"
Jul 22 16:07:21.076675 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.895262    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-224.ec2.internal"
Jul 22 16:07:21.076675 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:20.899986    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-224.ec2.internal"
Jul 22 16:07:21.076675 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:20.953051    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-224.ec2.internal\" not found"
Jul 22 16:07:21.076675 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:21.053509    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-224.ec2.internal\" not found"
Jul 22 16:07:21.289477 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:21.154369    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-224.ec2.internal\" not found"
Jul 22 16:07:21.289477 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:21.171662    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 22 16:07:21.289477 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:21.171790    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 22 16:07:21.289477 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:21.250888    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-21 16:02:20 +0000 UTC" deadline="2028-02-01 17:29:42.629533211 +0000 UTC"
Jul 22 16:07:21.289477 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:21.250921    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13417h22m21.378615627s"
Jul 22 16:07:21.289477 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:21.255399    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-224.ec2.internal\" not found"
Jul 22 16:07:21.440405 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:21.355964    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-224.ec2.internal\" not found"
Jul 22 16:07:21.648409 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:21.456725    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-224.ec2.internal\" not found"
Jul 22 16:07:21.648409 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:21.529550    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c7981c2b5b06a134fa86d61e672a2e9.slice/crio-feeb2c9a3169a08dcac88a086325afd88c35a5352aa82d4d6bcc6b689e65c341 WatchSource:0}: Error finding container feeb2c9a3169a08dcac88a086325afd88c35a5352aa82d4d6bcc6b689e65c341: Status 404 returned error can't find the container with id feeb2c9a3169a08dcac88a086325afd88c35a5352aa82d4d6bcc6b689e65c341
Jul 22 16:07:21.648409 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:21.530045    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b422f2f9cb0c4f90f53104ef61aedd3.slice/crio-2c0d27884a918d9cbe1990f922e6c2d1c984380de0cf3bffe9812ba44f0fd3ec WatchSource:0}: Error finding container 2c0d27884a918d9cbe1990f922e6c2d1c984380de0cf3bffe9812ba44f0fd3ec: Status 404 returned error can't find the container with id 2c0d27884a918d9cbe1990f922e6c2d1c984380de0cf3bffe9812ba44f0fd3ec
Jul 22 16:07:21.648409 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:21.536221    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 16:07:21.648409 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:21.557213    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-224.ec2.internal\" not found"
Jul 22 16:07:21.648409 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:21.630392    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 16:07:21.648409 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:21.631314    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-224.ec2.internal"
Jul 22 16:07:21.648409 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:21.641383    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 22 16:07:21.648409 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:21.643110    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-224.ec2.internal"
Jul 22 16:07:21.856640 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:21.656463    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 22 16:07:21.856640 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:21.757231    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 16:07:22.189409 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.178805    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 16:07:22.252744 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.213181    2563 apiserver.go:52] "Watching apiserver"
Jul 22 16:07:22.252744 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.223891    2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 22 16:07:22.252744 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.224438    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-2djqd","kube-system/global-pull-secret-syncer-5dhg6","kube-system/konnectivity-agent-6b2b2","kube-system/kube-apiserver-proxy-ip-10-0-142-224.ec2.internal","openshift-dns/node-resolver-xr6pt","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-224.ec2.internal","openshift-multus/multus-additional-cni-plugins-xtqkt","openshift-multus/network-metrics-daemon-87ltq","openshift-network-operator/iptables-alerter-cq5v7","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff565","openshift-cluster-node-tuning-operator/tuned-85sw4","openshift-image-registry/node-ca-bzc75","openshift-multus/multus-qj9nm","openshift-network-diagnostics/network-check-target-kq5l6"]
Jul 22 16:07:22.252744 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.226124    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff565"
Jul 22 16:07:22.252744 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.228337    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.252744 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.230133    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 22 16:07:22.252744 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.230276    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 22 16:07:22.252744 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.230694    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-qsrjk\""
Jul 22 16:07:22.252744 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.230997    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 22 16:07:22.252744 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.231047    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-q49f7\""
Jul 22 16:07:22.252744 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.231367    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 22 16:07:22.252744 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.231371    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6b2b2"
Jul 22 16:07:22.252744 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.232023    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 22 16:07:22.252744 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.232193    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 22 16:07:22.252744 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.232458    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 22 16:07:22.252744 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.233719    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 22 16:07:22.252744 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.234496    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 22 16:07:22.252744 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.234553    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-mc5d4\""
Jul 22 16:07:22.252744 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.234893    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kq5l6"
Jul 22 16:07:22.252744 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.234905    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 22 16:07:22.252744 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:22.234984    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kq5l6" podUID="36f0cd2c-f5c3-47ce-bb45-0f604e33aed9"
Jul 22 16:07:22.252744 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.236144    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xr6pt"
Jul 22 16:07:22.252744 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.237282    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ltq"
Jul 22 16:07:22.252744 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:22.237359    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87ltq" podUID="6440de1a-aff3-4b9d-a374-4e029ece2bfe"
Jul 22 16:07:22.252744 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.238953    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 22 16:07:22.252744 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.239136    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6m6lz\""
Jul 22 16:07:22.254272 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.239362    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cq5v7"
Jul 22 16:07:22.254272 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.239892    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 22 16:07:22.254272 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.240996    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xtqkt"
Jul 22 16:07:22.254272 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.242081    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 22 16:07:22.254272 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.242117    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 22 16:07:22.254272 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.242253    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-n9fbq\""
Jul 22 16:07:22.254272 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.242753    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 22 16:07:22.254272 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.243023    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:07:22.254272 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:22.243114    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dhg6" podUID="1cd85e6d-9e7c-4c21-bc75-98cc25975bfc"
Jul 22 16:07:22.254272 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.243722    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-host-var-lib-cni-multus\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.254272 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.243751    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-host-var-lib-kubelet\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.254272 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.243773    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/861b97be-45d1-4aee-90eb-b2387ede2142-cni-binary-copy\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.254272 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.243787    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-host-var-lib-cni-bin\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.254272 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.243801    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-etc-kubernetes\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.254272 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.243822    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac-konnectivity-ca\") pod \"konnectivity-agent-6b2b2\" (UID: \"b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac\") " pod="kube-system/konnectivity-agent-6b2b2"
Jul 22 16:07:22.254272 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.243927    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fed3e161-7e00-481a-a172-a751ec55d8cc-registration-dir\") pod \"aws-ebs-csi-driver-node-ff565\" (UID: \"fed3e161-7e00-481a-a172-a751ec55d8cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff565"
Jul 22 16:07:22.254272 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.243955    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fed3e161-7e00-481a-a172-a751ec55d8cc-sys-fs\") pod \"aws-ebs-csi-driver-node-ff565\" (UID: \"fed3e161-7e00-481a-a172-a751ec55d8cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff565"
Jul 22 16:07:22.254272 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.243980    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-system-cni-dir\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.255822 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.244011    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-multus-socket-dir-parent\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.255822 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.244060    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-host-run-netns\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.255822 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.244078    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-hostroot\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.255822 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.244094    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac-agent-certs\") pod \"konnectivity-agent-6b2b2\" (UID: \"b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac\") " pod="kube-system/konnectivity-agent-6b2b2"
Jul 22 16:07:22.255822 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.244154    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fed3e161-7e00-481a-a172-a751ec55d8cc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ff565\" (UID: \"fed3e161-7e00-481a-a172-a751ec55d8cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff565"
Jul 22 16:07:22.255822 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.244170    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fed3e161-7e00-481a-a172-a751ec55d8cc-device-dir\") pod \"aws-ebs-csi-driver-node-ff565\" (UID: \"fed3e161-7e00-481a-a172-a751ec55d8cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff565"
Jul 22 16:07:22.255822 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.244184    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fed3e161-7e00-481a-a172-a751ec55d8cc-etc-selinux\") pod \"aws-ebs-csi-driver-node-ff565\" (UID: \"fed3e161-7e00-481a-a172-a751ec55d8cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff565"
Jul 22 16:07:22.255822 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.244209    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsk2z\" (UniqueName: \"kubernetes.io/projected/fed3e161-7e00-481a-a172-a751ec55d8cc-kube-api-access-gsk2z\") pod \"aws-ebs-csi-driver-node-ff565\" (UID: \"fed3e161-7e00-481a-a172-a751ec55d8cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff565"
Jul 22 16:07:22.255822 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.244232    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-multus-cni-dir\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.255822 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.244268    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-os-release\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.255822 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.244334    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-host-run-k8s-cni-cncf-io\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.255822 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.244378    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-multus-conf-dir\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.255822 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.244409    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/861b97be-45d1-4aee-90eb-b2387ede2142-multus-daemon-config\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.255822 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.244435    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n8vf\" (UniqueName: \"kubernetes.io/projected/861b97be-45d1-4aee-90eb-b2387ede2142-kube-api-access-4n8vf\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.256690 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.244500    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g44rx\" (UniqueName: \"kubernetes.io/projected/36f0cd2c-f5c3-47ce-bb45-0f604e33aed9-kube-api-access-g44rx\") pod \"network-check-target-kq5l6\" (UID: \"36f0cd2c-f5c3-47ce-bb45-0f604e33aed9\") " pod="openshift-network-diagnostics/network-check-target-kq5l6"
Jul 22 16:07:22.256690 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.244526    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fed3e161-7e00-481a-a172-a751ec55d8cc-socket-dir\") pod \"aws-ebs-csi-driver-node-ff565\" (UID: \"fed3e161-7e00-481a-a172-a751ec55d8cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff565"
Jul 22 16:07:22.256690 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.244548    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-cnibin\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.256690 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.244572    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-host-run-multus-certs\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.256690 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.244661    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.256690 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.245271    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 22 16:07:22.256690 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.245618    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 22 16:07:22.256690 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.245724    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-sfcqh\""
Jul 22 16:07:22.256690 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.246435    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.256690 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.248014    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bzc75"
Jul 22 16:07:22.256690 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.253380    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 22 16:07:22.256690 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.253652    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 22 16:07:22.256690 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.253825    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 22 16:07:22.256690 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.254039    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 22 16:07:22.256690 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.254213    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 22 16:07:22.256690 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.254578    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 22 16:07:22.256690 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.254822    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 22 16:07:22.256690 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.254829    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-99sks\""
Jul 22 16:07:22.256690 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.254841    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-v5rm7\""
Jul 22 16:07:22.256690 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.254892    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 22 16:07:22.256690 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.255020    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 22 16:07:22.270589 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.270283    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 22 16:07:22.270589 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.270387    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 22 16:07:22.270791 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.270773    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 22 16:07:22.272018 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.271829    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-g8hlf\""
Jul 22 16:07:22.333671 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.333641    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 22 16:07:22.345233 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.345067    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-etc-kubernetes\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.345233 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.345098    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-host-kubelet\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.345233 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.345119    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-host-slash\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.345233 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.345142    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/64b5ff5e-8328-4e1b-9a8d-341aa0cebba5-cnibin\") pod \"multus-additional-cni-plugins-xtqkt\" (UID: \"64b5ff5e-8328-4e1b-9a8d-341aa0cebba5\") " pod="openshift-multus/multus-additional-cni-plugins-xtqkt"
Jul 22 16:07:22.345233 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.345164    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5e6f48d6-b0c6-4827-a1de-17ec418ccaf2-serviceca\") pod \"node-ca-bzc75\" (UID: \"5e6f48d6-b0c6-4827-a1de-17ec418ccaf2\") " pod="openshift-image-registry/node-ca-bzc75"
Jul 22 16:07:22.345233 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.345191    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9l2kr\" (UniqueName: \"kubernetes.io/projected/5e6f48d6-b0c6-4827-a1de-17ec418ccaf2-kube-api-access-9l2kr\") pod \"node-ca-bzc75\" (UID: \"5e6f48d6-b0c6-4827-a1de-17ec418ccaf2\") " pod="openshift-image-registry/node-ca-bzc75"
Jul 22 16:07:22.345606 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.345259    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fed3e161-7e00-481a-a172-a751ec55d8cc-sys-fs\") pod \"aws-ebs-csi-driver-node-ff565\" (UID: \"fed3e161-7e00-481a-a172-a751ec55d8cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff565"
Jul 22 16:07:22.345606 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.345342    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-host-run-netns\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.345606 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.345350    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-etc-kubernetes\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.345606 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.345350    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fed3e161-7e00-481a-a172-a751ec55d8cc-sys-fs\") pod \"aws-ebs-csi-driver-node-ff565\" (UID: \"fed3e161-7e00-481a-a172-a751ec55d8cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff565"
Jul 22 16:07:22.345606 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.345382    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/64b5ff5e-8328-4e1b-9a8d-341aa0cebba5-cni-binary-copy\") pod \"multus-additional-cni-plugins-xtqkt\" (UID: \"64b5ff5e-8328-4e1b-9a8d-341aa0cebba5\") " pod="openshift-multus/multus-additional-cni-plugins-xtqkt"
Jul 22 16:07:22.345606 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.345390    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-host-run-netns\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.345606 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.345435    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a2dfe635-d673-4f73-94cc-d852424b78d9-lib-modules\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.345606 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.345464    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.345606 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.345495    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22kgp\" (UniqueName: \"kubernetes.io/projected/d5ae481a-e39e-4c39-8f8e-e9255121886d-kube-api-access-22kgp\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.345606 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.345523    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/64b5ff5e-8328-4e1b-9a8d-341aa0cebba5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xtqkt\" (UID: \"64b5ff5e-8328-4e1b-9a8d-341aa0cebba5\") " pod="openshift-multus/multus-additional-cni-plugins-xtqkt"
Jul 22 16:07:22.345606 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.345553    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fed3e161-7e00-481a-a172-a751ec55d8cc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ff565\" (UID: \"fed3e161-7e00-481a-a172-a751ec55d8cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff565"
Jul 22 16:07:22.345606 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.345579    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fed3e161-7e00-481a-a172-a751ec55d8cc-device-dir\") pod \"aws-ebs-csi-driver-node-ff565\" (UID: \"fed3e161-7e00-481a-a172-a751ec55d8cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff565"
Jul 22 16:07:22.345606 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.345606    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fed3e161-7e00-481a-a172-a751ec55d8cc-etc-selinux\") pod \"aws-ebs-csi-driver-node-ff565\" (UID: \"fed3e161-7e00-481a-a172-a751ec55d8cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff565"
Jul 22 16:07:22.346172 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.345634    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-host-run-k8s-cni-cncf-io\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.346172 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.345660    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g44rx\" (UniqueName: \"kubernetes.io/projected/36f0cd2c-f5c3-47ce-bb45-0f604e33aed9-kube-api-access-g44rx\") pod \"network-check-target-kq5l6\" (UID: \"36f0cd2c-f5c3-47ce-bb45-0f604e33aed9\") " pod="openshift-network-diagnostics/network-check-target-kq5l6"
Jul 22 16:07:22.346172 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.345674    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fed3e161-7e00-481a-a172-a751ec55d8cc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ff565\" (UID: \"fed3e161-7e00-481a-a172-a751ec55d8cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff565"
Jul 22 16:07:22.346172 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.345685    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6440de1a-aff3-4b9d-a374-4e029ece2bfe-metrics-certs\") pod \"network-metrics-daemon-87ltq\" (UID: \"6440de1a-aff3-4b9d-a374-4e029ece2bfe\") " pod="openshift-multus/network-metrics-daemon-87ltq"
Jul 22 16:07:22.346172 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.345720    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1cd85e6d-9e7c-4c21-bc75-98cc25975bfc-dbus\") pod \"global-pull-secret-syncer-5dhg6\" (UID: \"1cd85e6d-9e7c-4c21-bc75-98cc25975bfc\") " pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:07:22.346172 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.345745    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-etc-openvswitch\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.346172 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.345756    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fed3e161-7e00-481a-a172-a751ec55d8cc-device-dir\") pod \"aws-ebs-csi-driver-node-ff565\" (UID: \"fed3e161-7e00-481a-a172-a751ec55d8cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff565"
Jul 22 16:07:22.346172 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.345930    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fed3e161-7e00-481a-a172-a751ec55d8cc-etc-selinux\") pod \"aws-ebs-csi-driver-node-ff565\" (UID: \"fed3e161-7e00-481a-a172-a751ec55d8cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff565"
Jul 22 16:07:22.346172 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.345987    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-host-run-k8s-cni-cncf-io\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.346552 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.346249    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tcrz\" (UniqueName: \"kubernetes.io/projected/6440de1a-aff3-4b9d-a374-4e029ece2bfe-kube-api-access-4tcrz\") pod \"network-metrics-daemon-87ltq\" (UID: \"6440de1a-aff3-4b9d-a374-4e029ece2bfe\") " pod="openshift-multus/network-metrics-daemon-87ltq"
Jul 22 16:07:22.346552 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.346276    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-run-ovn\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.346552 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.346297    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2193e465-f388-4298-80aa-c4fd96ec3f2f-tmp-dir\") pod \"node-resolver-xr6pt\" (UID: \"2193e465-f388-4298-80aa-c4fd96ec3f2f\") " pod="openshift-dns/node-resolver-xr6pt"
Jul 22 16:07:22.346552 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.346335    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/64b5ff5e-8328-4e1b-9a8d-341aa0cebba5-os-release\") pod \"multus-additional-cni-plugins-xtqkt\" (UID: \"64b5ff5e-8328-4e1b-9a8d-341aa0cebba5\") " pod="openshift-multus/multus-additional-cni-plugins-xtqkt"
Jul 22 16:07:22.346552 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.346357    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2193e465-f388-4298-80aa-c4fd96ec3f2f-hosts-file\") pod \"node-resolver-xr6pt\" (UID: \"2193e465-f388-4298-80aa-c4fd96ec3f2f\") " pod="openshift-dns/node-resolver-xr6pt"
Jul 22 16:07:22.346552 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.346385    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fed3e161-7e00-481a-a172-a751ec55d8cc-socket-dir\") pod \"aws-ebs-csi-driver-node-ff565\" (UID: \"fed3e161-7e00-481a-a172-a751ec55d8cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff565"
Jul 22 16:07:22.346552 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.346417    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-cnibin\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.346552 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.346443    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/662d650d-8f3e-45ce-900d-e3a0ed385342-host-slash\") pod \"iptables-alerter-cq5v7\" (UID: \"662d650d-8f3e-45ce-900d-e3a0ed385342\") " pod="openshift-network-operator/iptables-alerter-cq5v7"
Jul 22 16:07:22.346552 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.346466    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64b5ff5e-8328-4e1b-9a8d-341aa0cebba5-system-cni-dir\") pod \"multus-additional-cni-plugins-xtqkt\" (UID: \"64b5ff5e-8328-4e1b-9a8d-341aa0cebba5\") " pod="openshift-multus/multus-additional-cni-plugins-xtqkt"
Jul 22 16:07:22.346552 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.346487    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a2dfe635-d673-4f73-94cc-d852424b78d9-etc-modprobe-d\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.346552 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.346510    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-host-var-lib-kubelet\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.346552 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.346530    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-run-systemd\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.347137 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.346565    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-node-log\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.347137 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.346583    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fed3e161-7e00-481a-a172-a751ec55d8cc-socket-dir\") pod \"aws-ebs-csi-driver-node-ff565\" (UID: \"fed3e161-7e00-481a-a172-a751ec55d8cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff565"
Jul 22 16:07:22.347137 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.346588    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d5ae481a-e39e-4c39-8f8e-e9255121886d-ovnkube-script-lib\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.347137 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.346630    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/662d650d-8f3e-45ce-900d-e3a0ed385342-iptables-alerter-script\") pod \"iptables-alerter-cq5v7\" (UID: \"662d650d-8f3e-45ce-900d-e3a0ed385342\") " pod="openshift-network-operator/iptables-alerter-cq5v7"
Jul 22 16:07:22.347137 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.346640    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-cnibin\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.347137 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.346655    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a2dfe635-d673-4f73-94cc-d852424b78d9-etc-systemd\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.347137 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.346678    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a2dfe635-d673-4f73-94cc-d852424b78d9-tmp\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.347137 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.346705    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a2dfe635-d673-4f73-94cc-d852424b78d9-etc-sysctl-conf\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.347137 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.346726    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-host-var-lib-kubelet\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.347137 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.346767    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/861b97be-45d1-4aee-90eb-b2387ede2142-cni-binary-copy\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.347137 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.346808    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac-konnectivity-ca\") pod \"konnectivity-agent-6b2b2\" (UID: \"b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac\") " pod="kube-system/konnectivity-agent-6b2b2"
Jul 22 16:07:22.347137 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.346908    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1cd85e6d-9e7c-4c21-bc75-98cc25975bfc-original-pull-secret\") pod \"global-pull-secret-syncer-5dhg6\" (UID: \"1cd85e6d-9e7c-4c21-bc75-98cc25975bfc\") " pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:07:22.347137 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.346937    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8gn4\" (UniqueName: \"kubernetes.io/projected/2193e465-f388-4298-80aa-c4fd96ec3f2f-kube-api-access-n8gn4\") pod \"node-resolver-xr6pt\" (UID: \"2193e465-f388-4298-80aa-c4fd96ec3f2f\") " pod="openshift-dns/node-resolver-xr6pt"
Jul 22 16:07:22.347137 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.346984    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/64b5ff5e-8328-4e1b-9a8d-341aa0cebba5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xtqkt\" (UID: \"64b5ff5e-8328-4e1b-9a8d-341aa0cebba5\") " pod="openshift-multus/multus-additional-cni-plugins-xtqkt"
Jul 22 16:07:22.347741 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.347010    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a2dfe635-d673-4f73-94cc-d852424b78d9-run\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.347741 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.347398    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fed3e161-7e00-481a-a172-a751ec55d8cc-registration-dir\") pod \"aws-ebs-csi-driver-node-ff565\" (UID: \"fed3e161-7e00-481a-a172-a751ec55d8cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff565"
Jul 22 16:07:22.347982 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.347960    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/861b97be-45d1-4aee-90eb-b2387ede2142-cni-binary-copy\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.348084 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.348060    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fed3e161-7e00-481a-a172-a751ec55d8cc-registration-dir\") pod \"aws-ebs-csi-driver-node-ff565\" (UID: \"fed3e161-7e00-481a-a172-a751ec55d8cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff565"
Jul 22 16:07:22.348418 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.348387    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-system-cni-dir\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.348500 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.348437    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-multus-socket-dir-parent\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.348500 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.348493    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-hostroot\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.348587 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.348528    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-host-run-netns\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.348587 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.348556    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac-konnectivity-ca\") pod \"konnectivity-agent-6b2b2\" (UID: \"b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac\") " pod="kube-system/konnectivity-agent-6b2b2"
Jul 22 16:07:22.348587 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.348562    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-run-openvswitch\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.348675 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.348588    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a2dfe635-d673-4f73-94cc-d852424b78d9-host\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.348675 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.348634    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac-agent-certs\") pod \"konnectivity-agent-6b2b2\" (UID: \"b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac\") " pod="kube-system/konnectivity-agent-6b2b2"
Jul 22 16:07:22.348735 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.348717    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-log-socket\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.348735 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.348719    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-multus-socket-dir-parent\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.348793 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.348760    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-host-cni-netd\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.348825 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.348793    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5e6f48d6-b0c6-4827-a1de-17ec418ccaf2-host\") pod \"node-ca-bzc75\" (UID: \"5e6f48d6-b0c6-4827-a1de-17ec418ccaf2\") " pod="openshift-image-registry/node-ca-bzc75"
Jul 22 16:07:22.348825 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.348793    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-system-cni-dir\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.349497 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.349218    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 22 16:07:22.349497 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.349295    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-hostroot\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.349497 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.349368    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsk2z\" (UniqueName: \"kubernetes.io/projected/fed3e161-7e00-481a-a172-a751ec55d8cc-kube-api-access-gsk2z\") pod \"aws-ebs-csi-driver-node-ff565\" (UID: \"fed3e161-7e00-481a-a172-a751ec55d8cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff565"
Jul 22 16:07:22.349497 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.349421    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-multus-cni-dir\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.349497 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.349455    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-os-release\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.349759 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.349506    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-multus-conf-dir\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.349759 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.349540    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1cd85e6d-9e7c-4c21-bc75-98cc25975bfc-kubelet-config\") pod \"global-pull-secret-syncer-5dhg6\" (UID: \"1cd85e6d-9e7c-4c21-bc75-98cc25975bfc\") " pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:07:22.349759 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.349569    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-host-cni-bin\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.349759 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.349602    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a2dfe635-d673-4f73-94cc-d852424b78d9-etc-sysconfig\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.349759 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.349654    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/861b97be-45d1-4aee-90eb-b2387ede2142-multus-daemon-config\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.349759 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.349681    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n8vf\" (UniqueName: \"kubernetes.io/projected/861b97be-45d1-4aee-90eb-b2387ede2142-kube-api-access-4n8vf\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.349759 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.349705    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-var-lib-openvswitch\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.349759 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.349731    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d5ae481a-e39e-4c39-8f8e-e9255121886d-ovn-node-metrics-cert\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.350116 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.349776    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-multus-cni-dir\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.350116 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.349787    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/64b5ff5e-8328-4e1b-9a8d-341aa0cebba5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xtqkt\" (UID: \"64b5ff5e-8328-4e1b-9a8d-341aa0cebba5\") " pod="openshift-multus/multus-additional-cni-plugins-xtqkt"
Jul 22 16:07:22.350116 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.349881    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-os-release\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.350298 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.350202    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-multus-conf-dir\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.350298 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.350242    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a2dfe635-d673-4f73-94cc-d852424b78d9-etc-sysctl-d\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.350298 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.350292    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/861b97be-45d1-4aee-90eb-b2387ede2142-multus-daemon-config\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.350419 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.350346    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns8k6\" (UniqueName: \"kubernetes.io/projected/a2dfe635-d673-4f73-94cc-d852424b78d9-kube-api-access-ns8k6\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.350419 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.350402    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-host-run-multus-certs\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.350509 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.350439    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-host-run-ovn-kubernetes\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.350509 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.350476    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-host-run-multus-certs\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.350600 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.350506    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s96jr\" (UniqueName: \"kubernetes.io/projected/64b5ff5e-8328-4e1b-9a8d-341aa0cebba5-kube-api-access-s96jr\") pod \"multus-additional-cni-plugins-xtqkt\" (UID: \"64b5ff5e-8328-4e1b-9a8d-341aa0cebba5\") " pod="openshift-multus/multus-additional-cni-plugins-xtqkt"
Jul 22 16:07:22.350600 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.350538    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2dfe635-d673-4f73-94cc-d852424b78d9-var-lib-kubelet\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.350600 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.350582    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-host-var-lib-cni-multus\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.350735 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.350627    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-host-var-lib-cni-multus\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.350735 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.350671    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcrwr\" (UniqueName: \"kubernetes.io/projected/662d650d-8f3e-45ce-900d-e3a0ed385342-kube-api-access-tcrwr\") pod \"iptables-alerter-cq5v7\" (UID: \"662d650d-8f3e-45ce-900d-e3a0ed385342\") " pod="openshift-network-operator/iptables-alerter-cq5v7"
Jul 22 16:07:22.350735 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.350717    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2dfe635-d673-4f73-94cc-d852424b78d9-etc-kubernetes\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.350885 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.350762    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-systemd-units\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.350885 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.350810    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d5ae481a-e39e-4c39-8f8e-e9255121886d-ovnkube-config\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.350885 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.350847    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d5ae481a-e39e-4c39-8f8e-e9255121886d-env-overrides\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.351023 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.350897    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2dfe635-d673-4f73-94cc-d852424b78d9-sys\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.351023 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.350934    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a2dfe635-d673-4f73-94cc-d852424b78d9-etc-tuned\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.351023 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.350980    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-host-var-lib-cni-bin\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.351023 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.351016    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/861b97be-45d1-4aee-90eb-b2387ede2142-host-var-lib-cni-bin\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.353794 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.353774    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac-agent-certs\") pod \"konnectivity-agent-6b2b2\" (UID: \"b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac\") " pod="kube-system/konnectivity-agent-6b2b2"
Jul 22 16:07:22.354169 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:22.354153    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 16:07:22.354253 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:22.354176    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 16:07:22.354253 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:22.354191    2563 projected.go:196] Error preparing data for projected volume kube-api-access-g44rx for pod openshift-network-diagnostics/network-check-target-kq5l6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 16:07:22.354345 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:22.354265    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/36f0cd2c-f5c3-47ce-bb45-0f604e33aed9-kube-api-access-g44rx podName:36f0cd2c-f5c3-47ce-bb45-0f604e33aed9 nodeName:}" failed. No retries permitted until 2026-07-22 16:07:22.854245853 +0000 UTC m=+2.984715873 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-g44rx" (UniqueName: "kubernetes.io/projected/36f0cd2c-f5c3-47ce-bb45-0f604e33aed9-kube-api-access-g44rx") pod "network-check-target-kq5l6" (UID: "36f0cd2c-f5c3-47ce-bb45-0f604e33aed9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 16:07:22.372121 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.372089    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n8vf\" (UniqueName: \"kubernetes.io/projected/861b97be-45d1-4aee-90eb-b2387ede2142-kube-api-access-4n8vf\") pod \"multus-qj9nm\" (UID: \"861b97be-45d1-4aee-90eb-b2387ede2142\") " pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.372705 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.372679    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsk2z\" (UniqueName: \"kubernetes.io/projected/fed3e161-7e00-481a-a172-a751ec55d8cc-kube-api-access-gsk2z\") pod \"aws-ebs-csi-driver-node-ff565\" (UID: \"fed3e161-7e00-481a-a172-a751ec55d8cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff565"
Jul 22 16:07:22.379392 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.379357    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-q2ljm"
Jul 22 16:07:22.418804 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.418759    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-q2ljm"
Jul 22 16:07:22.447688 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.447596    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-224.ec2.internal" event={"ID":"7b422f2f9cb0c4f90f53104ef61aedd3","Type":"ContainerStarted","Data":"2c0d27884a918d9cbe1990f922e6c2d1c984380de0cf3bffe9812ba44f0fd3ec"}
Jul 22 16:07:22.449364 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.449333    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-224.ec2.internal" event={"ID":"3c7981c2b5b06a134fa86d61e672a2e9","Type":"ContainerStarted","Data":"feeb2c9a3169a08dcac88a086325afd88c35a5352aa82d4d6bcc6b689e65c341"}
Jul 22 16:07:22.452319 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.452291    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a2dfe635-d673-4f73-94cc-d852424b78d9-etc-sysctl-conf\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.452438 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.452336    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1cd85e6d-9e7c-4c21-bc75-98cc25975bfc-original-pull-secret\") pod \"global-pull-secret-syncer-5dhg6\" (UID: \"1cd85e6d-9e7c-4c21-bc75-98cc25975bfc\") " pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:07:22.452438 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.452373    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8gn4\" (UniqueName: \"kubernetes.io/projected/2193e465-f388-4298-80aa-c4fd96ec3f2f-kube-api-access-n8gn4\") pod \"node-resolver-xr6pt\" (UID: \"2193e465-f388-4298-80aa-c4fd96ec3f2f\") " pod="openshift-dns/node-resolver-xr6pt"
Jul 22 16:07:22.452438 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.452401    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/64b5ff5e-8328-4e1b-9a8d-341aa0cebba5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xtqkt\" (UID: \"64b5ff5e-8328-4e1b-9a8d-341aa0cebba5\") " pod="openshift-multus/multus-additional-cni-plugins-xtqkt"
Jul 22 16:07:22.452438 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.452429    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a2dfe635-d673-4f73-94cc-d852424b78d9-run\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.452633 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.452460    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-host-run-netns\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.452633 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.452487    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-run-openvswitch\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.452633 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.452495    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a2dfe635-d673-4f73-94cc-d852424b78d9-etc-sysctl-conf\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.452633 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.452513    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a2dfe635-d673-4f73-94cc-d852424b78d9-host\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.452633 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.452514    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-host-run-netns\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.452633 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.452524    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a2dfe635-d673-4f73-94cc-d852424b78d9-run\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.452633 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.452552    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-log-socket\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.452633 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.452581    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-host-cni-netd\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.452633 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:22.452586    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 16:07:22.452633 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.452606    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5e6f48d6-b0c6-4827-a1de-17ec418ccaf2-host\") pod \"node-ca-bzc75\" (UID: \"5e6f48d6-b0c6-4827-a1de-17ec418ccaf2\") " pod="openshift-image-registry/node-ca-bzc75"
Jul 22 16:07:22.452633 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.452616    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-log-socket\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.452633 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.452637    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1cd85e6d-9e7c-4c21-bc75-98cc25975bfc-kubelet-config\") pod \"global-pull-secret-syncer-5dhg6\" (UID: \"1cd85e6d-9e7c-4c21-bc75-98cc25975bfc\") " pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:07:22.453182 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.452640    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-host-cni-netd\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.453182 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.452659    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5e6f48d6-b0c6-4827-a1de-17ec418ccaf2-host\") pod \"node-ca-bzc75\" (UID: \"5e6f48d6-b0c6-4827-a1de-17ec418ccaf2\") " pod="openshift-image-registry/node-ca-bzc75"
Jul 22 16:07:22.453182 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:22.452665    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cd85e6d-9e7c-4c21-bc75-98cc25975bfc-original-pull-secret podName:1cd85e6d-9e7c-4c21-bc75-98cc25975bfc nodeName:}" failed. No retries permitted until 2026-07-22 16:07:22.952646354 +0000 UTC m=+3.083116359 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1cd85e6d-9e7c-4c21-bc75-98cc25975bfc-original-pull-secret") pod "global-pull-secret-syncer-5dhg6" (UID: "1cd85e6d-9e7c-4c21-bc75-98cc25975bfc") : object "kube-system"/"original-pull-secret" not registered
Jul 22 16:07:22.453182 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.452581    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-run-openvswitch\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.453182 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.452683    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1cd85e6d-9e7c-4c21-bc75-98cc25975bfc-kubelet-config\") pod \"global-pull-secret-syncer-5dhg6\" (UID: \"1cd85e6d-9e7c-4c21-bc75-98cc25975bfc\") " pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:07:22.453182 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.452604    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a2dfe635-d673-4f73-94cc-d852424b78d9-host\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.453182 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.452915    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-host-cni-bin\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.453182 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.452948    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a2dfe635-d673-4f73-94cc-d852424b78d9-etc-sysconfig\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.453182 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.452973    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-host-cni-bin\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.453182 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.452978    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-var-lib-openvswitch\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.453182 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453004    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d5ae481a-e39e-4c39-8f8e-e9255121886d-ovn-node-metrics-cert\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.453182 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453016    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-var-lib-openvswitch\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.453182 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453016    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a2dfe635-d673-4f73-94cc-d852424b78d9-etc-sysconfig\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.453182 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453024    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/64b5ff5e-8328-4e1b-9a8d-341aa0cebba5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xtqkt\" (UID: \"64b5ff5e-8328-4e1b-9a8d-341aa0cebba5\") " pod="openshift-multus/multus-additional-cni-plugins-xtqkt"
Jul 22 16:07:22.453829 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453054    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a2dfe635-d673-4f73-94cc-d852424b78d9-etc-sysctl-d\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.453829 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453079    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns8k6\" (UniqueName: \"kubernetes.io/projected/a2dfe635-d673-4f73-94cc-d852424b78d9-kube-api-access-ns8k6\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.453829 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453109    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-host-run-ovn-kubernetes\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.453829 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453134    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s96jr\" (UniqueName: \"kubernetes.io/projected/64b5ff5e-8328-4e1b-9a8d-341aa0cebba5-kube-api-access-s96jr\") pod \"multus-additional-cni-plugins-xtqkt\" (UID: \"64b5ff5e-8328-4e1b-9a8d-341aa0cebba5\") " pod="openshift-multus/multus-additional-cni-plugins-xtqkt"
Jul 22 16:07:22.453829 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453153    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/64b5ff5e-8328-4e1b-9a8d-341aa0cebba5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xtqkt\" (UID: \"64b5ff5e-8328-4e1b-9a8d-341aa0cebba5\") " pod="openshift-multus/multus-additional-cni-plugins-xtqkt"
Jul 22 16:07:22.453829 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453160    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2dfe635-d673-4f73-94cc-d852424b78d9-var-lib-kubelet\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.453829 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453190    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcrwr\" (UniqueName: \"kubernetes.io/projected/662d650d-8f3e-45ce-900d-e3a0ed385342-kube-api-access-tcrwr\") pod \"iptables-alerter-cq5v7\" (UID: \"662d650d-8f3e-45ce-900d-e3a0ed385342\") " pod="openshift-network-operator/iptables-alerter-cq5v7"
Jul 22 16:07:22.453829 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453212    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2dfe635-d673-4f73-94cc-d852424b78d9-var-lib-kubelet\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.453829 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453215    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2dfe635-d673-4f73-94cc-d852424b78d9-etc-kubernetes\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.453829 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453223    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a2dfe635-d673-4f73-94cc-d852424b78d9-etc-sysctl-d\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.453829 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453233    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/64b5ff5e-8328-4e1b-9a8d-341aa0cebba5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xtqkt\" (UID: \"64b5ff5e-8328-4e1b-9a8d-341aa0cebba5\") " pod="openshift-multus/multus-additional-cni-plugins-xtqkt"
Jul 22 16:07:22.453829 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453244    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-systemd-units\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.453829 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453249    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2dfe635-d673-4f73-94cc-d852424b78d9-etc-kubernetes\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.453829 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453275    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d5ae481a-e39e-4c39-8f8e-e9255121886d-ovnkube-config\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.454512 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453293    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-host-run-ovn-kubernetes\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.454512 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453298    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d5ae481a-e39e-4c39-8f8e-e9255121886d-env-overrides\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.454512 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453327    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2dfe635-d673-4f73-94cc-d852424b78d9-sys\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.454512 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453349    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a2dfe635-d673-4f73-94cc-d852424b78d9-etc-tuned\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.454512 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453366    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-systemd-units\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.454512 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453375    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-host-kubelet\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.454512 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453413    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-host-slash\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.454512 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453436    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-host-kubelet\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.454512 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453445    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/64b5ff5e-8328-4e1b-9a8d-341aa0cebba5-cnibin\") pod \"multus-additional-cni-plugins-xtqkt\" (UID: \"64b5ff5e-8328-4e1b-9a8d-341aa0cebba5\") " pod="openshift-multus/multus-additional-cni-plugins-xtqkt"
Jul 22 16:07:22.454512 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453480    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2dfe635-d673-4f73-94cc-d852424b78d9-sys\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.454512 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453492    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/64b5ff5e-8328-4e1b-9a8d-341aa0cebba5-cnibin\") pod \"multus-additional-cni-plugins-xtqkt\" (UID: \"64b5ff5e-8328-4e1b-9a8d-341aa0cebba5\") " pod="openshift-multus/multus-additional-cni-plugins-xtqkt"
Jul 22 16:07:22.454512 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453513    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-host-slash\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.454512 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453554    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5e6f48d6-b0c6-4827-a1de-17ec418ccaf2-serviceca\") pod \"node-ca-bzc75\" (UID: \"5e6f48d6-b0c6-4827-a1de-17ec418ccaf2\") " pod="openshift-image-registry/node-ca-bzc75"
Jul 22 16:07:22.454512 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453600    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l2kr\" (UniqueName: \"kubernetes.io/projected/5e6f48d6-b0c6-4827-a1de-17ec418ccaf2-kube-api-access-9l2kr\") pod \"node-ca-bzc75\" (UID: \"5e6f48d6-b0c6-4827-a1de-17ec418ccaf2\") " pod="openshift-image-registry/node-ca-bzc75"
Jul 22 16:07:22.454512 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453631    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/64b5ff5e-8328-4e1b-9a8d-341aa0cebba5-cni-binary-copy\") pod \"multus-additional-cni-plugins-xtqkt\" (UID: \"64b5ff5e-8328-4e1b-9a8d-341aa0cebba5\") " pod="openshift-multus/multus-additional-cni-plugins-xtqkt"
Jul 22 16:07:22.455278 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453659    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a2dfe635-d673-4f73-94cc-d852424b78d9-lib-modules\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.455278 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453687    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.455278 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453715    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22kgp\" (UniqueName: \"kubernetes.io/projected/d5ae481a-e39e-4c39-8f8e-e9255121886d-kube-api-access-22kgp\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.455278 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453739    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/64b5ff5e-8328-4e1b-9a8d-341aa0cebba5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xtqkt\" (UID: \"64b5ff5e-8328-4e1b-9a8d-341aa0cebba5\") " pod="openshift-multus/multus-additional-cni-plugins-xtqkt"
Jul 22 16:07:22.455278 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453743    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d5ae481a-e39e-4c39-8f8e-e9255121886d-env-overrides\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.455278 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453793    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6440de1a-aff3-4b9d-a374-4e029ece2bfe-metrics-certs\") pod \"network-metrics-daemon-87ltq\" (UID: \"6440de1a-aff3-4b9d-a374-4e029ece2bfe\") " pod="openshift-multus/network-metrics-daemon-87ltq"
Jul 22 16:07:22.455278 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453819    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1cd85e6d-9e7c-4c21-bc75-98cc25975bfc-dbus\") pod \"global-pull-secret-syncer-5dhg6\" (UID: \"1cd85e6d-9e7c-4c21-bc75-98cc25975bfc\") " pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:07:22.455278 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453844    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-etc-openvswitch\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.455278 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453887    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tcrz\" (UniqueName: \"kubernetes.io/projected/6440de1a-aff3-4b9d-a374-4e029ece2bfe-kube-api-access-4tcrz\") pod \"network-metrics-daemon-87ltq\" (UID: \"6440de1a-aff3-4b9d-a374-4e029ece2bfe\") " pod="openshift-multus/network-metrics-daemon-87ltq"
Jul 22 16:07:22.455278 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453895    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a2dfe635-d673-4f73-94cc-d852424b78d9-lib-modules\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.455278 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453914    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-run-ovn\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.455278 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453940    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2193e465-f388-4298-80aa-c4fd96ec3f2f-tmp-dir\") pod \"node-resolver-xr6pt\" (UID: \"2193e465-f388-4298-80aa-c4fd96ec3f2f\") " pod="openshift-dns/node-resolver-xr6pt"
Jul 22 16:07:22.455278 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453965    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/64b5ff5e-8328-4e1b-9a8d-341aa0cebba5-os-release\") pod \"multus-additional-cni-plugins-xtqkt\" (UID: \"64b5ff5e-8328-4e1b-9a8d-341aa0cebba5\") " pod="openshift-multus/multus-additional-cni-plugins-xtqkt"
Jul 22 16:07:22.455278 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453989    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2193e465-f388-4298-80aa-c4fd96ec3f2f-hosts-file\") pod \"node-resolver-xr6pt\" (UID: \"2193e465-f388-4298-80aa-c4fd96ec3f2f\") " pod="openshift-dns/node-resolver-xr6pt"
Jul 22 16:07:22.455974 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.454006    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5e6f48d6-b0c6-4827-a1de-17ec418ccaf2-serviceca\") pod \"node-ca-bzc75\" (UID: \"5e6f48d6-b0c6-4827-a1de-17ec418ccaf2\") " pod="openshift-image-registry/node-ca-bzc75"
Jul 22 16:07:22.455974 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.454025    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/662d650d-8f3e-45ce-900d-e3a0ed385342-host-slash\") pod \"iptables-alerter-cq5v7\" (UID: \"662d650d-8f3e-45ce-900d-e3a0ed385342\") " pod="openshift-network-operator/iptables-alerter-cq5v7"
Jul 22 16:07:22.455974 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.454050    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1cd85e6d-9e7c-4c21-bc75-98cc25975bfc-dbus\") pod \"global-pull-secret-syncer-5dhg6\" (UID: \"1cd85e6d-9e7c-4c21-bc75-98cc25975bfc\") " pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:07:22.455974 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.454051    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64b5ff5e-8328-4e1b-9a8d-341aa0cebba5-system-cni-dir\") pod \"multus-additional-cni-plugins-xtqkt\" (UID: \"64b5ff5e-8328-4e1b-9a8d-341aa0cebba5\") " pod="openshift-multus/multus-additional-cni-plugins-xtqkt"
Jul 22 16:07:22.455974 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.454098    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64b5ff5e-8328-4e1b-9a8d-341aa0cebba5-system-cni-dir\") pod \"multus-additional-cni-plugins-xtqkt\" (UID: \"64b5ff5e-8328-4e1b-9a8d-341aa0cebba5\") " pod="openshift-multus/multus-additional-cni-plugins-xtqkt"
Jul 22 16:07:22.455974 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.454106    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a2dfe635-d673-4f73-94cc-d852424b78d9-etc-modprobe-d\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.455974 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.454137    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-run-systemd\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.455974 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.454170    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-node-log\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.455974 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.454199    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a2dfe635-d673-4f73-94cc-d852424b78d9-etc-modprobe-d\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.455974 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.454228    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d5ae481a-e39e-4c39-8f8e-e9255121886d-ovnkube-script-lib\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.455974 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.454244    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.455974 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.454266    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2193e465-f388-4298-80aa-c4fd96ec3f2f-tmp-dir\") pod \"node-resolver-xr6pt\" (UID: \"2193e465-f388-4298-80aa-c4fd96ec3f2f\") " pod="openshift-dns/node-resolver-xr6pt"
Jul 22 16:07:22.455974 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.454262    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/662d650d-8f3e-45ce-900d-e3a0ed385342-iptables-alerter-script\") pod \"iptables-alerter-cq5v7\" (UID: \"662d650d-8f3e-45ce-900d-e3a0ed385342\") " pod="openshift-network-operator/iptables-alerter-cq5v7"
Jul 22 16:07:22.455974 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.454304    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a2dfe635-d673-4f73-94cc-d852424b78d9-etc-systemd\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.456417 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.454344    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a2dfe635-d673-4f73-94cc-d852424b78d9-tmp\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.456417 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.454361    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a2dfe635-d673-4f73-94cc-d852424b78d9-etc-systemd\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.456417 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.453989    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d5ae481a-e39e-4c39-8f8e-e9255121886d-ovnkube-config\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.456417 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.454435    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2193e465-f388-4298-80aa-c4fd96ec3f2f-hosts-file\") pod \"node-resolver-xr6pt\" (UID: \"2193e465-f388-4298-80aa-c4fd96ec3f2f\") " pod="openshift-dns/node-resolver-xr6pt"
Jul 22 16:07:22.456417 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.454436    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-etc-openvswitch\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.456417 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.454482    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/64b5ff5e-8328-4e1b-9a8d-341aa0cebba5-cni-binary-copy\") pod \"multus-additional-cni-plugins-xtqkt\" (UID: \"64b5ff5e-8328-4e1b-9a8d-341aa0cebba5\") " pod="openshift-multus/multus-additional-cni-plugins-xtqkt"
Jul 22 16:07:22.456417 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.454635    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/662d650d-8f3e-45ce-900d-e3a0ed385342-host-slash\") pod \"iptables-alerter-cq5v7\" (UID: \"662d650d-8f3e-45ce-900d-e3a0ed385342\") " pod="openshift-network-operator/iptables-alerter-cq5v7"
Jul 22 16:07:22.456417 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.454641    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-run-ovn\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.456417 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:22.454672    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 16:07:22.456417 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.454706    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/64b5ff5e-8328-4e1b-9a8d-341aa0cebba5-os-release\") pod \"multus-additional-cni-plugins-xtqkt\" (UID: \"64b5ff5e-8328-4e1b-9a8d-341aa0cebba5\") " pod="openshift-multus/multus-additional-cni-plugins-xtqkt"
Jul 22 16:07:22.456417 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:22.454731    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6440de1a-aff3-4b9d-a374-4e029ece2bfe-metrics-certs podName:6440de1a-aff3-4b9d-a374-4e029ece2bfe nodeName:}" failed. No retries permitted until 2026-07-22 16:07:22.954713746 +0000 UTC m=+3.085183729 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6440de1a-aff3-4b9d-a374-4e029ece2bfe-metrics-certs") pod "network-metrics-daemon-87ltq" (UID: "6440de1a-aff3-4b9d-a374-4e029ece2bfe") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 16:07:22.456417 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.454778    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-node-log\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.456417 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.454838    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d5ae481a-e39e-4c39-8f8e-e9255121886d-run-systemd\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.456417 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.455270    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/64b5ff5e-8328-4e1b-9a8d-341aa0cebba5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xtqkt\" (UID: \"64b5ff5e-8328-4e1b-9a8d-341aa0cebba5\") " pod="openshift-multus/multus-additional-cni-plugins-xtqkt"
Jul 22 16:07:22.457122 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.455329    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/662d650d-8f3e-45ce-900d-e3a0ed385342-iptables-alerter-script\") pod \"iptables-alerter-cq5v7\" (UID: \"662d650d-8f3e-45ce-900d-e3a0ed385342\") " pod="openshift-network-operator/iptables-alerter-cq5v7"
Jul 22 16:07:22.457122 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.455343    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d5ae481a-e39e-4c39-8f8e-e9255121886d-ovnkube-script-lib\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.457122 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.456033    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a2dfe635-d673-4f73-94cc-d852424b78d9-etc-tuned\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.457122 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.456467    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d5ae481a-e39e-4c39-8f8e-e9255121886d-ovn-node-metrics-cert\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.457371 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.457259    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a2dfe635-d673-4f73-94cc-d852424b78d9-tmp\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.470144 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.470065    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8gn4\" (UniqueName: \"kubernetes.io/projected/2193e465-f388-4298-80aa-c4fd96ec3f2f-kube-api-access-n8gn4\") pod \"node-resolver-xr6pt\" (UID: \"2193e465-f388-4298-80aa-c4fd96ec3f2f\") " pod="openshift-dns/node-resolver-xr6pt"
Jul 22 16:07:22.470710 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.470687    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9l2kr\" (UniqueName: \"kubernetes.io/projected/5e6f48d6-b0c6-4827-a1de-17ec418ccaf2-kube-api-access-9l2kr\") pod \"node-ca-bzc75\" (UID: \"5e6f48d6-b0c6-4827-a1de-17ec418ccaf2\") " pod="openshift-image-registry/node-ca-bzc75"
Jul 22 16:07:22.471055 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.471013    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns8k6\" (UniqueName: \"kubernetes.io/projected/a2dfe635-d673-4f73-94cc-d852424b78d9-kube-api-access-ns8k6\") pod \"tuned-85sw4\" (UID: \"a2dfe635-d673-4f73-94cc-d852424b78d9\") " pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.471494 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.471472    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tcrz\" (UniqueName: \"kubernetes.io/projected/6440de1a-aff3-4b9d-a374-4e029ece2bfe-kube-api-access-4tcrz\") pod \"network-metrics-daemon-87ltq\" (UID: \"6440de1a-aff3-4b9d-a374-4e029ece2bfe\") " pod="openshift-multus/network-metrics-daemon-87ltq"
Jul 22 16:07:22.471563 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.471539    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcrwr\" (UniqueName: \"kubernetes.io/projected/662d650d-8f3e-45ce-900d-e3a0ed385342-kube-api-access-tcrwr\") pod \"iptables-alerter-cq5v7\" (UID: \"662d650d-8f3e-45ce-900d-e3a0ed385342\") " pod="openshift-network-operator/iptables-alerter-cq5v7"
Jul 22 16:07:22.472168 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.472150    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s96jr\" (UniqueName: \"kubernetes.io/projected/64b5ff5e-8328-4e1b-9a8d-341aa0cebba5-kube-api-access-s96jr\") pod \"multus-additional-cni-plugins-xtqkt\" (UID: \"64b5ff5e-8328-4e1b-9a8d-341aa0cebba5\") " pod="openshift-multus/multus-additional-cni-plugins-xtqkt"
Jul 22 16:07:22.472455 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.472434    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22kgp\" (UniqueName: \"kubernetes.io/projected/d5ae481a-e39e-4c39-8f8e-e9255121886d-kube-api-access-22kgp\") pod \"ovnkube-node-2djqd\" (UID: \"d5ae481a-e39e-4c39-8f8e-e9255121886d\") " pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.543447 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.543365    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff565"
Jul 22 16:07:22.551904 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.551872    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qj9nm"
Jul 22 16:07:22.564181 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.564041    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6b2b2"
Jul 22 16:07:22.572927 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.572897    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xr6pt"
Jul 22 16:07:22.589215 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.589192    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cq5v7"
Jul 22 16:07:22.596933 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.596911    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xtqkt"
Jul 22 16:07:22.605316 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.605298    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:22.614071 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.614045    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-85sw4"
Jul 22 16:07:22.621864 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.621829    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bzc75"
Jul 22 16:07:22.759836 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.759803    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 16:07:22.857996 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.857914    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g44rx\" (UniqueName: \"kubernetes.io/projected/36f0cd2c-f5c3-47ce-bb45-0f604e33aed9-kube-api-access-g44rx\") pod \"network-check-target-kq5l6\" (UID: \"36f0cd2c-f5c3-47ce-bb45-0f604e33aed9\") " pod="openshift-network-diagnostics/network-check-target-kq5l6"
Jul 22 16:07:22.858157 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:22.858097    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 16:07:22.858157 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:22.858117    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 16:07:22.858157 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:22.858126    2563 projected.go:196] Error preparing data for projected volume kube-api-access-g44rx for pod openshift-network-diagnostics/network-check-target-kq5l6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 16:07:22.858286 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:22.858179    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/36f0cd2c-f5c3-47ce-bb45-0f604e33aed9-kube-api-access-g44rx podName:36f0cd2c-f5c3-47ce-bb45-0f604e33aed9 nodeName:}" failed. No retries permitted until 2026-07-22 16:07:23.858166556 +0000 UTC m=+3.988636536 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-g44rx" (UniqueName: "kubernetes.io/projected/36f0cd2c-f5c3-47ce-bb45-0f604e33aed9-kube-api-access-g44rx") pod "network-check-target-kq5l6" (UID: "36f0cd2c-f5c3-47ce-bb45-0f604e33aed9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 16:07:22.958609 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.958571    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6440de1a-aff3-4b9d-a374-4e029ece2bfe-metrics-certs\") pod \"network-metrics-daemon-87ltq\" (UID: \"6440de1a-aff3-4b9d-a374-4e029ece2bfe\") " pod="openshift-multus/network-metrics-daemon-87ltq"
Jul 22 16:07:22.958780 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:22.958635    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1cd85e6d-9e7c-4c21-bc75-98cc25975bfc-original-pull-secret\") pod \"global-pull-secret-syncer-5dhg6\" (UID: \"1cd85e6d-9e7c-4c21-bc75-98cc25975bfc\") " pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:07:22.958780 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:22.958743    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 16:07:22.958938 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:22.958829    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6440de1a-aff3-4b9d-a374-4e029ece2bfe-metrics-certs podName:6440de1a-aff3-4b9d-a374-4e029ece2bfe nodeName:}" failed. No retries permitted until 2026-07-22 16:07:23.958807909 +0000 UTC m=+4.089277907 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6440de1a-aff3-4b9d-a374-4e029ece2bfe-metrics-certs") pod "network-metrics-daemon-87ltq" (UID: "6440de1a-aff3-4b9d-a374-4e029ece2bfe") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 16:07:22.958938 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:22.958743    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 16:07:22.958938 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:22.958912    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cd85e6d-9e7c-4c21-bc75-98cc25975bfc-original-pull-secret podName:1cd85e6d-9e7c-4c21-bc75-98cc25975bfc nodeName:}" failed. No retries permitted until 2026-07-22 16:07:23.958898672 +0000 UTC m=+4.089368657 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1cd85e6d-9e7c-4c21-bc75-98cc25975bfc-original-pull-secret") pod "global-pull-secret-syncer-5dhg6" (UID: "1cd85e6d-9e7c-4c21-bc75-98cc25975bfc") : object "kube-system"/"original-pull-secret" not registered
Jul 22 16:07:23.422192 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:23.422141    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 16:02:22 +0000 UTC" deadline="2028-01-29 18:11:07.3689437 +0000 UTC"
Jul 22 16:07:23.422192 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:23.422177    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13346h3m43.946771099s"
Jul 22 16:07:23.442554 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:23.442523    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ltq"
Jul 22 16:07:23.442713 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:23.442672    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87ltq" podUID="6440de1a-aff3-4b9d-a374-4e029ece2bfe"
Jul 22 16:07:23.483309 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:23.483272    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e6f48d6_b0c6_4827_a1de_17ec418ccaf2.slice/crio-c403fb6c8c07b7694e62473d72fdc80a716b0b080e96ae05e0f7673ae29003a0 WatchSource:0}: Error finding container c403fb6c8c07b7694e62473d72fdc80a716b0b080e96ae05e0f7673ae29003a0: Status 404 returned error can't find the container with id c403fb6c8c07b7694e62473d72fdc80a716b0b080e96ae05e0f7673ae29003a0
Jul 22 16:07:23.483812 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:23.483788    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6d03e93_ac5d_4b1a_87a2_2c3b2358dcac.slice/crio-b54645987de6e4d49a1fdbdde8a64c0b012052602237b7b8d6b6d3e7e0b59808 WatchSource:0}: Error finding container b54645987de6e4d49a1fdbdde8a64c0b012052602237b7b8d6b6d3e7e0b59808: Status 404 returned error can't find the container with id b54645987de6e4d49a1fdbdde8a64c0b012052602237b7b8d6b6d3e7e0b59808
Jul 22 16:07:23.485095 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:23.485075    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64b5ff5e_8328_4e1b_9a8d_341aa0cebba5.slice/crio-be6484efb1a9b1008fe5b95bf67b4f59f74901beb8e4eefa4f7ccc3912204924 WatchSource:0}: Error finding container be6484efb1a9b1008fe5b95bf67b4f59f74901beb8e4eefa4f7ccc3912204924: Status 404 returned error can't find the container with id be6484efb1a9b1008fe5b95bf67b4f59f74901beb8e4eefa4f7ccc3912204924
Jul 22 16:07:23.486787 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:23.486704    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2193e465_f388_4298_80aa_c4fd96ec3f2f.slice/crio-34927c9b8bc010b793c143052c7d32e14ff35f805b601436211a9812fa1e6789 WatchSource:0}: Error finding container 34927c9b8bc010b793c143052c7d32e14ff35f805b601436211a9812fa1e6789: Status 404 returned error can't find the container with id 34927c9b8bc010b793c143052c7d32e14ff35f805b601436211a9812fa1e6789
Jul 22 16:07:23.489726 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:23.489707    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod861b97be_45d1_4aee_90eb_b2387ede2142.slice/crio-62b66dc757984983144bcc4d46c04e6ad9b839d73a614eb92ebeda23785ca4bd WatchSource:0}: Error finding container 62b66dc757984983144bcc4d46c04e6ad9b839d73a614eb92ebeda23785ca4bd: Status 404 returned error can't find the container with id 62b66dc757984983144bcc4d46c04e6ad9b839d73a614eb92ebeda23785ca4bd
Jul 22 16:07:23.490576 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:23.490544    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5ae481a_e39e_4c39_8f8e_e9255121886d.slice/crio-71118e6595e090dd8fb0542eeb537ed2f85875b7d38d6f1fe35d7b6b88251e74 WatchSource:0}: Error finding container 71118e6595e090dd8fb0542eeb537ed2f85875b7d38d6f1fe35d7b6b88251e74: Status 404 returned error can't find the container with id 71118e6595e090dd8fb0542eeb537ed2f85875b7d38d6f1fe35d7b6b88251e74
Jul 22 16:07:23.492309 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:23.492165    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod662d650d_8f3e_45ce_900d_e3a0ed385342.slice/crio-7bbb88dac80c4a1599e3ae91b8a435ff9950f5b32ec574cb9201af4cf88aba86 WatchSource:0}: Error finding container 7bbb88dac80c4a1599e3ae91b8a435ff9950f5b32ec574cb9201af4cf88aba86: Status 404 returned error can't find the container with id 7bbb88dac80c4a1599e3ae91b8a435ff9950f5b32ec574cb9201af4cf88aba86
Jul 22 16:07:23.493572 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:23.493551    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2dfe635_d673_4f73_94cc_d852424b78d9.slice/crio-f399fe8cb948abf639ffdb819eb7ef21c726d1301da12610c16cd82fb8c87939 WatchSource:0}: Error finding container f399fe8cb948abf639ffdb819eb7ef21c726d1301da12610c16cd82fb8c87939: Status 404 returned error can't find the container with id f399fe8cb948abf639ffdb819eb7ef21c726d1301da12610c16cd82fb8c87939
Jul 22 16:07:23.495029 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:23.495012    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfed3e161_7e00_481a_a172_a751ec55d8cc.slice/crio-c8b926726c97694469aca5175598aa06ee876af3c20c7f97f3692883d10ffeb2 WatchSource:0}: Error finding container c8b926726c97694469aca5175598aa06ee876af3c20c7f97f3692883d10ffeb2: Status 404 returned error can't find the container with id c8b926726c97694469aca5175598aa06ee876af3c20c7f97f3692883d10ffeb2
Jul 22 16:07:23.865777 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:23.865435    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g44rx\" (UniqueName: \"kubernetes.io/projected/36f0cd2c-f5c3-47ce-bb45-0f604e33aed9-kube-api-access-g44rx\") pod \"network-check-target-kq5l6\" (UID: \"36f0cd2c-f5c3-47ce-bb45-0f604e33aed9\") " pod="openshift-network-diagnostics/network-check-target-kq5l6"
Jul 22 16:07:23.865977 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:23.865626    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 16:07:23.865977 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:23.865907    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 16:07:23.865977 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:23.865922    2563 projected.go:196] Error preparing data for projected volume kube-api-access-g44rx for pod openshift-network-diagnostics/network-check-target-kq5l6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 16:07:23.866128 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:23.865988    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/36f0cd2c-f5c3-47ce-bb45-0f604e33aed9-kube-api-access-g44rx podName:36f0cd2c-f5c3-47ce-bb45-0f604e33aed9 nodeName:}" failed. No retries permitted until 2026-07-22 16:07:25.865970035 +0000 UTC m=+5.996440031 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-g44rx" (UniqueName: "kubernetes.io/projected/36f0cd2c-f5c3-47ce-bb45-0f604e33aed9-kube-api-access-g44rx") pod "network-check-target-kq5l6" (UID: "36f0cd2c-f5c3-47ce-bb45-0f604e33aed9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 16:07:23.966705 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:23.966304    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6440de1a-aff3-4b9d-a374-4e029ece2bfe-metrics-certs\") pod \"network-metrics-daemon-87ltq\" (UID: \"6440de1a-aff3-4b9d-a374-4e029ece2bfe\") " pod="openshift-multus/network-metrics-daemon-87ltq"
Jul 22 16:07:23.966705 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:23.966362    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1cd85e6d-9e7c-4c21-bc75-98cc25975bfc-original-pull-secret\") pod \"global-pull-secret-syncer-5dhg6\" (UID: \"1cd85e6d-9e7c-4c21-bc75-98cc25975bfc\") " pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:07:23.966705 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:23.966505    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 16:07:23.966705 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:23.966507    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 16:07:23.966705 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:23.966571    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cd85e6d-9e7c-4c21-bc75-98cc25975bfc-original-pull-secret podName:1cd85e6d-9e7c-4c21-bc75-98cc25975bfc nodeName:}" failed. No retries permitted until 2026-07-22 16:07:25.966551937 +0000 UTC m=+6.097021940 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1cd85e6d-9e7c-4c21-bc75-98cc25975bfc-original-pull-secret") pod "global-pull-secret-syncer-5dhg6" (UID: "1cd85e6d-9e7c-4c21-bc75-98cc25975bfc") : object "kube-system"/"original-pull-secret" not registered
Jul 22 16:07:23.966705 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:23.966592    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6440de1a-aff3-4b9d-a374-4e029ece2bfe-metrics-certs podName:6440de1a-aff3-4b9d-a374-4e029ece2bfe nodeName:}" failed. No retries permitted until 2026-07-22 16:07:25.966580919 +0000 UTC m=+6.097050904 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6440de1a-aff3-4b9d-a374-4e029ece2bfe-metrics-certs") pod "network-metrics-daemon-87ltq" (UID: "6440de1a-aff3-4b9d-a374-4e029ece2bfe") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 16:07:24.423216 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:24.423173    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 16:02:22 +0000 UTC" deadline="2028-04-09 04:18:21.479561408 +0000 UTC"
Jul 22 16:07:24.423216 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:24.423211    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15036h10m57.056355435s"
Jul 22 16:07:24.444701 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:24.444671    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:07:24.444872 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:24.444804    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dhg6" podUID="1cd85e6d-9e7c-4c21-bc75-98cc25975bfc"
Jul 22 16:07:24.445240 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:24.445212    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kq5l6"
Jul 22 16:07:24.445358 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:24.445311    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kq5l6" podUID="36f0cd2c-f5c3-47ce-bb45-0f604e33aed9"
Jul 22 16:07:24.469356 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:24.469312    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-224.ec2.internal" event={"ID":"7b422f2f9cb0c4f90f53104ef61aedd3","Type":"ContainerStarted","Data":"1fd69c72f4d27aa347a1716c7d5ba0c0e23e95f7bbb9f359b7caa28b404ff51c"}
Jul 22 16:07:24.495026 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:24.494991    2563 generic.go:356] "Generic (PLEG): container finished" podID="3c7981c2b5b06a134fa86d61e672a2e9" containerID="65f599726873bdae59155ee42b52554f92aa9bf8753a97c1dc40f51b0f75fccc" exitCode=0
Jul 22 16:07:24.495244 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:24.495093    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-224.ec2.internal" event={"ID":"3c7981c2b5b06a134fa86d61e672a2e9","Type":"ContainerDied","Data":"65f599726873bdae59155ee42b52554f92aa9bf8753a97c1dc40f51b0f75fccc"}
Jul 22 16:07:24.497256 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:24.497228    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6b2b2" event={"ID":"b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac","Type":"ContainerStarted","Data":"b54645987de6e4d49a1fdbdde8a64c0b012052602237b7b8d6b6d3e7e0b59808"}
Jul 22 16:07:24.514299 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:24.514084    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xtqkt" event={"ID":"64b5ff5e-8328-4e1b-9a8d-341aa0cebba5","Type":"ContainerStarted","Data":"be6484efb1a9b1008fe5b95bf67b4f59f74901beb8e4eefa4f7ccc3912204924"}
Jul 22 16:07:24.524148 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:24.524077    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-224.ec2.internal" podStartSLOduration=3.524035226 podStartE2EDuration="3.524035226s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:07:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:07:24.49008281 +0000 UTC m=+4.620552826" watchObservedRunningTime="2026-07-22 16:07:24.524035226 +0000 UTC m=+4.654505231"
Jul 22 16:07:24.536689 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:24.536606    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2djqd" event={"ID":"d5ae481a-e39e-4c39-8f8e-e9255121886d","Type":"ContainerStarted","Data":"71118e6595e090dd8fb0542eeb537ed2f85875b7d38d6f1fe35d7b6b88251e74"}
Jul 22 16:07:24.549692 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:24.549658    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff565" event={"ID":"fed3e161-7e00-481a-a172-a751ec55d8cc","Type":"ContainerStarted","Data":"c8b926726c97694469aca5175598aa06ee876af3c20c7f97f3692883d10ffeb2"}
Jul 22 16:07:24.568996 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:24.568961    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-85sw4" event={"ID":"a2dfe635-d673-4f73-94cc-d852424b78d9","Type":"ContainerStarted","Data":"f399fe8cb948abf639ffdb819eb7ef21c726d1301da12610c16cd82fb8c87939"}
Jul 22 16:07:24.586569 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:24.584882    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cq5v7" event={"ID":"662d650d-8f3e-45ce-900d-e3a0ed385342","Type":"ContainerStarted","Data":"7bbb88dac80c4a1599e3ae91b8a435ff9950f5b32ec574cb9201af4cf88aba86"}
Jul 22 16:07:24.591845 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:24.591776    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qj9nm" event={"ID":"861b97be-45d1-4aee-90eb-b2387ede2142","Type":"ContainerStarted","Data":"62b66dc757984983144bcc4d46c04e6ad9b839d73a614eb92ebeda23785ca4bd"}
Jul 22 16:07:24.604684 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:24.604516    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xr6pt" event={"ID":"2193e465-f388-4298-80aa-c4fd96ec3f2f","Type":"ContainerStarted","Data":"34927c9b8bc010b793c143052c7d32e14ff35f805b601436211a9812fa1e6789"}
Jul 22 16:07:24.621124 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:24.621089    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bzc75" event={"ID":"5e6f48d6-b0c6-4827-a1de-17ec418ccaf2","Type":"ContainerStarted","Data":"c403fb6c8c07b7694e62473d72fdc80a716b0b080e96ae05e0f7673ae29003a0"}
Jul 22 16:07:25.442905 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:25.442379    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ltq"
Jul 22 16:07:25.442905 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:25.442542    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87ltq" podUID="6440de1a-aff3-4b9d-a374-4e029ece2bfe"
Jul 22 16:07:25.656918 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:25.656778    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-224.ec2.internal" event={"ID":"3c7981c2b5b06a134fa86d61e672a2e9","Type":"ContainerStarted","Data":"e4cc88b613c2d631fbb21e14e55914d5e823efcda0ec45c65b35432ea3160795"}
Jul 22 16:07:25.885345 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:25.885252    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g44rx\" (UniqueName: \"kubernetes.io/projected/36f0cd2c-f5c3-47ce-bb45-0f604e33aed9-kube-api-access-g44rx\") pod \"network-check-target-kq5l6\" (UID: \"36f0cd2c-f5c3-47ce-bb45-0f604e33aed9\") " pod="openshift-network-diagnostics/network-check-target-kq5l6"
Jul 22 16:07:25.885507 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:25.885493    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 16:07:25.885575 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:25.885515    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 16:07:25.885575 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:25.885529    2563 projected.go:196] Error preparing data for projected volume kube-api-access-g44rx for pod openshift-network-diagnostics/network-check-target-kq5l6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 16:07:25.885673 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:25.885589    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/36f0cd2c-f5c3-47ce-bb45-0f604e33aed9-kube-api-access-g44rx podName:36f0cd2c-f5c3-47ce-bb45-0f604e33aed9 nodeName:}" failed. No retries permitted until 2026-07-22 16:07:29.885571785 +0000 UTC m=+10.016041778 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-g44rx" (UniqueName: "kubernetes.io/projected/36f0cd2c-f5c3-47ce-bb45-0f604e33aed9-kube-api-access-g44rx") pod "network-check-target-kq5l6" (UID: "36f0cd2c-f5c3-47ce-bb45-0f604e33aed9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 16:07:25.986051 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:25.985787    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6440de1a-aff3-4b9d-a374-4e029ece2bfe-metrics-certs\") pod \"network-metrics-daemon-87ltq\" (UID: \"6440de1a-aff3-4b9d-a374-4e029ece2bfe\") " pod="openshift-multus/network-metrics-daemon-87ltq"
Jul 22 16:07:25.986051 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:25.985846    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1cd85e6d-9e7c-4c21-bc75-98cc25975bfc-original-pull-secret\") pod \"global-pull-secret-syncer-5dhg6\" (UID: \"1cd85e6d-9e7c-4c21-bc75-98cc25975bfc\") " pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:07:25.986051 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:25.986003    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 16:07:25.986051 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:25.986059    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cd85e6d-9e7c-4c21-bc75-98cc25975bfc-original-pull-secret podName:1cd85e6d-9e7c-4c21-bc75-98cc25975bfc nodeName:}" failed. No retries permitted until 2026-07-22 16:07:29.986043705 +0000 UTC m=+10.116513707 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1cd85e6d-9e7c-4c21-bc75-98cc25975bfc-original-pull-secret") pod "global-pull-secret-syncer-5dhg6" (UID: "1cd85e6d-9e7c-4c21-bc75-98cc25975bfc") : object "kube-system"/"original-pull-secret" not registered
Jul 22 16:07:25.986665 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:25.986439    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 16:07:25.986665 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:25.986494    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6440de1a-aff3-4b9d-a374-4e029ece2bfe-metrics-certs podName:6440de1a-aff3-4b9d-a374-4e029ece2bfe nodeName:}" failed. No retries permitted until 2026-07-22 16:07:29.986477248 +0000 UTC m=+10.116947236 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6440de1a-aff3-4b9d-a374-4e029ece2bfe-metrics-certs") pod "network-metrics-daemon-87ltq" (UID: "6440de1a-aff3-4b9d-a374-4e029ece2bfe") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 16:07:26.445742 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:26.445078    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:07:26.445742 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:26.445221    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dhg6" podUID="1cd85e6d-9e7c-4c21-bc75-98cc25975bfc"
Jul 22 16:07:26.445742 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:26.445301    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kq5l6"
Jul 22 16:07:26.445742 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:26.445370    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kq5l6" podUID="36f0cd2c-f5c3-47ce-bb45-0f604e33aed9"
Jul 22 16:07:27.443166 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:27.443128    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ltq"
Jul 22 16:07:27.443330 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:27.443285    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87ltq" podUID="6440de1a-aff3-4b9d-a374-4e029ece2bfe"
Jul 22 16:07:28.442927 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:28.442573    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:07:28.442927 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:28.442721    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dhg6" podUID="1cd85e6d-9e7c-4c21-bc75-98cc25975bfc"
Jul 22 16:07:28.442927 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:28.442785    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kq5l6"
Jul 22 16:07:28.442927 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:28.442905    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kq5l6" podUID="36f0cd2c-f5c3-47ce-bb45-0f604e33aed9"
Jul 22 16:07:29.442716 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:29.442678    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ltq"
Jul 22 16:07:29.442911 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:29.442820    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87ltq" podUID="6440de1a-aff3-4b9d-a374-4e029ece2bfe"
Jul 22 16:07:29.916163 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:29.916075    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g44rx\" (UniqueName: \"kubernetes.io/projected/36f0cd2c-f5c3-47ce-bb45-0f604e33aed9-kube-api-access-g44rx\") pod \"network-check-target-kq5l6\" (UID: \"36f0cd2c-f5c3-47ce-bb45-0f604e33aed9\") " pod="openshift-network-diagnostics/network-check-target-kq5l6"
Jul 22 16:07:29.916600 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:29.916297    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 16:07:29.916600 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:29.916320    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 16:07:29.916600 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:29.916334    2563 projected.go:196] Error preparing data for projected volume kube-api-access-g44rx for pod openshift-network-diagnostics/network-check-target-kq5l6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 16:07:29.916600 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:29.916399    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/36f0cd2c-f5c3-47ce-bb45-0f604e33aed9-kube-api-access-g44rx podName:36f0cd2c-f5c3-47ce-bb45-0f604e33aed9 nodeName:}" failed. No retries permitted until 2026-07-22 16:07:37.916380802 +0000 UTC m=+18.046850787 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-g44rx" (UniqueName: "kubernetes.io/projected/36f0cd2c-f5c3-47ce-bb45-0f604e33aed9-kube-api-access-g44rx") pod "network-check-target-kq5l6" (UID: "36f0cd2c-f5c3-47ce-bb45-0f604e33aed9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 16:07:30.017145 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:30.017105    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6440de1a-aff3-4b9d-a374-4e029ece2bfe-metrics-certs\") pod \"network-metrics-daemon-87ltq\" (UID: \"6440de1a-aff3-4b9d-a374-4e029ece2bfe\") " pod="openshift-multus/network-metrics-daemon-87ltq"
Jul 22 16:07:30.017314 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:30.017160    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1cd85e6d-9e7c-4c21-bc75-98cc25975bfc-original-pull-secret\") pod \"global-pull-secret-syncer-5dhg6\" (UID: \"1cd85e6d-9e7c-4c21-bc75-98cc25975bfc\") " pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:07:30.017314 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:30.017262    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 16:07:30.017314 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:30.017278    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 16:07:30.017492 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:30.017326    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cd85e6d-9e7c-4c21-bc75-98cc25975bfc-original-pull-secret podName:1cd85e6d-9e7c-4c21-bc75-98cc25975bfc nodeName:}" failed. No retries permitted until 2026-07-22 16:07:38.017307512 +0000 UTC m=+18.147777512 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1cd85e6d-9e7c-4c21-bc75-98cc25975bfc-original-pull-secret") pod "global-pull-secret-syncer-5dhg6" (UID: "1cd85e6d-9e7c-4c21-bc75-98cc25975bfc") : object "kube-system"/"original-pull-secret" not registered
Jul 22 16:07:30.017492 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:30.017349    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6440de1a-aff3-4b9d-a374-4e029ece2bfe-metrics-certs podName:6440de1a-aff3-4b9d-a374-4e029ece2bfe nodeName:}" failed. No retries permitted until 2026-07-22 16:07:38.01733678 +0000 UTC m=+18.147806766 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6440de1a-aff3-4b9d-a374-4e029ece2bfe-metrics-certs") pod "network-metrics-daemon-87ltq" (UID: "6440de1a-aff3-4b9d-a374-4e029ece2bfe") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 16:07:30.443887 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:30.442497    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:07:30.443887 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:30.443584    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dhg6" podUID="1cd85e6d-9e7c-4c21-bc75-98cc25975bfc"
Jul 22 16:07:30.443887 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:30.443667    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kq5l6"
Jul 22 16:07:30.443887 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:30.443731    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kq5l6" podUID="36f0cd2c-f5c3-47ce-bb45-0f604e33aed9"
Jul 22 16:07:31.442289 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:31.442245    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ltq"
Jul 22 16:07:31.442725 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:31.442388    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87ltq" podUID="6440de1a-aff3-4b9d-a374-4e029ece2bfe"
Jul 22 16:07:32.442269 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:32.442230    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kq5l6"
Jul 22 16:07:32.442456 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:32.442371    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kq5l6" podUID="36f0cd2c-f5c3-47ce-bb45-0f604e33aed9"
Jul 22 16:07:32.442456 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:32.442428    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:07:32.442897 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:32.442531    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dhg6" podUID="1cd85e6d-9e7c-4c21-bc75-98cc25975bfc"
Jul 22 16:07:33.443119 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:33.443069    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ltq"
Jul 22 16:07:33.443520 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:33.443229    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87ltq" podUID="6440de1a-aff3-4b9d-a374-4e029ece2bfe"
Jul 22 16:07:34.442645 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:34.442545    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:07:34.442645 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:34.442582    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kq5l6"
Jul 22 16:07:34.442887 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:34.442695    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dhg6" podUID="1cd85e6d-9e7c-4c21-bc75-98cc25975bfc"
Jul 22 16:07:34.442887 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:34.442831    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kq5l6" podUID="36f0cd2c-f5c3-47ce-bb45-0f604e33aed9"
Jul 22 16:07:35.442556 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:35.442521    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ltq"
Jul 22 16:07:35.443072 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:35.442658    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87ltq" podUID="6440de1a-aff3-4b9d-a374-4e029ece2bfe"
Jul 22 16:07:36.442195 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:36.442160    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:07:36.442386 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:36.442172    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kq5l6"
Jul 22 16:07:36.442386 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:36.442304    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dhg6" podUID="1cd85e6d-9e7c-4c21-bc75-98cc25975bfc"
Jul 22 16:07:36.442386 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:36.442374    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kq5l6" podUID="36f0cd2c-f5c3-47ce-bb45-0f604e33aed9"
Jul 22 16:07:37.442893 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:37.442864    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ltq"
Jul 22 16:07:37.443325 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:37.442979    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87ltq" podUID="6440de1a-aff3-4b9d-a374-4e029ece2bfe"
Jul 22 16:07:37.981893 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:37.981836    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g44rx\" (UniqueName: \"kubernetes.io/projected/36f0cd2c-f5c3-47ce-bb45-0f604e33aed9-kube-api-access-g44rx\") pod \"network-check-target-kq5l6\" (UID: \"36f0cd2c-f5c3-47ce-bb45-0f604e33aed9\") " pod="openshift-network-diagnostics/network-check-target-kq5l6"
Jul 22 16:07:37.982043 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:37.982002    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 16:07:37.982043 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:37.982022    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 16:07:37.982043 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:37.982031    2563 projected.go:196] Error preparing data for projected volume kube-api-access-g44rx for pod openshift-network-diagnostics/network-check-target-kq5l6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 16:07:37.982182 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:37.982081    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/36f0cd2c-f5c3-47ce-bb45-0f604e33aed9-kube-api-access-g44rx podName:36f0cd2c-f5c3-47ce-bb45-0f604e33aed9 nodeName:}" failed. No retries permitted until 2026-07-22 16:07:53.982065228 +0000 UTC m=+34.112535222 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-g44rx" (UniqueName: "kubernetes.io/projected/36f0cd2c-f5c3-47ce-bb45-0f604e33aed9-kube-api-access-g44rx") pod "network-check-target-kq5l6" (UID: "36f0cd2c-f5c3-47ce-bb45-0f604e33aed9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 16:07:38.082354 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:38.082319    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6440de1a-aff3-4b9d-a374-4e029ece2bfe-metrics-certs\") pod \"network-metrics-daemon-87ltq\" (UID: \"6440de1a-aff3-4b9d-a374-4e029ece2bfe\") " pod="openshift-multus/network-metrics-daemon-87ltq"
Jul 22 16:07:38.082530 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:38.082367    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1cd85e6d-9e7c-4c21-bc75-98cc25975bfc-original-pull-secret\") pod \"global-pull-secret-syncer-5dhg6\" (UID: \"1cd85e6d-9e7c-4c21-bc75-98cc25975bfc\") " pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:07:38.082530 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:38.082479    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 16:07:38.082639 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:38.082533    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cd85e6d-9e7c-4c21-bc75-98cc25975bfc-original-pull-secret podName:1cd85e6d-9e7c-4c21-bc75-98cc25975bfc nodeName:}" failed. No retries permitted until 2026-07-22 16:07:54.082515468 +0000 UTC m=+34.212985462 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1cd85e6d-9e7c-4c21-bc75-98cc25975bfc-original-pull-secret") pod "global-pull-secret-syncer-5dhg6" (UID: "1cd85e6d-9e7c-4c21-bc75-98cc25975bfc") : object "kube-system"/"original-pull-secret" not registered
Jul 22 16:07:38.082639 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:38.082480    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 16:07:38.082639 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:38.082582    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6440de1a-aff3-4b9d-a374-4e029ece2bfe-metrics-certs podName:6440de1a-aff3-4b9d-a374-4e029ece2bfe nodeName:}" failed. No retries permitted until 2026-07-22 16:07:54.082572607 +0000 UTC m=+34.213042602 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6440de1a-aff3-4b9d-a374-4e029ece2bfe-metrics-certs") pod "network-metrics-daemon-87ltq" (UID: "6440de1a-aff3-4b9d-a374-4e029ece2bfe") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 16:07:38.442507 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:38.442472    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:07:38.442705 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:38.442485    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kq5l6"
Jul 22 16:07:38.442705 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:38.442614    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dhg6" podUID="1cd85e6d-9e7c-4c21-bc75-98cc25975bfc"
Jul 22 16:07:38.442705 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:38.442682    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kq5l6" podUID="36f0cd2c-f5c3-47ce-bb45-0f604e33aed9"
Jul 22 16:07:39.442338 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:39.442302    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ltq"
Jul 22 16:07:39.442740 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:39.442422    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87ltq" podUID="6440de1a-aff3-4b9d-a374-4e029ece2bfe"
Jul 22 16:07:40.442990 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:40.442949    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:07:40.443475 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:40.443064    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dhg6" podUID="1cd85e6d-9e7c-4c21-bc75-98cc25975bfc"
Jul 22 16:07:40.443475 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:40.443122    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kq5l6"
Jul 22 16:07:40.443475 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:40.443210    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kq5l6" podUID="36f0cd2c-f5c3-47ce-bb45-0f604e33aed9"
Jul 22 16:07:41.442293 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:41.442260    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ltq"
Jul 22 16:07:41.442458 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:41.442389    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87ltq" podUID="6440de1a-aff3-4b9d-a374-4e029ece2bfe"
Jul 22 16:07:42.442708 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:42.442677    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:07:42.443378 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:42.442817    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dhg6" podUID="1cd85e6d-9e7c-4c21-bc75-98cc25975bfc"
Jul 22 16:07:42.443378 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:42.443012    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kq5l6"
Jul 22 16:07:42.443378 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:42.443112    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kq5l6" podUID="36f0cd2c-f5c3-47ce-bb45-0f604e33aed9"
Jul 22 16:07:42.693471 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:42.693239    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6b2b2" event={"ID":"b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac","Type":"ContainerStarted","Data":"f8ae311c84c4b757771745c04f1a678691bea6905231db0480e371e7e7a5f908"}
Jul 22 16:07:42.694703 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:42.694680    2563 generic.go:356] "Generic (PLEG): container finished" podID="64b5ff5e-8328-4e1b-9a8d-341aa0cebba5" containerID="e9d878cd9898ae347769d9ccffb37c2ea80beaa180a8eb9aaaf996981d845acf" exitCode=0
Jul 22 16:07:42.694803 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:42.694758    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xtqkt" event={"ID":"64b5ff5e-8328-4e1b-9a8d-341aa0cebba5","Type":"ContainerDied","Data":"e9d878cd9898ae347769d9ccffb37c2ea80beaa180a8eb9aaaf996981d845acf"}
Jul 22 16:07:42.696734 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:42.696712    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2djqd" event={"ID":"d5ae481a-e39e-4c39-8f8e-e9255121886d","Type":"ContainerStarted","Data":"4240571fcac93890a2f354e443bcb6a753206eac54130384f1f8d7110920316c"}
Jul 22 16:07:42.696964 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:42.696777    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2djqd" event={"ID":"d5ae481a-e39e-4c39-8f8e-e9255121886d","Type":"ContainerStarted","Data":"ad18133f3d892ba7c25e88f30a160751c14719f8d81b89f3cdc3c5cd2a305683"}
Jul 22 16:07:42.696964 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:42.696807    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2djqd" event={"ID":"d5ae481a-e39e-4c39-8f8e-e9255121886d","Type":"ContainerStarted","Data":"e4eb5197e2bed904dbb5aaffb8863290aab0640e36b0fe38b63ec4fe38293c07"}
Jul 22 16:07:42.698480 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:42.698461    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff565" event={"ID":"fed3e161-7e00-481a-a172-a751ec55d8cc","Type":"ContainerStarted","Data":"ecce44261b03eecbc1866c27aaad5feea9f3ed4a775f07d024e687d79bdd673c"}
Jul 22 16:07:42.699996 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:42.699964    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-85sw4" event={"ID":"a2dfe635-d673-4f73-94cc-d852424b78d9","Type":"ContainerStarted","Data":"c88e2baa43a7c9f608cc1c91e2abc74f60feabf1204667400cea5880b260d183"}
Jul 22 16:07:42.701729 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:42.701683    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qj9nm" event={"ID":"861b97be-45d1-4aee-90eb-b2387ede2142","Type":"ContainerStarted","Data":"f459de7bb518081fe328068cec07f52b3ecc16e75dc26b1a70c4624e7b6d3f5e"}
Jul 22 16:07:42.703204 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:42.703177    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xr6pt" event={"ID":"2193e465-f388-4298-80aa-c4fd96ec3f2f","Type":"ContainerStarted","Data":"2dc0dd7341891696967fb55c790c7364ead676b69295b07f7f14be1455dc65f0"}
Jul 22 16:07:42.704559 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:42.704540    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bzc75" event={"ID":"5e6f48d6-b0c6-4827-a1de-17ec418ccaf2","Type":"ContainerStarted","Data":"6815b9d68302c2605e8cc8db5e836cb3955d29168c2c08e7b9f27cf4fd4311a8"}
Jul 22 16:07:42.714800 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:42.712648    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-224.ec2.internal" podStartSLOduration=21.712629857 podStartE2EDuration="21.712629857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:07:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:07:25.687984563 +0000 UTC m=+5.818454577" watchObservedRunningTime="2026-07-22 16:07:42.712629857 +0000 UTC m=+22.843099861"
Jul 22 16:07:42.714800 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:42.713824    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6b2b2" podStartSLOduration=8.192377954 podStartE2EDuration="22.713809989s" totalImagesPullingTime="14.521432035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:07:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:07:42.712367838 +0000 UTC m=+22.842837841" watchObservedRunningTime="2026-07-22 16:07:42.713809989 +0000 UTC m=+22.844279992"
Jul 22 16:07:42.732868 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:42.732823    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-xr6pt" podStartSLOduration=3.985878149 podStartE2EDuration="22.732810693s" totalImagesPullingTime="18.746932544s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:07:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:07:42.732472473 +0000 UTC m=+22.862942475" watchObservedRunningTime="2026-07-22 16:07:42.732810693 +0000 UTC m=+22.863280694"
Jul 22 16:07:42.846163 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:42.846119    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-bzc75" podStartSLOduration=4.101476557 podStartE2EDuration="22.846102296s" totalImagesPullingTime="18.744625739s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:07:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:07:42.795088103 +0000 UTC m=+22.925558130" watchObservedRunningTime="2026-07-22 16:07:42.846102296 +0000 UTC m=+22.976572302"
Jul 22 16:07:42.846268 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:42.846194    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qj9nm" podStartSLOduration=4.048229863 podStartE2EDuration="22.846188702s" totalImagesPullingTime="18.797958839s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:07:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:07:42.845725619 +0000 UTC m=+22.976195639" watchObservedRunningTime="2026-07-22 16:07:42.846188702 +0000 UTC m=+22.976658704"
Jul 22 16:07:42.982225 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:42.982146    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6b2b2"
Jul 22 16:07:42.982816 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:42.982796    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6b2b2"
Jul 22 16:07:43.014809 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:43.014752    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-85sw4" podStartSLOduration=4.25118924 podStartE2EDuration="23.014730335s" totalImagesPullingTime="18.763541095s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:07:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:07:42.886191556 +0000 UTC m=+23.016661557" watchObservedRunningTime="2026-07-22 16:07:43.014730335 +0000 UTC m=+23.145200338"
Jul 22 16:07:43.443007 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:43.442982    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ltq"
Jul 22 16:07:43.443478 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:43.443081    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87ltq" podUID="6440de1a-aff3-4b9d-a374-4e029ece2bfe"
Jul 22 16:07:43.709968 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:43.709919    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2djqd" event={"ID":"d5ae481a-e39e-4c39-8f8e-e9255121886d","Type":"ContainerStarted","Data":"441e30db54ccb7ee1b6c750c497167682dee2f643232a622a0abe823388c7b50"}
Jul 22 16:07:43.710155 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:43.710028    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2djqd" event={"ID":"d5ae481a-e39e-4c39-8f8e-e9255121886d","Type":"ContainerStarted","Data":"003d2e329c5674a5c44ba2e50b71e8f91e7cd9da4601d421d1b3b1abc3bb0fd2"}
Jul 22 16:07:43.710155 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:43.710081    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2djqd" event={"ID":"d5ae481a-e39e-4c39-8f8e-e9255121886d","Type":"ContainerStarted","Data":"472252e2f010e89ccff25a880c4e324e2a9629d0d383fbdc65e94762a1772c87"}
Jul 22 16:07:43.711623 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:43.711599    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cq5v7" event={"ID":"662d650d-8f3e-45ce-900d-e3a0ed385342","Type":"ContainerStarted","Data":"14f39b82f1b6bb4b814d51f6896bd1284ac19bb2375e8e4cd5bbddaf4e9a9e3d"}
Jul 22 16:07:43.711929 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:43.711879    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6b2b2"
Jul 22 16:07:43.712356 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:43.712336    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6b2b2"
Jul 22 16:07:43.744329 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:43.744286    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-cq5v7" podStartSLOduration=5.009917683 podStartE2EDuration="23.744264599s" totalImagesPullingTime="18.734346916s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:07:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:07:43.728779438 +0000 UTC m=+23.859249464" watchObservedRunningTime="2026-07-22 16:07:43.744264599 +0000 UTC m=+23.874734599"
Jul 22 16:07:43.791845 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:43.791817    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 22 16:07:44.322785 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:44.322673    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-22T16:07:43.791839529Z","UUID":"2730bddb-6aaa-421d-957c-f8d6d12b7c8a","Handler":null,"Name":"","Endpoint":""}
Jul 22 16:07:44.325935 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:44.325605    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 22 16:07:44.325935 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:44.325653    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 22 16:07:44.442923 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:44.442887    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kq5l6"
Jul 22 16:07:44.443098 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:44.442887    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:07:44.443098 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:44.443017    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kq5l6" podUID="36f0cd2c-f5c3-47ce-bb45-0f604e33aed9"
Jul 22 16:07:44.443579 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:44.443115    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dhg6" podUID="1cd85e6d-9e7c-4c21-bc75-98cc25975bfc"
Jul 22 16:07:44.716604 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:44.716569    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff565" event={"ID":"fed3e161-7e00-481a-a172-a751ec55d8cc","Type":"ContainerStarted","Data":"3a216f696b97864f3edc6cea3d3b717b544a62ba5b2efe240243a992fd7e5b1b"}
Jul 22 16:07:45.442498 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:45.442468    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ltq"
Jul 22 16:07:45.442675 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:45.442589    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87ltq" podUID="6440de1a-aff3-4b9d-a374-4e029ece2bfe"
Jul 22 16:07:45.722688 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:45.722593    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2djqd" event={"ID":"d5ae481a-e39e-4c39-8f8e-e9255121886d","Type":"ContainerStarted","Data":"fc195b9df43a056243c32bb072e75385da15639d4902b2d89d906c04419224e8"}
Jul 22 16:07:45.724680 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:45.724658    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff565" event={"ID":"fed3e161-7e00-481a-a172-a751ec55d8cc","Type":"ContainerStarted","Data":"34b1b8951987e61f1cc7d2109ac07631d754e01510abb2b850dc2db76e0bbe84"}
Jul 22 16:07:45.749429 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:45.749368    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff565" podStartSLOduration=4.783537957 podStartE2EDuration="25.749350026s" totalImagesPullingTime="20.965812069s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:07:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:07:45.747564493 +0000 UTC m=+25.878034496" watchObservedRunningTime="2026-07-22 16:07:45.749350026 +0000 UTC m=+25.879820028"
Jul 22 16:07:46.442894 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:46.442842    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:07:46.443075 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:46.442842    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kq5l6"
Jul 22 16:07:46.443075 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:46.443008    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dhg6" podUID="1cd85e6d-9e7c-4c21-bc75-98cc25975bfc"
Jul 22 16:07:46.443179 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:46.443090    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kq5l6" podUID="36f0cd2c-f5c3-47ce-bb45-0f604e33aed9"
Jul 22 16:07:47.443036 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:47.442801    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ltq"
Jul 22 16:07:47.443658 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:47.443163    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87ltq" podUID="6440de1a-aff3-4b9d-a374-4e029ece2bfe"
Jul 22 16:07:47.730403 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:47.730364    2563 generic.go:356] "Generic (PLEG): container finished" podID="64b5ff5e-8328-4e1b-9a8d-341aa0cebba5" containerID="1a2e5d7ea3ddbcd9dfc58f5ec80f6f89afb8c949b3e3d596d7548dec3aa651dd" exitCode=0
Jul 22 16:07:47.730575 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:47.730443    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xtqkt" event={"ID":"64b5ff5e-8328-4e1b-9a8d-341aa0cebba5","Type":"ContainerDied","Data":"1a2e5d7ea3ddbcd9dfc58f5ec80f6f89afb8c949b3e3d596d7548dec3aa651dd"}
Jul 22 16:07:47.736307 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:47.736281    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2djqd" event={"ID":"d5ae481a-e39e-4c39-8f8e-e9255121886d","Type":"ContainerStarted","Data":"bb442ee60f2df41111655fc2ed761178f11e7979ec2843afd90b7b95bf3d0ece"}
Jul 22 16:07:47.736561 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:47.736533    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:47.736659 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:47.736613    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:47.751193 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:47.751172    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:47.783233 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:47.783184    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2djqd" podStartSLOduration=8.982490973 podStartE2EDuration="27.78316739s" totalImagesPullingTime="18.800676417s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:07:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:07:47.7824303 +0000 UTC m=+27.912900311" watchObservedRunningTime="2026-07-22 16:07:47.78316739 +0000 UTC m=+27.913637391"
Jul 22 16:07:48.442301 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:48.442272    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:07:48.442525 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:48.442378    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dhg6" podUID="1cd85e6d-9e7c-4c21-bc75-98cc25975bfc"
Jul 22 16:07:48.442525 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:48.442428    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kq5l6"
Jul 22 16:07:48.442525 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:48.442516    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kq5l6" podUID="36f0cd2c-f5c3-47ce-bb45-0f604e33aed9"
Jul 22 16:07:48.738558 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:48.738371    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:48.753558 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:48.753537    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:07:49.341936 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:49.341905    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kq5l6"]
Jul 22 16:07:49.342201 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:49.342071    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kq5l6"
Jul 22 16:07:49.342278 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:49.342196    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kq5l6" podUID="36f0cd2c-f5c3-47ce-bb45-0f604e33aed9"
Jul 22 16:07:49.346257 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:49.346230    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5dhg6"]
Jul 22 16:07:49.346384 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:49.346348    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:07:49.346463 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:49.346440    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dhg6" podUID="1cd85e6d-9e7c-4c21-bc75-98cc25975bfc"
Jul 22 16:07:49.347463 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:49.347438    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-87ltq"]
Jul 22 16:07:49.347578 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:49.347543    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ltq"
Jul 22 16:07:49.347657 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:49.347617    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87ltq" podUID="6440de1a-aff3-4b9d-a374-4e029ece2bfe"
Jul 22 16:07:49.742281 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:49.742247    2563 generic.go:356] "Generic (PLEG): container finished" podID="64b5ff5e-8328-4e1b-9a8d-341aa0cebba5" containerID="c203c52350ebbb166115aa323c2567ee681da36b48d18cbb916bf422d1339e19" exitCode=0
Jul 22 16:07:49.742753 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:49.742338    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xtqkt" event={"ID":"64b5ff5e-8328-4e1b-9a8d-341aa0cebba5","Type":"ContainerDied","Data":"c203c52350ebbb166115aa323c2567ee681da36b48d18cbb916bf422d1339e19"}
Jul 22 16:07:51.442333 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:51.442304    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ltq"
Jul 22 16:07:51.442333 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:51.442333    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kq5l6"
Jul 22 16:07:51.442800 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:51.442434    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87ltq" podUID="6440de1a-aff3-4b9d-a374-4e029ece2bfe"
Jul 22 16:07:51.442800 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:51.442479    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:07:51.442800 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:51.442586    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kq5l6" podUID="36f0cd2c-f5c3-47ce-bb45-0f604e33aed9"
Jul 22 16:07:51.443164 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:51.443134    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dhg6" podUID="1cd85e6d-9e7c-4c21-bc75-98cc25975bfc"
Jul 22 16:07:51.750302 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:51.750222    2563 generic.go:356] "Generic (PLEG): container finished" podID="64b5ff5e-8328-4e1b-9a8d-341aa0cebba5" containerID="70239b86bdea30d672d5fd76ab9be9d05bd3cc6921fd8118de6acb6b119faa0c" exitCode=0
Jul 22 16:07:51.750302 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:51.750261    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xtqkt" event={"ID":"64b5ff5e-8328-4e1b-9a8d-341aa0cebba5","Type":"ContainerDied","Data":"70239b86bdea30d672d5fd76ab9be9d05bd3cc6921fd8118de6acb6b119faa0c"}
Jul 22 16:07:53.442983 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:53.442773    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kq5l6"
Jul 22 16:07:53.443426 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:53.442814    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ltq"
Jul 22 16:07:53.443426 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:53.443080    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kq5l6" podUID="36f0cd2c-f5c3-47ce-bb45-0f604e33aed9"
Jul 22 16:07:53.443426 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:53.443192    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87ltq" podUID="6440de1a-aff3-4b9d-a374-4e029ece2bfe"
Jul 22 16:07:53.443426 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:53.442825    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:07:53.443426 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:53.443312    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5dhg6" podUID="1cd85e6d-9e7c-4c21-bc75-98cc25975bfc"
Jul 22 16:07:53.998040 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:53.998004    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g44rx\" (UniqueName: \"kubernetes.io/projected/36f0cd2c-f5c3-47ce-bb45-0f604e33aed9-kube-api-access-g44rx\") pod \"network-check-target-kq5l6\" (UID: \"36f0cd2c-f5c3-47ce-bb45-0f604e33aed9\") " pod="openshift-network-diagnostics/network-check-target-kq5l6"
Jul 22 16:07:53.998226 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:53.998156    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 16:07:53.998226 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:53.998178    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 16:07:53.998226 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:53.998192    2563 projected.go:196] Error preparing data for projected volume kube-api-access-g44rx for pod openshift-network-diagnostics/network-check-target-kq5l6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 16:07:53.998341 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:53.998252    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/36f0cd2c-f5c3-47ce-bb45-0f604e33aed9-kube-api-access-g44rx podName:36f0cd2c-f5c3-47ce-bb45-0f604e33aed9 nodeName:}" failed. No retries permitted until 2026-07-22 16:08:25.998235814 +0000 UTC m=+66.128705808 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-g44rx" (UniqueName: "kubernetes.io/projected/36f0cd2c-f5c3-47ce-bb45-0f604e33aed9-kube-api-access-g44rx") pod "network-check-target-kq5l6" (UID: "36f0cd2c-f5c3-47ce-bb45-0f604e33aed9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 16:07:54.099345 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:54.099309    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1cd85e6d-9e7c-4c21-bc75-98cc25975bfc-original-pull-secret\") pod \"global-pull-secret-syncer-5dhg6\" (UID: \"1cd85e6d-9e7c-4c21-bc75-98cc25975bfc\") " pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:07:54.099522 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:54.099380    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6440de1a-aff3-4b9d-a374-4e029ece2bfe-metrics-certs\") pod \"network-metrics-daemon-87ltq\" (UID: \"6440de1a-aff3-4b9d-a374-4e029ece2bfe\") " pod="openshift-multus/network-metrics-daemon-87ltq"
Jul 22 16:07:54.099522 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:54.099485    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 16:07:54.099612 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:54.099483    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 16:07:54.099612 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:54.099550    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6440de1a-aff3-4b9d-a374-4e029ece2bfe-metrics-certs podName:6440de1a-aff3-4b9d-a374-4e029ece2bfe nodeName:}" failed. No retries permitted until 2026-07-22 16:08:26.099531813 +0000 UTC m=+66.230001807 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6440de1a-aff3-4b9d-a374-4e029ece2bfe-metrics-certs") pod "network-metrics-daemon-87ltq" (UID: "6440de1a-aff3-4b9d-a374-4e029ece2bfe") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 16:07:54.099612 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:54.099594    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cd85e6d-9e7c-4c21-bc75-98cc25975bfc-original-pull-secret podName:1cd85e6d-9e7c-4c21-bc75-98cc25975bfc nodeName:}" failed. No retries permitted until 2026-07-22 16:08:26.099575463 +0000 UTC m=+66.230045456 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1cd85e6d-9e7c-4c21-bc75-98cc25975bfc-original-pull-secret") pod "global-pull-secret-syncer-5dhg6" (UID: "1cd85e6d-9e7c-4c21-bc75-98cc25975bfc") : object "kube-system"/"original-pull-secret" not registered
Jul 22 16:07:54.364665 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:54.364570    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:07:54.464937 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:54.464901    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:07:54.527395 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:54.527363    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:07:55.016577 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.016493    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-224.ec2.internal" event="NodeReady"
Jul 22 16:07:55.016742 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.016650    2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 22 16:07:55.079204 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.079167    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-vmgf6"]
Jul 22 16:07:55.105972 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.105942    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-xdxv8"]
Jul 22 16:07:55.106152 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.106052    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vmgf6"
Jul 22 16:07:55.109219 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.109191    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 22 16:07:55.109367 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.109350    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 22 16:07:55.109481 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.109463    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7zs9d\""
Jul 22 16:07:55.109539 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.109463    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 22 16:07:55.121441 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.121405    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vmgf6"]
Jul 22 16:07:55.121574 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.121451    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xdxv8"]
Jul 22 16:07:55.121574 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.121534    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xdxv8"
Jul 22 16:07:55.127277 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.127256    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 22 16:07:55.127517 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.127496    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 22 16:07:55.127885 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.127829    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wdbjp\""
Jul 22 16:07:55.206609 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.206572    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/196426fb-47a2-47d8-8a89-094572fbe64e-cert\") pod \"ingress-canary-vmgf6\" (UID: \"196426fb-47a2-47d8-8a89-094572fbe64e\") " pod="openshift-ingress-canary/ingress-canary-vmgf6"
Jul 22 16:07:55.206609 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.206614    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv6gl\" (UniqueName: \"kubernetes.io/projected/196426fb-47a2-47d8-8a89-094572fbe64e-kube-api-access-fv6gl\") pod \"ingress-canary-vmgf6\" (UID: \"196426fb-47a2-47d8-8a89-094572fbe64e\") " pod="openshift-ingress-canary/ingress-canary-vmgf6"
Jul 22 16:07:55.206831 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.206675    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/08a6a730-eb43-4bde-9079-47c469793f1d-config-volume\") pod \"dns-default-xdxv8\" (UID: \"08a6a730-eb43-4bde-9079-47c469793f1d\") " pod="openshift-dns/dns-default-xdxv8"
Jul 22 16:07:55.206831 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.206704    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2jw6\" (UniqueName: \"kubernetes.io/projected/08a6a730-eb43-4bde-9079-47c469793f1d-kube-api-access-c2jw6\") pod \"dns-default-xdxv8\" (UID: \"08a6a730-eb43-4bde-9079-47c469793f1d\") " pod="openshift-dns/dns-default-xdxv8"
Jul 22 16:07:55.206831 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.206730    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08a6a730-eb43-4bde-9079-47c469793f1d-tmp-dir\") pod \"dns-default-xdxv8\" (UID: \"08a6a730-eb43-4bde-9079-47c469793f1d\") " pod="openshift-dns/dns-default-xdxv8"
Jul 22 16:07:55.206831 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.206792    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08a6a730-eb43-4bde-9079-47c469793f1d-metrics-tls\") pod \"dns-default-xdxv8\" (UID: \"08a6a730-eb43-4bde-9079-47c469793f1d\") " pod="openshift-dns/dns-default-xdxv8"
Jul 22 16:07:55.308068 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.307981    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/08a6a730-eb43-4bde-9079-47c469793f1d-config-volume\") pod \"dns-default-xdxv8\" (UID: \"08a6a730-eb43-4bde-9079-47c469793f1d\") " pod="openshift-dns/dns-default-xdxv8"
Jul 22 16:07:55.308068 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.308040    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2jw6\" (UniqueName: \"kubernetes.io/projected/08a6a730-eb43-4bde-9079-47c469793f1d-kube-api-access-c2jw6\") pod \"dns-default-xdxv8\" (UID: \"08a6a730-eb43-4bde-9079-47c469793f1d\") " pod="openshift-dns/dns-default-xdxv8"
Jul 22 16:07:55.308068 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.308065    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08a6a730-eb43-4bde-9079-47c469793f1d-tmp-dir\") pod \"dns-default-xdxv8\" (UID: \"08a6a730-eb43-4bde-9079-47c469793f1d\") " pod="openshift-dns/dns-default-xdxv8"
Jul 22 16:07:55.308342 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.308123    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08a6a730-eb43-4bde-9079-47c469793f1d-metrics-tls\") pod \"dns-default-xdxv8\" (UID: \"08a6a730-eb43-4bde-9079-47c469793f1d\") " pod="openshift-dns/dns-default-xdxv8"
Jul 22 16:07:55.308342 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.308180    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/196426fb-47a2-47d8-8a89-094572fbe64e-cert\") pod \"ingress-canary-vmgf6\" (UID: \"196426fb-47a2-47d8-8a89-094572fbe64e\") " pod="openshift-ingress-canary/ingress-canary-vmgf6"
Jul 22 16:07:55.308342 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.308204    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fv6gl\" (UniqueName: \"kubernetes.io/projected/196426fb-47a2-47d8-8a89-094572fbe64e-kube-api-access-fv6gl\") pod \"ingress-canary-vmgf6\" (UID: \"196426fb-47a2-47d8-8a89-094572fbe64e\") " pod="openshift-ingress-canary/ingress-canary-vmgf6"
Jul 22 16:07:55.308342 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:55.308297    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 16:07:55.308556 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:55.308357    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08a6a730-eb43-4bde-9079-47c469793f1d-metrics-tls podName:08a6a730-eb43-4bde-9079-47c469793f1d nodeName:}" failed. No retries permitted until 2026-07-22 16:07:55.808337772 +0000 UTC m=+35.938807767 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08a6a730-eb43-4bde-9079-47c469793f1d-metrics-tls") pod "dns-default-xdxv8" (UID: "08a6a730-eb43-4bde-9079-47c469793f1d") : secret "dns-default-metrics-tls" not found
Jul 22 16:07:55.308556 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.308467    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08a6a730-eb43-4bde-9079-47c469793f1d-tmp-dir\") pod \"dns-default-xdxv8\" (UID: \"08a6a730-eb43-4bde-9079-47c469793f1d\") " pod="openshift-dns/dns-default-xdxv8"
Jul 22 16:07:55.308670 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.308648    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/08a6a730-eb43-4bde-9079-47c469793f1d-config-volume\") pod \"dns-default-xdxv8\" (UID: \"08a6a730-eb43-4bde-9079-47c469793f1d\") " pod="openshift-dns/dns-default-xdxv8"
Jul 22 16:07:55.312909 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.312884    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/196426fb-47a2-47d8-8a89-094572fbe64e-cert\") pod \"ingress-canary-vmgf6\" (UID: \"196426fb-47a2-47d8-8a89-094572fbe64e\") " pod="openshift-ingress-canary/ingress-canary-vmgf6"
Jul 22 16:07:55.321297 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.321269    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2jw6\" (UniqueName: \"kubernetes.io/projected/08a6a730-eb43-4bde-9079-47c469793f1d-kube-api-access-c2jw6\") pod \"dns-default-xdxv8\" (UID: \"08a6a730-eb43-4bde-9079-47c469793f1d\") " pod="openshift-dns/dns-default-xdxv8"
Jul 22 16:07:55.321725 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.321700    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv6gl\" (UniqueName: \"kubernetes.io/projected/196426fb-47a2-47d8-8a89-094572fbe64e-kube-api-access-fv6gl\") pod \"ingress-canary-vmgf6\" (UID: \"196426fb-47a2-47d8-8a89-094572fbe64e\") " pod="openshift-ingress-canary/ingress-canary-vmgf6"
Jul 22 16:07:55.418954 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.418917    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vmgf6"
Jul 22 16:07:55.443058 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.443033    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:07:55.443058 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.443059    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kq5l6"
Jul 22 16:07:55.443236 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.443059    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ltq"
Jul 22 16:07:55.453828 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.453805    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 22 16:07:55.454006 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.453937    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 22 16:07:55.454006 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.453987    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 22 16:07:55.454236 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.454220    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-s68m6\""
Jul 22 16:07:55.454435 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.454406    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rb5n6\""
Jul 22 16:07:55.454528 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.454445    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 22 16:07:55.817429 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.817312    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08a6a730-eb43-4bde-9079-47c469793f1d-metrics-tls\") pod \"dns-default-xdxv8\" (UID: \"08a6a730-eb43-4bde-9079-47c469793f1d\") " pod="openshift-dns/dns-default-xdxv8"
Jul 22 16:07:55.818341 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:55.817488    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 16:07:55.818341 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:55.817549    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08a6a730-eb43-4bde-9079-47c469793f1d-metrics-tls podName:08a6a730-eb43-4bde-9079-47c469793f1d nodeName:}" failed. No retries permitted until 2026-07-22 16:07:56.817531073 +0000 UTC m=+36.948001076 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08a6a730-eb43-4bde-9079-47c469793f1d-metrics-tls") pod "dns-default-xdxv8" (UID: "08a6a730-eb43-4bde-9079-47c469793f1d") : secret "dns-default-metrics-tls" not found
Jul 22 16:07:55.881796 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:55.881761    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vmgf6"]
Jul 22 16:07:55.904744 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:07:55.904709    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod196426fb_47a2_47d8_8a89_094572fbe64e.slice/crio-306d93a176f4563bf19e5fccf3a5f30978a3f9454618d7a49a7c4d281494ce5f WatchSource:0}: Error finding container 306d93a176f4563bf19e5fccf3a5f30978a3f9454618d7a49a7c4d281494ce5f: Status 404 returned error can't find the container with id 306d93a176f4563bf19e5fccf3a5f30978a3f9454618d7a49a7c4d281494ce5f
Jul 22 16:07:56.423169 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:56.422818    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:07:56.513653 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:56.513614    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:07:56.621838 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:56.621799    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:07:56.723138 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:56.723106    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:07:56.766653 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:56.766620    2563 generic.go:356] "Generic (PLEG): container finished" podID="64b5ff5e-8328-4e1b-9a8d-341aa0cebba5" containerID="b0c7de99b0bcf4a6c0f4360c01f1e764f9f6f4f910f166de61bf31ea7ad73581" exitCode=0
Jul 22 16:07:56.766819 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:56.766717    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xtqkt" event={"ID":"64b5ff5e-8328-4e1b-9a8d-341aa0cebba5","Type":"ContainerDied","Data":"b0c7de99b0bcf4a6c0f4360c01f1e764f9f6f4f910f166de61bf31ea7ad73581"}
Jul 22 16:07:56.767996 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:56.767967    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vmgf6" event={"ID":"196426fb-47a2-47d8-8a89-094572fbe64e","Type":"ContainerStarted","Data":"306d93a176f4563bf19e5fccf3a5f30978a3f9454618d7a49a7c4d281494ce5f"}
Jul 22 16:07:56.822630 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:56.822606    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08a6a730-eb43-4bde-9079-47c469793f1d-metrics-tls\") pod \"dns-default-xdxv8\" (UID: \"08a6a730-eb43-4bde-9079-47c469793f1d\") " pod="openshift-dns/dns-default-xdxv8"
Jul 22 16:07:56.822983 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:56.822775    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 16:07:56.822983 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:56.822838    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08a6a730-eb43-4bde-9079-47c469793f1d-metrics-tls podName:08a6a730-eb43-4bde-9079-47c469793f1d nodeName:}" failed. No retries permitted until 2026-07-22 16:07:58.82281969 +0000 UTC m=+38.953289675 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08a6a730-eb43-4bde-9079-47c469793f1d-metrics-tls") pod "dns-default-xdxv8" (UID: "08a6a730-eb43-4bde-9079-47c469793f1d") : secret "dns-default-metrics-tls" not found
Jul 22 16:07:57.774103 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:57.774068    2563 generic.go:356] "Generic (PLEG): container finished" podID="64b5ff5e-8328-4e1b-9a8d-341aa0cebba5" containerID="6165a74eee15c07eeec7b1945a4ecacffadbd587aebd1a91999b6dd04fdb2e02" exitCode=0
Jul 22 16:07:57.774271 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:57.774117    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xtqkt" event={"ID":"64b5ff5e-8328-4e1b-9a8d-341aa0cebba5","Type":"ContainerDied","Data":"6165a74eee15c07eeec7b1945a4ecacffadbd587aebd1a91999b6dd04fdb2e02"}
Jul 22 16:07:58.609130 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:58.609099    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:07:58.676318 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:58.676284    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:07:58.779768 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:58.779737    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xtqkt" event={"ID":"64b5ff5e-8328-4e1b-9a8d-341aa0cebba5","Type":"ContainerStarted","Data":"eb83047ecb3213199272c520817f0a38dac83a9da278710adf14d29698bd36b2"}
Jul 22 16:07:58.781179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:58.781143    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vmgf6" event={"ID":"196426fb-47a2-47d8-8a89-094572fbe64e","Type":"ContainerStarted","Data":"79d7b7a37683bfdbced4653fa7569386411fc2434f311e8b708757381f40a901"}
Jul 22 16:07:58.824843 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:58.824803    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xtqkt" podStartSLOduration=9.084889227 podStartE2EDuration="38.824788802s" totalImagesPullingTime="29.739899575s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:07:20 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:07:58.824723294 +0000 UTC m=+38.955193319" watchObservedRunningTime="2026-07-22 16:07:58.824788802 +0000 UTC m=+38.955258825"
Jul 22 16:07:58.838152 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:58.838119    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08a6a730-eb43-4bde-9079-47c469793f1d-metrics-tls\") pod \"dns-default-xdxv8\" (UID: \"08a6a730-eb43-4bde-9079-47c469793f1d\") " pod="openshift-dns/dns-default-xdxv8"
Jul 22 16:07:58.838273 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:58.838251    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 16:07:58.838311 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:07:58.838303    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08a6a730-eb43-4bde-9079-47c469793f1d-metrics-tls podName:08a6a730-eb43-4bde-9079-47c469793f1d nodeName:}" failed. No retries permitted until 2026-07-22 16:08:02.838289964 +0000 UTC m=+42.968759947 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08a6a730-eb43-4bde-9079-47c469793f1d-metrics-tls") pod "dns-default-xdxv8" (UID: "08a6a730-eb43-4bde-9079-47c469793f1d") : secret "dns-default-metrics-tls" not found
Jul 22 16:07:59.058249 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:59.058213    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:07:59.147020 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:59.146938    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:07:59.212611 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:59.212577    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:07:59.427190 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:59.427167    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:07:59.499920 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:59.499888    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:07:59.626558 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:59.626528    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:07:59.727940 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:07:59.727844    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:00.425376 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:00.425344    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:00.508911 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:00.508873    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:01.187309 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.187245    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-vmgf6" podStartSLOduration=4.101347206 podStartE2EDuration="6.187223681s" totalImagesPullingTime="2.085876475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:07:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:07:58.8563455 +0000 UTC m=+38.986815532" watchObservedRunningTime="2026-07-22 16:08:01.187223681 +0000 UTC m=+41.317693680"
Jul 22 16:08:01.187716 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.187530    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ntb4r"]
Jul 22 16:08:01.189560 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.189545    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ntb4r"
Jul 22 16:08:01.195243 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.195218    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 22 16:08:01.195413 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.195398    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 22 16:08:01.195866 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.195836    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 22 16:08:01.196309 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.196105    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 22 16:08:01.199832 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.199817    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cvcbr\""
Jul 22 16:08:01.207179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.207159    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ntb4r"]
Jul 22 16:08:01.227552 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.227520    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:01.255089 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.254969    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hnxj2"]
Jul 22 16:08:01.257042 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.257022    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hnxj2"
Jul 22 16:08:01.260911 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.260886    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-vqhmm\""
Jul 22 16:08:01.260911 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.260911    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 22 16:08:01.261092 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.260912    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 22 16:08:01.274824 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.274796    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hnxj2"]
Jul 22 16:08:01.325928 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.325897    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:01.356221 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.356190    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9c8214fa-bb50-454f-a84b-558b13bd799e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ntb4r\" (UID: \"9c8214fa-bb50-454f-a84b-558b13bd799e\") " pod="openshift-insights/insights-runtime-extractor-ntb4r"
Jul 22 16:08:01.356354 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.356224    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjmvb\" (UniqueName: \"kubernetes.io/projected/9c8214fa-bb50-454f-a84b-558b13bd799e-kube-api-access-tjmvb\") pod \"insights-runtime-extractor-ntb4r\" (UID: \"9c8214fa-bb50-454f-a84b-558b13bd799e\") " pod="openshift-insights/insights-runtime-extractor-ntb4r"
Jul 22 16:08:01.356354 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.356246    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qv5jp\" (UniqueName: \"kubernetes.io/projected/cec619e3-7150-417b-ab93-67b2161ca87c-kube-api-access-qv5jp\") pod \"migrator-7cc9c7ffd7-hnxj2\" (UID: \"cec619e3-7150-417b-ab93-67b2161ca87c\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hnxj2"
Jul 22 16:08:01.356354 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.356283    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9c8214fa-bb50-454f-a84b-558b13bd799e-crio-socket\") pod \"insights-runtime-extractor-ntb4r\" (UID: \"9c8214fa-bb50-454f-a84b-558b13bd799e\") " pod="openshift-insights/insights-runtime-extractor-ntb4r"
Jul 22 16:08:01.356354 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.356316    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9c8214fa-bb50-454f-a84b-558b13bd799e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ntb4r\" (UID: \"9c8214fa-bb50-454f-a84b-558b13bd799e\") " pod="openshift-insights/insights-runtime-extractor-ntb4r"
Jul 22 16:08:01.356354 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.356347    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9c8214fa-bb50-454f-a84b-558b13bd799e-data-volume\") pod \"insights-runtime-extractor-ntb4r\" (UID: \"9c8214fa-bb50-454f-a84b-558b13bd799e\") " pod="openshift-insights/insights-runtime-extractor-ntb4r"
Jul 22 16:08:01.419999 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.419959    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:01.457607 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.457532    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9c8214fa-bb50-454f-a84b-558b13bd799e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ntb4r\" (UID: \"9c8214fa-bb50-454f-a84b-558b13bd799e\") " pod="openshift-insights/insights-runtime-extractor-ntb4r"
Jul 22 16:08:01.457607 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.457571    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjmvb\" (UniqueName: \"kubernetes.io/projected/9c8214fa-bb50-454f-a84b-558b13bd799e-kube-api-access-tjmvb\") pod \"insights-runtime-extractor-ntb4r\" (UID: \"9c8214fa-bb50-454f-a84b-558b13bd799e\") " pod="openshift-insights/insights-runtime-extractor-ntb4r"
Jul 22 16:08:01.457607 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.457600    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv5jp\" (UniqueName: \"kubernetes.io/projected/cec619e3-7150-417b-ab93-67b2161ca87c-kube-api-access-qv5jp\") pod \"migrator-7cc9c7ffd7-hnxj2\" (UID: \"cec619e3-7150-417b-ab93-67b2161ca87c\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hnxj2"
Jul 22 16:08:01.457880 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.457636    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9c8214fa-bb50-454f-a84b-558b13bd799e-crio-socket\") pod \"insights-runtime-extractor-ntb4r\" (UID: \"9c8214fa-bb50-454f-a84b-558b13bd799e\") " pod="openshift-insights/insights-runtime-extractor-ntb4r"
Jul 22 16:08:01.457880 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:08:01.457660    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 16:08:01.457880 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.457666    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9c8214fa-bb50-454f-a84b-558b13bd799e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ntb4r\" (UID: \"9c8214fa-bb50-454f-a84b-558b13bd799e\") " pod="openshift-insights/insights-runtime-extractor-ntb4r"
Jul 22 16:08:01.457880 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:08:01.457741    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 16:08:01.457880 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:08:01.457750    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9c8214fa-bb50-454f-a84b-558b13bd799e-kube-rbac-proxy-cm podName:9c8214fa-bb50-454f-a84b-558b13bd799e nodeName:}" failed. No retries permitted until 2026-07-22 16:08:01.957728401 +0000 UTC m=+42.088198404 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9c8214fa-bb50-454f-a84b-558b13bd799e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ntb4r" (UID: "9c8214fa-bb50-454f-a84b-558b13bd799e") : configmap "kube-rbac-proxy" not found
Jul 22 16:08:01.457880 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.457766    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9c8214fa-bb50-454f-a84b-558b13bd799e-crio-socket\") pod \"insights-runtime-extractor-ntb4r\" (UID: \"9c8214fa-bb50-454f-a84b-558b13bd799e\") " pod="openshift-insights/insights-runtime-extractor-ntb4r"
Jul 22 16:08:01.457880 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:08:01.457798    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c8214fa-bb50-454f-a84b-558b13bd799e-insights-runtime-extractor-tls podName:9c8214fa-bb50-454f-a84b-558b13bd799e nodeName:}" failed. No retries permitted until 2026-07-22 16:08:01.957782372 +0000 UTC m=+42.088252373 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9c8214fa-bb50-454f-a84b-558b13bd799e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ntb4r" (UID: "9c8214fa-bb50-454f-a84b-558b13bd799e") : secret "insights-runtime-extractor-tls" not found
Jul 22 16:08:01.457880 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.457818    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9c8214fa-bb50-454f-a84b-558b13bd799e-data-volume\") pod \"insights-runtime-extractor-ntb4r\" (UID: \"9c8214fa-bb50-454f-a84b-558b13bd799e\") " pod="openshift-insights/insights-runtime-extractor-ntb4r"
Jul 22 16:08:01.458154 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.458100    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9c8214fa-bb50-454f-a84b-558b13bd799e-data-volume\") pod \"insights-runtime-extractor-ntb4r\" (UID: \"9c8214fa-bb50-454f-a84b-558b13bd799e\") " pod="openshift-insights/insights-runtime-extractor-ntb4r"
Jul 22 16:08:01.468123 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.468100    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjmvb\" (UniqueName: \"kubernetes.io/projected/9c8214fa-bb50-454f-a84b-558b13bd799e-kube-api-access-tjmvb\") pod \"insights-runtime-extractor-ntb4r\" (UID: \"9c8214fa-bb50-454f-a84b-558b13bd799e\") " pod="openshift-insights/insights-runtime-extractor-ntb4r"
Jul 22 16:08:01.469564 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.469546    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qv5jp\" (UniqueName: \"kubernetes.io/projected/cec619e3-7150-417b-ab93-67b2161ca87c-kube-api-access-qv5jp\") pod \"migrator-7cc9c7ffd7-hnxj2\" (UID: \"cec619e3-7150-417b-ab93-67b2161ca87c\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hnxj2"
Jul 22 16:08:01.565113 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.565081    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hnxj2"
Jul 22 16:08:01.622384 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.622359    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:01.695024 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.694991    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hnxj2"]
Jul 22 16:08:01.699149 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:08:01.699111    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcec619e3_7150_417b_ab93_67b2161ca87c.slice/crio-af5ef3c50e9e5e30f0c1849cfa9afd021e9d14e5cde7f91cfd22f1106b24c205 WatchSource:0}: Error finding container af5ef3c50e9e5e30f0c1849cfa9afd021e9d14e5cde7f91cfd22f1106b24c205: Status 404 returned error can't find the container with id af5ef3c50e9e5e30f0c1849cfa9afd021e9d14e5cde7f91cfd22f1106b24c205
Jul 22 16:08:01.763536 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.763465    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:01.789531 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.789500    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hnxj2" event={"ID":"cec619e3-7150-417b-ab93-67b2161ca87c","Type":"ContainerStarted","Data":"af5ef3c50e9e5e30f0c1849cfa9afd021e9d14e5cde7f91cfd22f1106b24c205"}
Jul 22 16:08:01.863993 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.863950    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:01.962019 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.961993    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9c8214fa-bb50-454f-a84b-558b13bd799e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ntb4r\" (UID: \"9c8214fa-bb50-454f-a84b-558b13bd799e\") " pod="openshift-insights/insights-runtime-extractor-ntb4r"
Jul 22 16:08:01.962177 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:01.962068    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9c8214fa-bb50-454f-a84b-558b13bd799e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ntb4r\" (UID: \"9c8214fa-bb50-454f-a84b-558b13bd799e\") " pod="openshift-insights/insights-runtime-extractor-ntb4r"
Jul 22 16:08:01.962177 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:08:01.962144    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 16:08:01.962177 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:08:01.962150    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 16:08:01.962273 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:08:01.962196    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9c8214fa-bb50-454f-a84b-558b13bd799e-kube-rbac-proxy-cm podName:9c8214fa-bb50-454f-a84b-558b13bd799e nodeName:}" failed. No retries permitted until 2026-07-22 16:08:02.962182545 +0000 UTC m=+43.092652528 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9c8214fa-bb50-454f-a84b-558b13bd799e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ntb4r" (UID: "9c8214fa-bb50-454f-a84b-558b13bd799e") : configmap "kube-rbac-proxy" not found
Jul 22 16:08:01.962273 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:08:01.962214    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c8214fa-bb50-454f-a84b-558b13bd799e-insights-runtime-extractor-tls podName:9c8214fa-bb50-454f-a84b-558b13bd799e nodeName:}" failed. No retries permitted until 2026-07-22 16:08:02.962200695 +0000 UTC m=+43.092670674 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9c8214fa-bb50-454f-a84b-558b13bd799e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ntb4r" (UID: "9c8214fa-bb50-454f-a84b-558b13bd799e") : secret "insights-runtime-extractor-tls" not found
Jul 22 16:08:02.341545 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:02.341514    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:02.421832 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:02.421801    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:02.871198 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:02.871151    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08a6a730-eb43-4bde-9079-47c469793f1d-metrics-tls\") pod \"dns-default-xdxv8\" (UID: \"08a6a730-eb43-4bde-9079-47c469793f1d\") " pod="openshift-dns/dns-default-xdxv8"
Jul 22 16:08:02.871393 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:08:02.871287    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 16:08:02.871393 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:08:02.871358    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08a6a730-eb43-4bde-9079-47c469793f1d-metrics-tls podName:08a6a730-eb43-4bde-9079-47c469793f1d nodeName:}" failed. No retries permitted until 2026-07-22 16:08:10.871336053 +0000 UTC m=+51.001806050 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08a6a730-eb43-4bde-9079-47c469793f1d-metrics-tls") pod "dns-default-xdxv8" (UID: "08a6a730-eb43-4bde-9079-47c469793f1d") : secret "dns-default-metrics-tls" not found
Jul 22 16:08:02.972272 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:02.972240    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9c8214fa-bb50-454f-a84b-558b13bd799e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ntb4r\" (UID: \"9c8214fa-bb50-454f-a84b-558b13bd799e\") " pod="openshift-insights/insights-runtime-extractor-ntb4r"
Jul 22 16:08:02.972466 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:02.972307    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9c8214fa-bb50-454f-a84b-558b13bd799e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ntb4r\" (UID: \"9c8214fa-bb50-454f-a84b-558b13bd799e\") " pod="openshift-insights/insights-runtime-extractor-ntb4r"
Jul 22 16:08:02.972466 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:08:02.972369    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 16:08:02.972466 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:08:02.972463    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9c8214fa-bb50-454f-a84b-558b13bd799e-kube-rbac-proxy-cm podName:9c8214fa-bb50-454f-a84b-558b13bd799e nodeName:}" failed. No retries permitted until 2026-07-22 16:08:04.97244158 +0000 UTC m=+45.102911577 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9c8214fa-bb50-454f-a84b-558b13bd799e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ntb4r" (UID: "9c8214fa-bb50-454f-a84b-558b13bd799e") : configmap "kube-rbac-proxy" not found
Jul 22 16:08:02.972644 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:08:02.972470    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 16:08:02.972644 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:08:02.972523    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c8214fa-bb50-454f-a84b-558b13bd799e-insights-runtime-extractor-tls podName:9c8214fa-bb50-454f-a84b-558b13bd799e nodeName:}" failed. No retries permitted until 2026-07-22 16:08:04.972508483 +0000 UTC m=+45.102978478 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9c8214fa-bb50-454f-a84b-558b13bd799e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ntb4r" (UID: "9c8214fa-bb50-454f-a84b-558b13bd799e") : secret "insights-runtime-extractor-tls" not found
Jul 22 16:08:03.408957 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:03.408926    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xr6pt_2193e465-f388-4298-80aa-c4fd96ec3f2f/dns-node-resolver/0.log"
Jul 22 16:08:03.795974 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:03.795941    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hnxj2" event={"ID":"cec619e3-7150-417b-ab93-67b2161ca87c","Type":"ContainerStarted","Data":"eb13d31ff7862bfae7386592a2f0c0134f107c18d2d4fd4c159fb1121b26fb75"}
Jul 22 16:08:03.928686 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:03.928608    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-bjsjv"]
Jul 22 16:08:03.941317 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:03.941291    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-bjsjv"
Jul 22 16:08:03.942727 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:03.942697    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-bjsjv"]
Jul 22 16:08:03.944650 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:03.944630    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 22 16:08:03.944731 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:03.944657    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 22 16:08:03.945998 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:03.945978    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-6xf9m\""
Jul 22 16:08:03.946095 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:03.946015    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 22 16:08:03.946095 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:03.946081    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 22 16:08:04.009257 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:04.009232    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bzc75_5e6f48d6-b0c6-4827-a1de-17ec418ccaf2/node-ca/0.log"
Jul 22 16:08:04.025059 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:04.025028    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:04.080225 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:04.080191    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8ab8c418-0c70-4871-bea9-22de7835e94a-signing-key\") pod \"service-ca-58598695f7-bjsjv\" (UID: \"8ab8c418-0c70-4871-bea9-22de7835e94a\") " pod="openshift-service-ca/service-ca-58598695f7-bjsjv"
Jul 22 16:08:04.080384 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:04.080247    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8ab8c418-0c70-4871-bea9-22de7835e94a-signing-cabundle\") pod \"service-ca-58598695f7-bjsjv\" (UID: \"8ab8c418-0c70-4871-bea9-22de7835e94a\") " pod="openshift-service-ca/service-ca-58598695f7-bjsjv"
Jul 22 16:08:04.080384 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:04.080268    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj99c\" (UniqueName: \"kubernetes.io/projected/8ab8c418-0c70-4871-bea9-22de7835e94a-kube-api-access-gj99c\") pod \"service-ca-58598695f7-bjsjv\" (UID: \"8ab8c418-0c70-4871-bea9-22de7835e94a\") " pod="openshift-service-ca/service-ca-58598695f7-bjsjv"
Jul 22 16:08:04.125957 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:04.125928    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:04.181682 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:04.181606    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8ab8c418-0c70-4871-bea9-22de7835e94a-signing-cabundle\") pod \"service-ca-58598695f7-bjsjv\" (UID: \"8ab8c418-0c70-4871-bea9-22de7835e94a\") " pod="openshift-service-ca/service-ca-58598695f7-bjsjv"
Jul 22 16:08:04.181682 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:04.181636    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj99c\" (UniqueName: \"kubernetes.io/projected/8ab8c418-0c70-4871-bea9-22de7835e94a-kube-api-access-gj99c\") pod \"service-ca-58598695f7-bjsjv\" (UID: \"8ab8c418-0c70-4871-bea9-22de7835e94a\") " pod="openshift-service-ca/service-ca-58598695f7-bjsjv"
Jul 22 16:08:04.181913 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:04.181704    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8ab8c418-0c70-4871-bea9-22de7835e94a-signing-key\") pod \"service-ca-58598695f7-bjsjv\" (UID: \"8ab8c418-0c70-4871-bea9-22de7835e94a\") " pod="openshift-service-ca/service-ca-58598695f7-bjsjv"
Jul 22 16:08:04.182406 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:04.182381    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8ab8c418-0c70-4871-bea9-22de7835e94a-signing-cabundle\") pod \"service-ca-58598695f7-bjsjv\" (UID: \"8ab8c418-0c70-4871-bea9-22de7835e94a\") " pod="openshift-service-ca/service-ca-58598695f7-bjsjv"
Jul 22 16:08:04.184105 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:04.184078    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8ab8c418-0c70-4871-bea9-22de7835e94a-signing-key\") pod \"service-ca-58598695f7-bjsjv\" (UID: \"8ab8c418-0c70-4871-bea9-22de7835e94a\") " pod="openshift-service-ca/service-ca-58598695f7-bjsjv"
Jul 22 16:08:04.191568 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:04.191549    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj99c\" (UniqueName: \"kubernetes.io/projected/8ab8c418-0c70-4871-bea9-22de7835e94a-kube-api-access-gj99c\") pod \"service-ca-58598695f7-bjsjv\" (UID: \"8ab8c418-0c70-4871-bea9-22de7835e94a\") " pod="openshift-service-ca/service-ca-58598695f7-bjsjv"
Jul 22 16:08:04.251153 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:04.251119    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-bjsjv"
Jul 22 16:08:04.389370 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:04.389340    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-bjsjv"]
Jul 22 16:08:04.392939 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:08:04.392906    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ab8c418_0c70_4871_bea9_22de7835e94a.slice/crio-f4c2e1edf6a87c29592488a0e4a50e5b2da3013d4857509f76da77b187cc00bc WatchSource:0}: Error finding container f4c2e1edf6a87c29592488a0e4a50e5b2da3013d4857509f76da77b187cc00bc: Status 404 returned error can't find the container with id f4c2e1edf6a87c29592488a0e4a50e5b2da3013d4857509f76da77b187cc00bc
Jul 22 16:08:04.799639 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:04.799598    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-bjsjv" event={"ID":"8ab8c418-0c70-4871-bea9-22de7835e94a","Type":"ContainerStarted","Data":"f4c2e1edf6a87c29592488a0e4a50e5b2da3013d4857509f76da77b187cc00bc"}
Jul 22 16:08:04.801418 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:04.801392    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hnxj2" event={"ID":"cec619e3-7150-417b-ab93-67b2161ca87c","Type":"ContainerStarted","Data":"05bb1d20bd067cdbca3b38cb4d1938a13d126743ee8d2c1749ee14d0a29c0ff0"}
Jul 22 16:08:04.809717 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:04.809695    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vmgf6_196426fb-47a2-47d8-8a89-094572fbe64e/serve-healthcheck-canary/0.log"
Jul 22 16:08:04.822096 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:04.822043    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hnxj2" podStartSLOduration=1.869267728 podStartE2EDuration="3.82202813s" totalImagesPullingTime="1.952760402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:08:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:08:04.820846028 +0000 UTC m=+44.951316049" watchObservedRunningTime="2026-07-22 16:08:04.82202813 +0000 UTC m=+44.952498189"
Jul 22 16:08:04.961340 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:04.961303    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:04.988424 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:04.988393    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9c8214fa-bb50-454f-a84b-558b13bd799e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ntb4r\" (UID: \"9c8214fa-bb50-454f-a84b-558b13bd799e\") " pod="openshift-insights/insights-runtime-extractor-ntb4r"
Jul 22 16:08:04.988606 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:04.988535    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9c8214fa-bb50-454f-a84b-558b13bd799e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ntb4r\" (UID: \"9c8214fa-bb50-454f-a84b-558b13bd799e\") " pod="openshift-insights/insights-runtime-extractor-ntb4r"
Jul 22 16:08:04.988606 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:08:04.988602    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 16:08:04.988702 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:08:04.988650    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 16:08:04.988702 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:08:04.988654    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9c8214fa-bb50-454f-a84b-558b13bd799e-kube-rbac-proxy-cm podName:9c8214fa-bb50-454f-a84b-558b13bd799e nodeName:}" failed. No retries permitted until 2026-07-22 16:08:08.988641035 +0000 UTC m=+49.119111018 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9c8214fa-bb50-454f-a84b-558b13bd799e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ntb4r" (UID: "9c8214fa-bb50-454f-a84b-558b13bd799e") : configmap "kube-rbac-proxy" not found
Jul 22 16:08:04.988795 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:08:04.988711    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c8214fa-bb50-454f-a84b-558b13bd799e-insights-runtime-extractor-tls podName:9c8214fa-bb50-454f-a84b-558b13bd799e nodeName:}" failed. No retries permitted until 2026-07-22 16:08:08.9886952 +0000 UTC m=+49.119165180 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9c8214fa-bb50-454f-a84b-558b13bd799e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ntb4r" (UID: "9c8214fa-bb50-454f-a84b-558b13bd799e") : secret "insights-runtime-extractor-tls" not found
Jul 22 16:08:06.767591 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:06.767561    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:07.810202 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:07.810161    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-bjsjv" event={"ID":"8ab8c418-0c70-4871-bea9-22de7835e94a","Type":"ContainerStarted","Data":"f3e32995d3f9d9e5517f47eb3f4a27973f47baffd8a645ae87ed12cd5178d401"}
Jul 22 16:08:07.830298 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:07.830247    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-bjsjv" podStartSLOduration=2.287152463 podStartE2EDuration="4.830232763s" totalImagesPullingTime="2.5430803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:08:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:08:07.829378052 +0000 UTC m=+47.959848053" watchObservedRunningTime="2026-07-22 16:08:07.830232763 +0000 UTC m=+47.960702819"
Jul 22 16:08:08.261562 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:08.261527    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:08.424390 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:08.424346    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:08.547171 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:08.547072    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:08.619829 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:08.619785    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:08.805031 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:08.804941    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:08.931485 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:08.931451    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:09.023073 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:09.023040    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9c8214fa-bb50-454f-a84b-558b13bd799e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ntb4r\" (UID: \"9c8214fa-bb50-454f-a84b-558b13bd799e\") " pod="openshift-insights/insights-runtime-extractor-ntb4r"
Jul 22 16:08:09.023244 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:09.023095    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9c8214fa-bb50-454f-a84b-558b13bd799e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ntb4r\" (UID: \"9c8214fa-bb50-454f-a84b-558b13bd799e\") " pod="openshift-insights/insights-runtime-extractor-ntb4r"
Jul 22 16:08:09.023244 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:08:09.023192    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 16:08:09.023321 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:08:09.023249    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c8214fa-bb50-454f-a84b-558b13bd799e-insights-runtime-extractor-tls podName:9c8214fa-bb50-454f-a84b-558b13bd799e nodeName:}" failed. No retries permitted until 2026-07-22 16:08:17.023235573 +0000 UTC m=+57.153705554 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9c8214fa-bb50-454f-a84b-558b13bd799e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ntb4r" (UID: "9c8214fa-bb50-454f-a84b-558b13bd799e") : secret "insights-runtime-extractor-tls" not found
Jul 22 16:08:09.023675 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:09.023651    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9c8214fa-bb50-454f-a84b-558b13bd799e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ntb4r\" (UID: \"9c8214fa-bb50-454f-a84b-558b13bd799e\") " pod="openshift-insights/insights-runtime-extractor-ntb4r"
Jul 22 16:08:09.045842 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:09.045813    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:09.146783 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:09.146700    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:09.765795 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:09.765762    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:10.378309 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:10.378276    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:10.574896 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:10.574845    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:10.768024 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:10.767991    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:10.938518 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:10.938480    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08a6a730-eb43-4bde-9079-47c469793f1d-metrics-tls\") pod \"dns-default-xdxv8\" (UID: \"08a6a730-eb43-4bde-9079-47c469793f1d\") " pod="openshift-dns/dns-default-xdxv8"
Jul 22 16:08:10.938712 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:08:10.938629    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 16:08:10.938712 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:08:10.938700    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08a6a730-eb43-4bde-9079-47c469793f1d-metrics-tls podName:08a6a730-eb43-4bde-9079-47c469793f1d nodeName:}" failed. No retries permitted until 2026-07-22 16:08:26.938683268 +0000 UTC m=+67.069153267 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08a6a730-eb43-4bde-9079-47c469793f1d-metrics-tls") pod "dns-default-xdxv8" (UID: "08a6a730-eb43-4bde-9079-47c469793f1d") : secret "dns-default-metrics-tls" not found
Jul 22 16:08:11.159239 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:11.159146    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:11.567748 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:11.567717    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:11.762457 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:11.762418    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:11.955322 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:11.955297    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:12.362177 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:12.362101    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:12.422115 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:12.422087    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:12.558000 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:12.557970    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:12.960358 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:12.960317    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:13.175473 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:13.175437    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:13.558923 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:13.558884    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:13.957902 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:13.957877    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:14.590709 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:14.590678    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:14.660500 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:14.660467    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:14.809227 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:14.809195    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:14.966419 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:14.966384    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:15.357978 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:15.357906    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:15.560679 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:15.560647    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:15.962667 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:15.962637    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:16.753538 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:16.753504    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:16.820085 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:16.820057    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:17.084503 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:17.084417    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9c8214fa-bb50-454f-a84b-558b13bd799e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ntb4r\" (UID: \"9c8214fa-bb50-454f-a84b-558b13bd799e\") " pod="openshift-insights/insights-runtime-extractor-ntb4r"
Jul 22 16:08:17.086706 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:17.086688    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9c8214fa-bb50-454f-a84b-558b13bd799e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ntb4r\" (UID: \"9c8214fa-bb50-454f-a84b-558b13bd799e\") " pod="openshift-insights/insights-runtime-extractor-ntb4r"
Jul 22 16:08:17.098680 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:17.098652    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ntb4r"
Jul 22 16:08:17.196338 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:17.196275    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:17.273800 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:17.273765    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ntb4r"]
Jul 22 16:08:17.279660 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:08:17.279637    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c8214fa_bb50_454f_a84b_558b13bd799e.slice/crio-e7ad0f8c147dd3a4087cfca52d0e060555f4131a044a5bf422f061ac832ca429 WatchSource:0}: Error finding container e7ad0f8c147dd3a4087cfca52d0e060555f4131a044a5bf422f061ac832ca429: Status 404 returned error can't find the container with id e7ad0f8c147dd3a4087cfca52d0e060555f4131a044a5bf422f061ac832ca429
Jul 22 16:08:17.396697 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:17.396616    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:17.585223 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:17.585191    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:17.774477 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:17.774441    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:17.837296 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:17.837255    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ntb4r" event={"ID":"9c8214fa-bb50-454f-a84b-558b13bd799e","Type":"ContainerStarted","Data":"e7ad0f8c147dd3a4087cfca52d0e060555f4131a044a5bf422f061ac832ca429"}
Jul 22 16:08:18.572140 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:18.572101    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:18.781531 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:18.781492    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:18.988499 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:18.988465    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:20.757746 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:20.757711    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2djqd"
Jul 22 16:08:22.851769 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:22.851735    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ntb4r" event={"ID":"9c8214fa-bb50-454f-a84b-558b13bd799e","Type":"ContainerStarted","Data":"07a74fd6dcbeeedca4a13bf93abeec4ecb9de9deae7dd530e6e90d0bb201c1f9"}
Jul 22 16:08:24.865296 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:24.865263    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-585444ddf-rk5wc"]
Jul 22 16:08:24.879607 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:24.879573    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-585444ddf-rk5wc"
Jul 22 16:08:24.884749 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:24.884660    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 22 16:08:24.884749 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:24.884688    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 22 16:08:24.884749 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:24.884706    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 22 16:08:24.884749 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:24.884723    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 22 16:08:24.885124 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:24.884664    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 22 16:08:24.885124 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:24.884915    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 22 16:08:24.886279 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:24.886256    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 22 16:08:24.912556 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:24.912508    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-585444ddf-rk5wc"]
Jul 22 16:08:24.948293 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:24.948253    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxfgw\" (UniqueName: \"kubernetes.io/projected/1fe5655e-89d1-4695-bd8b-c75821f01afc-kube-api-access-dxfgw\") pod \"cluster-proxy-proxy-agent-585444ddf-rk5wc\" (UID: \"1fe5655e-89d1-4695-bd8b-c75821f01afc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-585444ddf-rk5wc"
Jul 22 16:08:24.948492 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:24.948361    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/1fe5655e-89d1-4695-bd8b-c75821f01afc-hub\") pod \"cluster-proxy-proxy-agent-585444ddf-rk5wc\" (UID: \"1fe5655e-89d1-4695-bd8b-c75821f01afc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-585444ddf-rk5wc"
Jul 22 16:08:24.948492 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:24.948427    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fe5655e-89d1-4695-bd8b-c75821f01afc-additional-service-ca\") pod \"cluster-proxy-proxy-agent-585444ddf-rk5wc\" (UID: \"1fe5655e-89d1-4695-bd8b-c75821f01afc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-585444ddf-rk5wc"
Jul 22 16:08:24.948492 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:24.948460    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/1fe5655e-89d1-4695-bd8b-c75821f01afc-ca\") pod \"cluster-proxy-proxy-agent-585444ddf-rk5wc\" (UID: \"1fe5655e-89d1-4695-bd8b-c75821f01afc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-585444ddf-rk5wc"
Jul 22 16:08:24.948492 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:24.948481    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/1fe5655e-89d1-4695-bd8b-c75821f01afc-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-585444ddf-rk5wc\" (UID: \"1fe5655e-89d1-4695-bd8b-c75821f01afc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-585444ddf-rk5wc"
Jul 22 16:08:24.948702 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:24.948516    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1fe5655e-89d1-4695-bd8b-c75821f01afc-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-585444ddf-rk5wc\" (UID: \"1fe5655e-89d1-4695-bd8b-c75821f01afc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-585444ddf-rk5wc"
Jul 22 16:08:25.049592 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:25.049554    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fe5655e-89d1-4695-bd8b-c75821f01afc-additional-service-ca\") pod \"cluster-proxy-proxy-agent-585444ddf-rk5wc\" (UID: \"1fe5655e-89d1-4695-bd8b-c75821f01afc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-585444ddf-rk5wc"
Jul 22 16:08:25.049592 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:25.049602    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/1fe5655e-89d1-4695-bd8b-c75821f01afc-ca\") pod \"cluster-proxy-proxy-agent-585444ddf-rk5wc\" (UID: \"1fe5655e-89d1-4695-bd8b-c75821f01afc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-585444ddf-rk5wc"
Jul 22 16:08:25.049834 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:25.049635    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/1fe5655e-89d1-4695-bd8b-c75821f01afc-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-585444ddf-rk5wc\" (UID: \"1fe5655e-89d1-4695-bd8b-c75821f01afc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-585444ddf-rk5wc"
Jul 22 16:08:25.049834 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:25.049666    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1fe5655e-89d1-4695-bd8b-c75821f01afc-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-585444ddf-rk5wc\" (UID: \"1fe5655e-89d1-4695-bd8b-c75821f01afc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-585444ddf-rk5wc"
Jul 22 16:08:25.049834 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:25.049700    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxfgw\" (UniqueName: \"kubernetes.io/projected/1fe5655e-89d1-4695-bd8b-c75821f01afc-kube-api-access-dxfgw\") pod \"cluster-proxy-proxy-agent-585444ddf-rk5wc\" (UID: \"1fe5655e-89d1-4695-bd8b-c75821f01afc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-585444ddf-rk5wc"
Jul 22 16:08:25.049834 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:25.049760    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/1fe5655e-89d1-4695-bd8b-c75821f01afc-hub\") pod \"cluster-proxy-proxy-agent-585444ddf-rk5wc\" (UID: \"1fe5655e-89d1-4695-bd8b-c75821f01afc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-585444ddf-rk5wc"
Jul 22 16:08:25.050484 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:25.050431    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fe5655e-89d1-4695-bd8b-c75821f01afc-additional-service-ca\") pod \"cluster-proxy-proxy-agent-585444ddf-rk5wc\" (UID: \"1fe5655e-89d1-4695-bd8b-c75821f01afc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-585444ddf-rk5wc"
Jul 22 16:08:25.052738 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:25.052712    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/1fe5655e-89d1-4695-bd8b-c75821f01afc-hub\") pod \"cluster-proxy-proxy-agent-585444ddf-rk5wc\" (UID: \"1fe5655e-89d1-4695-bd8b-c75821f01afc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-585444ddf-rk5wc"
Jul 22 16:08:25.052934 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:25.052907    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/1fe5655e-89d1-4695-bd8b-c75821f01afc-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-585444ddf-rk5wc\" (UID: \"1fe5655e-89d1-4695-bd8b-c75821f01afc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-585444ddf-rk5wc"
Jul 22 16:08:25.053050 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:25.053038    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/1fe5655e-89d1-4695-bd8b-c75821f01afc-ca\") pod \"cluster-proxy-proxy-agent-585444ddf-rk5wc\" (UID: \"1fe5655e-89d1-4695-bd8b-c75821f01afc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-585444ddf-rk5wc"
Jul 22 16:08:25.053408 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:25.053387    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1fe5655e-89d1-4695-bd8b-c75821f01afc-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-585444ddf-rk5wc\" (UID: \"1fe5655e-89d1-4695-bd8b-c75821f01afc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-585444ddf-rk5wc"
Jul 22 16:08:25.056921 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:25.056891    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:25.068111 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:25.068088    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxfgw\" (UniqueName: \"kubernetes.io/projected/1fe5655e-89d1-4695-bd8b-c75821f01afc-kube-api-access-dxfgw\") pod \"cluster-proxy-proxy-agent-585444ddf-rk5wc\" (UID: \"1fe5655e-89d1-4695-bd8b-c75821f01afc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-585444ddf-rk5wc"
Jul 22 16:08:25.150024 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:25.149924    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:25.197542 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:25.197507    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-585444ddf-rk5wc"
Jul 22 16:08:25.251702 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:25.251673    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:25.353074 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:25.353031    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:25.451347 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:25.451292    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:26.058085 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:26.058054    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g44rx\" (UniqueName: \"kubernetes.io/projected/36f0cd2c-f5c3-47ce-bb45-0f604e33aed9-kube-api-access-g44rx\") pod \"network-check-target-kq5l6\" (UID: \"36f0cd2c-f5c3-47ce-bb45-0f604e33aed9\") " pod="openshift-network-diagnostics/network-check-target-kq5l6"
Jul 22 16:08:26.060962 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:26.060941    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 22 16:08:26.072285 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:26.072262    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 22 16:08:26.082442 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:26.082393    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g44rx\" (UniqueName: \"kubernetes.io/projected/36f0cd2c-f5c3-47ce-bb45-0f604e33aed9-kube-api-access-g44rx\") pod \"network-check-target-kq5l6\" (UID: \"36f0cd2c-f5c3-47ce-bb45-0f604e33aed9\") " pod="openshift-network-diagnostics/network-check-target-kq5l6"
Jul 22 16:08:26.108669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:26.107034    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-585444ddf-rk5wc"]
Jul 22 16:08:26.159257 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:26.159214    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6440de1a-aff3-4b9d-a374-4e029ece2bfe-metrics-certs\") pod \"network-metrics-daemon-87ltq\" (UID: \"6440de1a-aff3-4b9d-a374-4e029ece2bfe\") " pod="openshift-multus/network-metrics-daemon-87ltq"
Jul 22 16:08:26.159428 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:26.159283    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1cd85e6d-9e7c-4c21-bc75-98cc25975bfc-original-pull-secret\") pod \"global-pull-secret-syncer-5dhg6\" (UID: \"1cd85e6d-9e7c-4c21-bc75-98cc25975bfc\") " pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:08:26.162181 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:26.162163    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 22 16:08:26.162226 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:26.162201    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 22 16:08:26.171647 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:26.171617    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6440de1a-aff3-4b9d-a374-4e029ece2bfe-metrics-certs\") pod \"network-metrics-daemon-87ltq\" (UID: \"6440de1a-aff3-4b9d-a374-4e029ece2bfe\") " pod="openshift-multus/network-metrics-daemon-87ltq"
Jul 22 16:08:26.171763 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:26.171628    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1cd85e6d-9e7c-4c21-bc75-98cc25975bfc-original-pull-secret\") pod \"global-pull-secret-syncer-5dhg6\" (UID: \"1cd85e6d-9e7c-4c21-bc75-98cc25975bfc\") " pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:08:26.355459 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:26.355374    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5dhg6"
Jul 22 16:08:26.366191 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:26.366164    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-s68m6\""
Jul 22 16:08:26.371774 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:26.371754    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rb5n6\""
Jul 22 16:08:26.373871 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:26.373831    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kq5l6"
Jul 22 16:08:26.380497 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:26.380469    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ltq"
Jul 22 16:08:26.506018 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:26.505987    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5dhg6"]
Jul 22 16:08:26.506163 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:08:26.506026    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cd85e6d_9e7c_4c21_bc75_98cc25975bfc.slice/crio-5743fb64aec76f5391aa6e6c695ccbcace6d82486a4fcd3afec9bddb9c5aebfc WatchSource:0}: Error finding container 5743fb64aec76f5391aa6e6c695ccbcace6d82486a4fcd3afec9bddb9c5aebfc: Status 404 returned error can't find the container with id 5743fb64aec76f5391aa6e6c695ccbcace6d82486a4fcd3afec9bddb9c5aebfc
Jul 22 16:08:26.519946 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:26.519920    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kq5l6"]
Jul 22 16:08:26.522917 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:08:26.522884    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36f0cd2c_f5c3_47ce_bb45_0f604e33aed9.slice/crio-e0f5375ffe1dd525374b18c9eb7092c72c857b6e9d0318bb158c5fe7f55005a7 WatchSource:0}: Error finding container e0f5375ffe1dd525374b18c9eb7092c72c857b6e9d0318bb158c5fe7f55005a7: Status 404 returned error can't find the container with id e0f5375ffe1dd525374b18c9eb7092c72c857b6e9d0318bb158c5fe7f55005a7
Jul 22 16:08:26.536955 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:26.536927    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-87ltq"]
Jul 22 16:08:26.539645 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:08:26.539618    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6440de1a_aff3_4b9d_a374_4e029ece2bfe.slice/crio-212345c710195c651f89c0a1e7447b57d96c1650da4f53cb18c9219a79ba945d WatchSource:0}: Error finding container 212345c710195c651f89c0a1e7447b57d96c1650da4f53cb18c9219a79ba945d: Status 404 returned error can't find the container with id 212345c710195c651f89c0a1e7447b57d96c1650da4f53cb18c9219a79ba945d
Jul 22 16:08:26.865375 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:26.865303    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kq5l6" event={"ID":"36f0cd2c-f5c3-47ce-bb45-0f604e33aed9","Type":"ContainerStarted","Data":"e0f5375ffe1dd525374b18c9eb7092c72c857b6e9d0318bb158c5fe7f55005a7"}
Jul 22 16:08:26.866620 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:26.866583    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5dhg6" event={"ID":"1cd85e6d-9e7c-4c21-bc75-98cc25975bfc","Type":"ContainerStarted","Data":"5743fb64aec76f5391aa6e6c695ccbcace6d82486a4fcd3afec9bddb9c5aebfc"}
Jul 22 16:08:26.867918 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:26.867889    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-585444ddf-rk5wc" event={"ID":"1fe5655e-89d1-4695-bd8b-c75821f01afc","Type":"ContainerStarted","Data":"436e2f27b9c945e7cd777c8544a398b74108a97340107952ca9feb58341bcda8"}
Jul 22 16:08:26.869294 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:26.869247    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-87ltq" event={"ID":"6440de1a-aff3-4b9d-a374-4e029ece2bfe","Type":"ContainerStarted","Data":"212345c710195c651f89c0a1e7447b57d96c1650da4f53cb18c9219a79ba945d"}
Jul 22 16:08:26.871729 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:26.871699    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ntb4r" event={"ID":"9c8214fa-bb50-454f-a84b-558b13bd799e","Type":"ContainerStarted","Data":"46540af1f63eb74e3792a7d6e25c5cbd34000b764131fe7a9bd4dff3ed3245cc"}
Jul 22 16:08:26.964863 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:26.964812    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08a6a730-eb43-4bde-9079-47c469793f1d-metrics-tls\") pod \"dns-default-xdxv8\" (UID: \"08a6a730-eb43-4bde-9079-47c469793f1d\") " pod="openshift-dns/dns-default-xdxv8"
Jul 22 16:08:26.967795 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:26.967771    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08a6a730-eb43-4bde-9079-47c469793f1d-metrics-tls\") pod \"dns-default-xdxv8\" (UID: \"08a6a730-eb43-4bde-9079-47c469793f1d\") " pod="openshift-dns/dns-default-xdxv8"
Jul 22 16:08:27.235209 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:27.235177    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wdbjp\""
Jul 22 16:08:27.243030 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:27.242998    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xdxv8"
Jul 22 16:08:27.421197 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:27.419897    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xdxv8"]
Jul 22 16:08:27.878189 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:27.878151    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xdxv8" event={"ID":"08a6a730-eb43-4bde-9079-47c469793f1d","Type":"ContainerStarted","Data":"ebca868714c2a1ce4f4c3d90f82c8082ad499c1794e14cfc0d444de3af837b3c"}
Jul 22 16:08:28.886471 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:28.886145    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-87ltq" event={"ID":"6440de1a-aff3-4b9d-a374-4e029ece2bfe","Type":"ContainerStarted","Data":"9ca3632c0fb289b1fa7afd9d6cc446d2cee5f4ebf8a37a73a8ae33f24df59e61"}
Jul 22 16:08:28.886471 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:28.886214    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-87ltq" event={"ID":"6440de1a-aff3-4b9d-a374-4e029ece2bfe","Type":"ContainerStarted","Data":"a1f1e46e9a0e4a89ec6826f54b7e490cd9a156c95ab1eb128ef615d7a4ec8616"}
Jul 22 16:08:28.914111 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:28.914048    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-87ltq" podStartSLOduration=67.628015056 podStartE2EDuration="1m8.914024024s" totalImagesPullingTime="1.286008968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:07:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:08:28.913830309 +0000 UTC m=+69.044300335" watchObservedRunningTime="2026-07-22 16:08:28.914024024 +0000 UTC m=+69.044494028"
Jul 22 16:08:29.363107 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:29.363013    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:29.539116 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:29.539083    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:29.672934 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:29.672836    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:29.770947 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:29.770907    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:31.517475 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:31.517434    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:31.623885 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:31.623831    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:31.726138 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:31.726103    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:31.941716 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:31.941678    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:34.909177 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:34.909134    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xdxv8" event={"ID":"08a6a730-eb43-4bde-9079-47c469793f1d","Type":"ContainerStarted","Data":"72cb2d94afab0d249ed1826d1527eae082891555ff5f148f921ba3b0991f87a1"}
Jul 22 16:08:34.909685 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:34.909198    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xdxv8" event={"ID":"08a6a730-eb43-4bde-9079-47c469793f1d","Type":"ContainerStarted","Data":"a714c02af5dadabf01491aa28f36c27eab89fd62ac7e4d932693adddfaa781eb"}
Jul 22 16:08:34.909685 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:34.909276    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-xdxv8"
Jul 22 16:08:34.911002 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:34.910972    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kq5l6" event={"ID":"36f0cd2c-f5c3-47ce-bb45-0f604e33aed9","Type":"ContainerStarted","Data":"24bf889879f8b4edc0a8eb9d7cb448f2d3b19d1417500474e025d9bf5ada51db"}
Jul 22 16:08:34.911145 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:34.911038    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kq5l6"
Jul 22 16:08:34.912680 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:34.912651    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5dhg6" event={"ID":"1cd85e6d-9e7c-4c21-bc75-98cc25975bfc","Type":"ContainerStarted","Data":"731b9edb0c4370d0502da2b39d80b134142d8d5560c57397ddc0b087cc9cf50e"}
Jul 22 16:08:34.915064 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:34.915040    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ntb4r" event={"ID":"9c8214fa-bb50-454f-a84b-558b13bd799e","Type":"ContainerStarted","Data":"1d7de0c3c1f01feda74ce565b65a683c078406ecd406eb8c9feb55e686d25d21"}
Jul 22 16:08:34.933351 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:34.933305    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-xdxv8" podStartSLOduration=33.351602457 podStartE2EDuration="39.933290891s" totalImagesPullingTime="6.581688434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:07:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:08:34.931656696 +0000 UTC m=+75.062126732" watchObservedRunningTime="2026-07-22 16:08:34.933290891 +0000 UTC m=+75.063760892"
Jul 22 16:08:34.951521 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:34.951467    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ntb4r" podStartSLOduration=17.349178372 podStartE2EDuration="33.95144902s" totalImagesPullingTime="16.602270648s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:08:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:08:34.951113022 +0000 UTC m=+75.081583025" watchObservedRunningTime="2026-07-22 16:08:34.95144902 +0000 UTC m=+75.081919039"
Jul 22 16:08:34.968500 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:34.968444    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-5dhg6" podStartSLOduration=67.470949181 podStartE2EDuration="1m14.96842233s" totalImagesPullingTime="7.497473149s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:07:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:08:34.967946454 +0000 UTC m=+75.098416456" watchObservedRunningTime="2026-07-22 16:08:34.96842233 +0000 UTC m=+75.098892334"
Jul 22 16:08:34.985531 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:34.985450    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kq5l6" podStartSLOduration=67.504134164 podStartE2EDuration="1m14.985427902s" totalImagesPullingTime="7.481293738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:07:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:08:34.984981086 +0000 UTC m=+75.115451089" watchObservedRunningTime="2026-07-22 16:08:34.985427902 +0000 UTC m=+75.115897906"
Jul 22 16:08:36.656404 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:36.656372    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:36.787324 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:36.787215    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:36.924186 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:36.924151    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-585444ddf-rk5wc" event={"ID":"1fe5655e-89d1-4695-bd8b-c75821f01afc","Type":"ContainerStarted","Data":"f94e4e906d86a944e00f3ab1a36599215a144dc5f4db248b34d9978813bdeb27"}
Jul 22 16:08:36.924359 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:36.924196    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-585444ddf-rk5wc" event={"ID":"1fe5655e-89d1-4695-bd8b-c75821f01afc","Type":"ContainerStarted","Data":"cfcc38ffacd4b1e1c243bc4fba3efbbce02cf9adf9cc4440067bbc41827f07c3"}
Jul 22 16:08:36.924359 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:36.924213    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-585444ddf-rk5wc" event={"ID":"1fe5655e-89d1-4695-bd8b-c75821f01afc","Type":"ContainerStarted","Data":"021fd7ce7e36513903d511d24a472e5be24f97f18ecd48d00d7bd17fad6caa56"}
Jul 22 16:08:36.949366 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:36.949316    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-585444ddf-rk5wc" podStartSLOduration=2.534096824 podStartE2EDuration="12.949300447s" totalImagesPullingTime="10.415203623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:08:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:08:36.949062314 +0000 UTC m=+77.079532316" watchObservedRunningTime="2026-07-22 16:08:36.949300447 +0000 UTC m=+77.079770449"
Jul 22 16:08:37.721380 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:37.721347    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:37.794487 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:37.794456    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:37.849998 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:37.849970    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:39.630636 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:39.630595    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:41.895194 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:41.895072    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:42.107575 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.107535    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:42.119054 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.119017    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-pcfcd"]
Jul 22 16:08:42.121481 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.121456    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-pcfcd"
Jul 22 16:08:42.128121 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.128064    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 22 16:08:42.128257 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.128165    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 22 16:08:42.128351 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.128335    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 22 16:08:42.128470 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.128453    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-4cxl7\""
Jul 22 16:08:42.128686 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.128666    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 22 16:08:42.128870 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.128841    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 22 16:08:42.135708 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.135681    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-pcfcd"]
Jul 22 16:08:42.163048 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.162974    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-h6kg9"]
Jul 22 16:08:42.165189 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.165169    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h6kg9"
Jul 22 16:08:42.168107 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.168068    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jqfxs\""
Jul 22 16:08:42.168331 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.168313    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 22 16:08:42.168430 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.168312    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 22 16:08:42.168543 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.168527    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 22 16:08:42.180895 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.180835    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a1d3f28a-7b56-4185-b5af-6bad89156abc-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-pcfcd\" (UID: \"a1d3f28a-7b56-4185-b5af-6bad89156abc\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-pcfcd"
Jul 22 16:08:42.181013 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.180950    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1d3f28a-7b56-4185-b5af-6bad89156abc-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-pcfcd\" (UID: \"a1d3f28a-7b56-4185-b5af-6bad89156abc\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-pcfcd"
Jul 22 16:08:42.181013 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.180995    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1d3f28a-7b56-4185-b5af-6bad89156abc-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-pcfcd\" (UID: \"a1d3f28a-7b56-4185-b5af-6bad89156abc\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-pcfcd"
Jul 22 16:08:42.181121 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.181034    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b77dv\" (UniqueName: \"kubernetes.io/projected/a1d3f28a-7b56-4185-b5af-6bad89156abc-kube-api-access-b77dv\") pod \"openshift-state-metrics-79f6958cdf-pcfcd\" (UID: \"a1d3f28a-7b56-4185-b5af-6bad89156abc\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-pcfcd"
Jul 22 16:08:42.199422 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.199396    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:42.264513 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.264477    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:42.281873 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.281789    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1d3f28a-7b56-4185-b5af-6bad89156abc-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-pcfcd\" (UID: \"a1d3f28a-7b56-4185-b5af-6bad89156abc\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-pcfcd"
Jul 22 16:08:42.282010 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.281906    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1d3f28a-7b56-4185-b5af-6bad89156abc-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-pcfcd\" (UID: \"a1d3f28a-7b56-4185-b5af-6bad89156abc\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-pcfcd"
Jul 22 16:08:42.282010 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.281931    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a1d3f28a-7b56-4185-b5af-6bad89156abc-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-pcfcd\" (UID: \"a1d3f28a-7b56-4185-b5af-6bad89156abc\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-pcfcd"
Jul 22 16:08:42.282010 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.281961    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c7280975-6841-4d25-82f2-18702e8cf8ef-node-exporter-wtmp\") pod \"node-exporter-h6kg9\" (UID: \"c7280975-6841-4d25-82f2-18702e8cf8ef\") " pod="openshift-monitoring/node-exporter-h6kg9"
Jul 22 16:08:42.282010 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.281986    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvdf8\" (UniqueName: \"kubernetes.io/projected/c7280975-6841-4d25-82f2-18702e8cf8ef-kube-api-access-vvdf8\") pod \"node-exporter-h6kg9\" (UID: \"c7280975-6841-4d25-82f2-18702e8cf8ef\") " pod="openshift-monitoring/node-exporter-h6kg9"
Jul 22 16:08:42.282210 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.282011    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c7280975-6841-4d25-82f2-18702e8cf8ef-metrics-client-ca\") pod \"node-exporter-h6kg9\" (UID: \"c7280975-6841-4d25-82f2-18702e8cf8ef\") " pod="openshift-monitoring/node-exporter-h6kg9"
Jul 22 16:08:42.282210 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.282083    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c7280975-6841-4d25-82f2-18702e8cf8ef-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h6kg9\" (UID: \"c7280975-6841-4d25-82f2-18702e8cf8ef\") " pod="openshift-monitoring/node-exporter-h6kg9"
Jul 22 16:08:42.282210 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.282129    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c7280975-6841-4d25-82f2-18702e8cf8ef-node-exporter-textfile\") pod \"node-exporter-h6kg9\" (UID: \"c7280975-6841-4d25-82f2-18702e8cf8ef\") " pod="openshift-monitoring/node-exporter-h6kg9"
Jul 22 16:08:42.282360 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.282217    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c7280975-6841-4d25-82f2-18702e8cf8ef-node-exporter-accelerators-collector-config\") pod \"node-exporter-h6kg9\" (UID: \"c7280975-6841-4d25-82f2-18702e8cf8ef\") " pod="openshift-monitoring/node-exporter-h6kg9"
Jul 22 16:08:42.282360 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.282248    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c7280975-6841-4d25-82f2-18702e8cf8ef-root\") pod \"node-exporter-h6kg9\" (UID: \"c7280975-6841-4d25-82f2-18702e8cf8ef\") " pod="openshift-monitoring/node-exporter-h6kg9"
Jul 22 16:08:42.282360 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.282280    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b77dv\" (UniqueName: \"kubernetes.io/projected/a1d3f28a-7b56-4185-b5af-6bad89156abc-kube-api-access-b77dv\") pod \"openshift-state-metrics-79f6958cdf-pcfcd\" (UID: \"a1d3f28a-7b56-4185-b5af-6bad89156abc\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-pcfcd"
Jul 22 16:08:42.282360 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.282340    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7280975-6841-4d25-82f2-18702e8cf8ef-sys\") pod \"node-exporter-h6kg9\" (UID: \"c7280975-6841-4d25-82f2-18702e8cf8ef\") " pod="openshift-monitoring/node-exporter-h6kg9"
Jul 22 16:08:42.282539 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.282384    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c7280975-6841-4d25-82f2-18702e8cf8ef-node-exporter-tls\") pod \"node-exporter-h6kg9\" (UID: \"c7280975-6841-4d25-82f2-18702e8cf8ef\") " pod="openshift-monitoring/node-exporter-h6kg9"
Jul 22 16:08:42.282587 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.282566    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1d3f28a-7b56-4185-b5af-6bad89156abc-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-pcfcd\" (UID: \"a1d3f28a-7b56-4185-b5af-6bad89156abc\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-pcfcd"
Jul 22 16:08:42.284435 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.284408    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1d3f28a-7b56-4185-b5af-6bad89156abc-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-pcfcd\" (UID: \"a1d3f28a-7b56-4185-b5af-6bad89156abc\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-pcfcd"
Jul 22 16:08:42.284435 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.284422    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a1d3f28a-7b56-4185-b5af-6bad89156abc-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-pcfcd\" (UID: \"a1d3f28a-7b56-4185-b5af-6bad89156abc\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-pcfcd"
Jul 22 16:08:42.292756 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.292734    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b77dv\" (UniqueName: \"kubernetes.io/projected/a1d3f28a-7b56-4185-b5af-6bad89156abc-kube-api-access-b77dv\") pod \"openshift-state-metrics-79f6958cdf-pcfcd\" (UID: \"a1d3f28a-7b56-4185-b5af-6bad89156abc\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-pcfcd"
Jul 22 16:08:42.374275 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.374242    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:42.382966 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.382943    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c7280975-6841-4d25-82f2-18702e8cf8ef-root\") pod \"node-exporter-h6kg9\" (UID: \"c7280975-6841-4d25-82f2-18702e8cf8ef\") " pod="openshift-monitoring/node-exporter-h6kg9"
Jul 22 16:08:42.383067 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.382972    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7280975-6841-4d25-82f2-18702e8cf8ef-sys\") pod \"node-exporter-h6kg9\" (UID: \"c7280975-6841-4d25-82f2-18702e8cf8ef\") " pod="openshift-monitoring/node-exporter-h6kg9"
Jul 22 16:08:42.383067 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.382989    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c7280975-6841-4d25-82f2-18702e8cf8ef-node-exporter-tls\") pod \"node-exporter-h6kg9\" (UID: \"c7280975-6841-4d25-82f2-18702e8cf8ef\") " pod="openshift-monitoring/node-exporter-h6kg9"
Jul 22 16:08:42.383067 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.383017    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c7280975-6841-4d25-82f2-18702e8cf8ef-node-exporter-wtmp\") pod \"node-exporter-h6kg9\" (UID: \"c7280975-6841-4d25-82f2-18702e8cf8ef\") " pod="openshift-monitoring/node-exporter-h6kg9"
Jul 22 16:08:42.383067 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.383052    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c7280975-6841-4d25-82f2-18702e8cf8ef-root\") pod \"node-exporter-h6kg9\" (UID: \"c7280975-6841-4d25-82f2-18702e8cf8ef\") " pod="openshift-monitoring/node-exporter-h6kg9"
Jul 22 16:08:42.383233 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.383069    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvdf8\" (UniqueName: \"kubernetes.io/projected/c7280975-6841-4d25-82f2-18702e8cf8ef-kube-api-access-vvdf8\") pod \"node-exporter-h6kg9\" (UID: \"c7280975-6841-4d25-82f2-18702e8cf8ef\") " pod="openshift-monitoring/node-exporter-h6kg9"
Jul 22 16:08:42.383233 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.383069    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7280975-6841-4d25-82f2-18702e8cf8ef-sys\") pod \"node-exporter-h6kg9\" (UID: \"c7280975-6841-4d25-82f2-18702e8cf8ef\") " pod="openshift-monitoring/node-exporter-h6kg9"
Jul 22 16:08:42.383233 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.383104    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c7280975-6841-4d25-82f2-18702e8cf8ef-metrics-client-ca\") pod \"node-exporter-h6kg9\" (UID: \"c7280975-6841-4d25-82f2-18702e8cf8ef\") " pod="openshift-monitoring/node-exporter-h6kg9"
Jul 22 16:08:42.383233 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.383129    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c7280975-6841-4d25-82f2-18702e8cf8ef-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h6kg9\" (UID: \"c7280975-6841-4d25-82f2-18702e8cf8ef\") " pod="openshift-monitoring/node-exporter-h6kg9"
Jul 22 16:08:42.383233 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.383149    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c7280975-6841-4d25-82f2-18702e8cf8ef-node-exporter-textfile\") pod \"node-exporter-h6kg9\" (UID: \"c7280975-6841-4d25-82f2-18702e8cf8ef\") " pod="openshift-monitoring/node-exporter-h6kg9"
Jul 22 16:08:42.383233 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.383179    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c7280975-6841-4d25-82f2-18702e8cf8ef-node-exporter-wtmp\") pod \"node-exporter-h6kg9\" (UID: \"c7280975-6841-4d25-82f2-18702e8cf8ef\") " pod="openshift-monitoring/node-exporter-h6kg9"
Jul 22 16:08:42.383233 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.383209    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c7280975-6841-4d25-82f2-18702e8cf8ef-node-exporter-accelerators-collector-config\") pod \"node-exporter-h6kg9\" (UID: \"c7280975-6841-4d25-82f2-18702e8cf8ef\") " pod="openshift-monitoring/node-exporter-h6kg9"
Jul 22 16:08:42.383667 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.383638    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c7280975-6841-4d25-82f2-18702e8cf8ef-node-exporter-textfile\") pod \"node-exporter-h6kg9\" (UID: \"c7280975-6841-4d25-82f2-18702e8cf8ef\") " pod="openshift-monitoring/node-exporter-h6kg9"
Jul 22 16:08:42.383787 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.383747    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c7280975-6841-4d25-82f2-18702e8cf8ef-metrics-client-ca\") pod \"node-exporter-h6kg9\" (UID: \"c7280975-6841-4d25-82f2-18702e8cf8ef\") " pod="openshift-monitoring/node-exporter-h6kg9"
Jul 22 16:08:42.383787 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.383758    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c7280975-6841-4d25-82f2-18702e8cf8ef-node-exporter-accelerators-collector-config\") pod \"node-exporter-h6kg9\" (UID: \"c7280975-6841-4d25-82f2-18702e8cf8ef\") " pod="openshift-monitoring/node-exporter-h6kg9"
Jul 22 16:08:42.385636 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.385616    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c7280975-6841-4d25-82f2-18702e8cf8ef-node-exporter-tls\") pod \"node-exporter-h6kg9\" (UID: \"c7280975-6841-4d25-82f2-18702e8cf8ef\") " pod="openshift-monitoring/node-exporter-h6kg9"
Jul 22 16:08:42.385750 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.385670    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c7280975-6841-4d25-82f2-18702e8cf8ef-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h6kg9\" (UID: \"c7280975-6841-4d25-82f2-18702e8cf8ef\") " pod="openshift-monitoring/node-exporter-h6kg9"
Jul 22 16:08:42.399717 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.399692    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvdf8\" (UniqueName: \"kubernetes.io/projected/c7280975-6841-4d25-82f2-18702e8cf8ef-kube-api-access-vvdf8\") pod \"node-exporter-h6kg9\" (UID: \"c7280975-6841-4d25-82f2-18702e8cf8ef\") " pod="openshift-monitoring/node-exporter-h6kg9"
Jul 22 16:08:42.432946 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.432919    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-pcfcd"
Jul 22 16:08:42.467649 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.467594    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:42.475558 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.475532    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h6kg9"
Jul 22 16:08:42.487629 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:08:42.487588    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7280975_6841_4d25_82f2_18702e8cf8ef.slice/crio-fa1e7f6d67189af2cd47ad7ade5cd8f7f2eb3c0308a1bad5b6ae52fbda2a12d1 WatchSource:0}: Error finding container fa1e7f6d67189af2cd47ad7ade5cd8f7f2eb3c0308a1bad5b6ae52fbda2a12d1: Status 404 returned error can't find the container with id fa1e7f6d67189af2cd47ad7ade5cd8f7f2eb3c0308a1bad5b6ae52fbda2a12d1
Jul 22 16:08:42.566068 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.564172    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-pcfcd"]
Jul 22 16:08:42.567591 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:08:42.567565    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1d3f28a_7b56_4185_b5af_6bad89156abc.slice/crio-d159b1ca4ff4f2c1e74c252e482b69d0f375a68cea2b613bfd98b08f63ddab25 WatchSource:0}: Error finding container d159b1ca4ff4f2c1e74c252e482b69d0f375a68cea2b613bfd98b08f63ddab25: Status 404 returned error can't find the container with id d159b1ca4ff4f2c1e74c252e482b69d0f375a68cea2b613bfd98b08f63ddab25
Jul 22 16:08:42.588627 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.588604    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:42.698496 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.698463    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:42.794375 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.794349    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:42.897579 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.897523    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:42.945229 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.945146    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-pcfcd" event={"ID":"a1d3f28a-7b56-4185-b5af-6bad89156abc","Type":"ContainerStarted","Data":"6ce8d3af529f53b947e6c6ba5fc9e770f02be5cd0b87a99c3ef7ccd5af1df11a"}
Jul 22 16:08:42.945229 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.945207    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-pcfcd" event={"ID":"a1d3f28a-7b56-4185-b5af-6bad89156abc","Type":"ContainerStarted","Data":"782fe31ee72547b222a6aaa663ca546c33f1d1cc5f3a30158381fe41c428d6ba"}
Jul 22 16:08:42.945229 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.945229    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-pcfcd" event={"ID":"a1d3f28a-7b56-4185-b5af-6bad89156abc","Type":"ContainerStarted","Data":"d159b1ca4ff4f2c1e74c252e482b69d0f375a68cea2b613bfd98b08f63ddab25"}
Jul 22 16:08:42.946273 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:42.946254    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h6kg9" event={"ID":"c7280975-6841-4d25-82f2-18702e8cf8ef","Type":"ContainerStarted","Data":"fa1e7f6d67189af2cd47ad7ade5cd8f7f2eb3c0308a1bad5b6ae52fbda2a12d1"}
Jul 22 16:08:43.115514 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.115487    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:43.209937 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.209840    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:43.214827 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.214798    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 16:08:43.217539 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.217511    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.220494 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.220470    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 22 16:08:43.220712 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.220688    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 22 16:08:43.220920 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.220876    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 22 16:08:43.221133 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.221112    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-g27bv\""
Jul 22 16:08:43.221246 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.221115    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 22 16:08:43.221312 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.221282    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 22 16:08:43.221404 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.221388    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 22 16:08:43.221531 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.221514    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 22 16:08:43.221787 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.221742    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 22 16:08:43.221929 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.221904    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 22 16:08:43.235236 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.235196    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 16:08:43.289244 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.289205    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6c4b2472-6e15-4d35-94e1-efa2a862db97-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.289414 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.289260    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.289414 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.289298    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkp5b\" (UniqueName: \"kubernetes.io/projected/6c4b2472-6e15-4d35-94e1-efa2a862db97-kube-api-access-rkp5b\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.289414 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.289333    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.289414 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.289359    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c4b2472-6e15-4d35-94e1-efa2a862db97-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.289414 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.289408    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6c4b2472-6e15-4d35-94e1-efa2a862db97-config-out\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.289666 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.289436    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-web-config\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.289666 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.289477    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.289666 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.289514    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.289666 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.289564    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-config-volume\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.289666 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.289611    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6c4b2472-6e15-4d35-94e1-efa2a862db97-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.289666 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.289653    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.289975 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.289699    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c4b2472-6e15-4d35-94e1-efa2a862db97-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.391985 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.390874    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-config-volume\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.391985 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.390941    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6c4b2472-6e15-4d35-94e1-efa2a862db97-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.391985 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.390979    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.391985 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.391008    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c4b2472-6e15-4d35-94e1-efa2a862db97-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.391985 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.391044    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6c4b2472-6e15-4d35-94e1-efa2a862db97-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.391985 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.391067    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.391985 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.391096    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkp5b\" (UniqueName: \"kubernetes.io/projected/6c4b2472-6e15-4d35-94e1-efa2a862db97-kube-api-access-rkp5b\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.391985 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.391125    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.391985 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.391151    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c4b2472-6e15-4d35-94e1-efa2a862db97-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.391985 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.391183    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6c4b2472-6e15-4d35-94e1-efa2a862db97-config-out\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.391985 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.391211    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-web-config\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.391985 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.391256    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.391985 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.391291    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.391985 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:08:43.391565    2563 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 22 16:08:43.392780 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:08:43.391629    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-secret-alertmanager-main-tls podName:6c4b2472-6e15-4d35-94e1-efa2a862db97 nodeName:}" failed. No retries permitted until 2026-07-22 16:08:43.891608623 +0000 UTC m=+84.022078616 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "6c4b2472-6e15-4d35-94e1-efa2a862db97") : secret "alertmanager-main-tls" not found
Jul 22 16:08:43.392780 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.391660    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6c4b2472-6e15-4d35-94e1-efa2a862db97-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.392936 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.392916    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c4b2472-6e15-4d35-94e1-efa2a862db97-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.395345 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.395315    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.395483 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:08:43.395439    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c4b2472-6e15-4d35-94e1-efa2a862db97-alertmanager-trusted-ca-bundle podName:6c4b2472-6e15-4d35-94e1-efa2a862db97 nodeName:}" failed. No retries permitted until 2026-07-22 16:08:43.895406702 +0000 UTC m=+84.025876683 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/6c4b2472-6e15-4d35-94e1-efa2a862db97-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "6c4b2472-6e15-4d35-94e1-efa2a862db97") : configmap references non-existent config key: ca-bundle.crt
Jul 22 16:08:43.397139 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.397093    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.397245 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.397194    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-config-volume\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.398792 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.398739    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.399132 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.399108    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-web-config\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.399725 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.399696    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.400320 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.400098    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6c4b2472-6e15-4d35-94e1-efa2a862db97-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.400320 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.400285    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6c4b2472-6e15-4d35-94e1-efa2a862db97-config-out\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.401716 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.401669    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkp5b\" (UniqueName: \"kubernetes.io/projected/6c4b2472-6e15-4d35-94e1-efa2a862db97-kube-api-access-rkp5b\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.456155 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.456063    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:43.894871 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.894833    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.897572 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.897263    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.951921 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.951886    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h6kg9" event={"ID":"c7280975-6841-4d25-82f2-18702e8cf8ef","Type":"ContainerStarted","Data":"6d1508b9327480be81b0098c9ca7f19828013b36f82f6d6c33c3d68cd12ddd98"}
Jul 22 16:08:43.954620 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.954592    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-pcfcd" event={"ID":"a1d3f28a-7b56-4185-b5af-6bad89156abc","Type":"ContainerStarted","Data":"1c389cb7631d762ab66ba6dc00111dbf358f5d12339c3c7e96f38fab4e71ca03"}
Jul 22 16:08:43.989153 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.989106    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-pcfcd" podStartSLOduration=0.82342423 podStartE2EDuration="1.989089728s" totalImagesPullingTime="1.165665498s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:08:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:08:43.988169077 +0000 UTC m=+84.118639097" watchObservedRunningTime="2026-07-22 16:08:43.989089728 +0000 UTC m=+84.119559729"
Jul 22 16:08:43.995674 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.995648    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c4b2472-6e15-4d35-94e1-efa2a862db97-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:43.996460 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:43.996443    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c4b2472-6e15-4d35-94e1-efa2a862db97-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:44.130050 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:44.129962    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:08:44.263232 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:44.263210    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 16:08:44.265069 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:08:44.265043    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c4b2472_6e15_4d35_94e1_efa2a862db97.slice/crio-898803ce5b23a56085805c71bd4994b07fa53c6d4a8f0bbc60e455de211da4a6 WatchSource:0}: Error finding container 898803ce5b23a56085805c71bd4994b07fa53c6d4a8f0bbc60e455de211da4a6: Status 404 returned error can't find the container with id 898803ce5b23a56085805c71bd4994b07fa53c6d4a8f0bbc60e455de211da4a6
Jul 22 16:08:44.607702 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:44.607670    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:44.707503 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:44.707467    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:44.925149 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:44.921633    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-xdxv8"
Jul 22 16:08:44.959927 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:44.959884    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c4b2472-6e15-4d35-94e1-efa2a862db97","Type":"ContainerStarted","Data":"898803ce5b23a56085805c71bd4994b07fa53c6d4a8f0bbc60e455de211da4a6"}
Jul 22 16:08:44.961590 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:44.961554    2563 generic.go:356] "Generic (PLEG): container finished" podID="c7280975-6841-4d25-82f2-18702e8cf8ef" containerID="6d1508b9327480be81b0098c9ca7f19828013b36f82f6d6c33c3d68cd12ddd98" exitCode=0
Jul 22 16:08:44.961698 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:44.961609    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h6kg9" event={"ID":"c7280975-6841-4d25-82f2-18702e8cf8ef","Type":"ContainerDied","Data":"6d1508b9327480be81b0098c9ca7f19828013b36f82f6d6c33c3d68cd12ddd98"}
Jul 22 16:08:45.065646 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:45.065576    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:45.166644 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:45.166617    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:45.584020 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:45.583995    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:45.673642 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:45.673609    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:45.966817 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:45.966787    2563 generic.go:356] "Generic (PLEG): container finished" podID="6c4b2472-6e15-4d35-94e1-efa2a862db97" containerID="b3fecac0fc5c4bc9f63c73c6a58159f6c919aced4c9910f80aa324bc557e3014" exitCode=0
Jul 22 16:08:45.967249 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:45.966882    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c4b2472-6e15-4d35-94e1-efa2a862db97","Type":"ContainerDied","Data":"b3fecac0fc5c4bc9f63c73c6a58159f6c919aced4c9910f80aa324bc557e3014"}
Jul 22 16:08:45.969303 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:45.969273    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h6kg9" event={"ID":"c7280975-6841-4d25-82f2-18702e8cf8ef","Type":"ContainerStarted","Data":"37553520353d74fcc19ec3fe9412c4d6b4b5824f63c6c602a88feefbdf9adb5c"}
Jul 22 16:08:45.969407 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:45.969324    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h6kg9" event={"ID":"c7280975-6841-4d25-82f2-18702e8cf8ef","Type":"ContainerStarted","Data":"eac23b643c87910690a90974023d735d807e6425689d3aaaf877e107f835cf90"}
Jul 22 16:08:46.013247 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:46.013194    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-h6kg9" podStartSLOduration=2.606659398 podStartE2EDuration="4.013176527s" totalImagesPullingTime="1.338747428s" totalInitContainerRuntime="67.769701ms" isStatefulPod=true podCreationTimestamp="2026-07-22 16:08:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:08:46.012376885 +0000 UTC m=+86.142846887" watchObservedRunningTime="2026-07-22 16:08:46.013176527 +0000 UTC m=+86.143646566"
Jul 22 16:08:46.370130 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:46.370040    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:46.911528 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:46.911493    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:46.915751 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:46.915724    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-dsckk"]
Jul 22 16:08:46.917830 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:46.917814    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-dsckk"
Jul 22 16:08:46.921113 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:46.921092    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 22 16:08:46.921213 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:46.921202    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-ddvrh\""
Jul 22 16:08:46.934587 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:46.934558    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-dsckk"]
Jul 22 16:08:47.019026 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:47.018985    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8kb2\" (UniqueName: \"kubernetes.io/projected/62ca114b-81f3-470f-8f20-65212dfc953e-kube-api-access-f8kb2\") pod \"monitoring-plugin-7b7b7bc8bf-dsckk\" (UID: \"62ca114b-81f3-470f-8f20-65212dfc953e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-dsckk"
Jul 22 16:08:47.019486 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:47.019159    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/62ca114b-81f3-470f-8f20-65212dfc953e-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-dsckk\" (UID: \"62ca114b-81f3-470f-8f20-65212dfc953e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-dsckk"
Jul 22 16:08:47.120168 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:47.120132    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/62ca114b-81f3-470f-8f20-65212dfc953e-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-dsckk\" (UID: \"62ca114b-81f3-470f-8f20-65212dfc953e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-dsckk"
Jul 22 16:08:47.120335 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:47.120181    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8kb2\" (UniqueName: \"kubernetes.io/projected/62ca114b-81f3-470f-8f20-65212dfc953e-kube-api-access-f8kb2\") pod \"monitoring-plugin-7b7b7bc8bf-dsckk\" (UID: \"62ca114b-81f3-470f-8f20-65212dfc953e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-dsckk"
Jul 22 16:08:47.120335 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:08:47.120227    2563 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 22 16:08:47.120335 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:08:47.120290    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62ca114b-81f3-470f-8f20-65212dfc953e-monitoring-plugin-cert podName:62ca114b-81f3-470f-8f20-65212dfc953e nodeName:}" failed. No retries permitted until 2026-07-22 16:08:47.620271161 +0000 UTC m=+87.750741160 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/62ca114b-81f3-470f-8f20-65212dfc953e-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-dsckk" (UID: "62ca114b-81f3-470f-8f20-65212dfc953e") : secret "monitoring-plugin-cert" not found
Jul 22 16:08:47.133204 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:47.133171    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8kb2\" (UniqueName: \"kubernetes.io/projected/62ca114b-81f3-470f-8f20-65212dfc953e-kube-api-access-f8kb2\") pod \"monitoring-plugin-7b7b7bc8bf-dsckk\" (UID: \"62ca114b-81f3-470f-8f20-65212dfc953e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-dsckk"
Jul 22 16:08:47.343215 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:47.343159    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:47.624866 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:47.624825    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/62ca114b-81f3-470f-8f20-65212dfc953e-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-dsckk\" (UID: \"62ca114b-81f3-470f-8f20-65212dfc953e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-dsckk"
Jul 22 16:08:47.627132 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:47.627106    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/62ca114b-81f3-470f-8f20-65212dfc953e-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-dsckk\" (UID: \"62ca114b-81f3-470f-8f20-65212dfc953e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-dsckk"
Jul 22 16:08:47.740832 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:47.740801    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:47.826490 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:47.826468    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-dsckk"
Jul 22 16:08:47.841415 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:47.841347    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:47.976261 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:47.976240    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-dsckk"]
Jul 22 16:08:47.978406 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:08:47.978368    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62ca114b_81f3_470f_8f20_65212dfc953e.slice/crio-32274fb30c758d96423b85e8bc3e81ff267538be8f5fecc8c702d7368294fe3a WatchSource:0}: Error finding container 32274fb30c758d96423b85e8bc3e81ff267538be8f5fecc8c702d7368294fe3a: Status 404 returned error can't find the container with id 32274fb30c758d96423b85e8bc3e81ff267538be8f5fecc8c702d7368294fe3a
Jul 22 16:08:47.980730 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:47.980711    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c4b2472-6e15-4d35-94e1-efa2a862db97","Type":"ContainerStarted","Data":"37ec2e044dc90789e462ef5e69362617a7ebbc30f2c03fe143a1174e2e87fb3d"}
Jul 22 16:08:47.980795 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:47.980766    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c4b2472-6e15-4d35-94e1-efa2a862db97","Type":"ContainerStarted","Data":"589046f23ee2b1388a2fc2a63c6d4edf26ec18272514512837b1babf524072e1"}
Jul 22 16:08:47.980795 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:47.980792    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c4b2472-6e15-4d35-94e1-efa2a862db97","Type":"ContainerStarted","Data":"c5a5260ee3e27fde598c757f0bf5957ca837921e0a8cc14dc966cded406325c6"}
Jul 22 16:08:47.980907 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:47.980816    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c4b2472-6e15-4d35-94e1-efa2a862db97","Type":"ContainerStarted","Data":"f367881ec54e59da9b116a8b9f691dc19711c3e935e69f4052a0ead91803d6e0"}
Jul 22 16:08:47.980907 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:47.980837    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c4b2472-6e15-4d35-94e1-efa2a862db97","Type":"ContainerStarted","Data":"13f04e4b4e865b7184e1b789031a18923fdefffbb4b68c9d36a3aa32b252d8fa"}
Jul 22 16:08:48.005569 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.005534    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:48.105386 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.105344    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:48.206847 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.206816    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:48.305458 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.305400    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:48.365731 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.365702    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 16:08:48.368675 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.368656    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.371670 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.371649    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 22 16:08:48.371785 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.371651    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-14st95tns06dl\""
Jul 22 16:08:48.371844 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.371811    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 22 16:08:48.371844 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.371833    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 22 16:08:48.371982 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.371966    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-6676b\""
Jul 22 16:08:48.372107 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.372083    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 22 16:08:48.372163 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.372122    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 22 16:08:48.373056 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.373032    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 22 16:08:48.373160 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.373124    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 22 16:08:48.373275 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.373262    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 22 16:08:48.373426 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.373376    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 22 16:08:48.373426 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.373386    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 22 16:08:48.373553 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.373380    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 22 16:08:48.373553 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.373484    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 22 16:08:48.373754 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.373736    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 22 16:08:48.373868 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.373769    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 22 16:08:48.375721 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.375705    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 22 16:08:48.393873 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.393828    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 16:08:48.407255 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.407223    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:48.431006 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.430979    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.431138 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.431019    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.431138 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.431043    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.431138 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.431060    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.431138 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.431099    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/91cd4a98-ad75-4c0d-942e-3482dcf02e61-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.431263 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.431138    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-config\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.431263 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.431157    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.431263 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.431181    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.431263 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.431208    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/91cd4a98-ad75-4c0d-942e-3482dcf02e61-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.431263 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.431244    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/91cd4a98-ad75-4c0d-942e-3482dcf02e61-config-out\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.431402 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.431287    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.431402 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.431315    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.431402 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.431334    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.431402 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.431374    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.431402 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.431391    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.431578 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.431406    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.431578 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.431424    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.431578 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.431443    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mvl7\" (UniqueName: \"kubernetes.io/projected/91cd4a98-ad75-4c0d-942e-3482dcf02e61-kube-api-access-7mvl7\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.431578 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.431469    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.431578 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.431490    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-web-config\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.532189 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.532099    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.532189 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.532155    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/91cd4a98-ad75-4c0d-942e-3482dcf02e61-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.532189 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.532182    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/91cd4a98-ad75-4c0d-942e-3482dcf02e61-config-out\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.532600 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.532211    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.532600 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.532249    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.532600 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.532279    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.532600 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.532316    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.532600 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.532343    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.532892 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.532807    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.532892 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.532842    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.532995 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.532904    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mvl7\" (UniqueName: \"kubernetes.io/projected/91cd4a98-ad75-4c0d-942e-3482dcf02e61-kube-api-access-7mvl7\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.532995 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.532932    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.532995 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.532962    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-web-config\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.533133 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.532999    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.533133 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.533012    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.533133 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.532998    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.533133 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.533067    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.533133 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.533117    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.533353 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.533150    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.533353 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.533175    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/91cd4a98-ad75-4c0d-942e-3482dcf02e61-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.533353 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.533216    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-config\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.533353 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.533241    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.534003 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.533804    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.534003 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.533893    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.535111 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.535088    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.536520 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.535879    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.536520 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.536148    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/91cd4a98-ad75-4c0d-942e-3482dcf02e61-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.536520 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.536166    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.537770 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.537715    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.538797 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.538747    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.539280 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.539215    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/91cd4a98-ad75-4c0d-942e-3482dcf02e61-config-out\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.539752 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.539710    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.539876 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.539777    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.540332 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.540307    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.540332 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.540322    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.540760 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.540738    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-config\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.540846 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.540825    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.541359 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.541283    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/91cd4a98-ad75-4c0d-942e-3482dcf02e61-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.542948 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.542823    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-web-config\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.546939 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.546919    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mvl7\" (UniqueName: \"kubernetes.io/projected/91cd4a98-ad75-4c0d-942e-3482dcf02e61-kube-api-access-7mvl7\") pod \"prometheus-k8s-0\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.668893 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.668830    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:48.679363 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.679332    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:48.770999 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.770960    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:48.968292 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.968263    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 16:08:48.970102 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:08:48.970073    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91cd4a98_ad75_4c0d_942e_3482dcf02e61.slice/crio-56ff8ac9a581be0aa06dec1780a4835e3df6d330bb4ed6c2786de5b7084e6ae9 WatchSource:0}: Error finding container 56ff8ac9a581be0aa06dec1780a4835e3df6d330bb4ed6c2786de5b7084e6ae9: Status 404 returned error can't find the container with id 56ff8ac9a581be0aa06dec1780a4835e3df6d330bb4ed6c2786de5b7084e6ae9
Jul 22 16:08:48.988501 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.988463    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"91cd4a98-ad75-4c0d-942e-3482dcf02e61","Type":"ContainerStarted","Data":"56ff8ac9a581be0aa06dec1780a4835e3df6d330bb4ed6c2786de5b7084e6ae9"}
Jul 22 16:08:48.989911 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.989883    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-dsckk" event={"ID":"62ca114b-81f3-470f-8f20-65212dfc953e","Type":"ContainerStarted","Data":"32274fb30c758d96423b85e8bc3e81ff267538be8f5fecc8c702d7368294fe3a"}
Jul 22 16:08:48.993496 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:48.993461    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c4b2472-6e15-4d35-94e1-efa2a862db97","Type":"ContainerStarted","Data":"9ff0c6e9260b9ec367ef07189426973f5b9963bb98f7a99ae2ff9220e4440be5"}
Jul 22 16:08:49.390838 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:49.390809    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:49.612067 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:49.611977    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:49.817130 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:49.817095    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:49.885236 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:49.885155    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:49.998548 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:49.998514    2563 generic.go:356] "Generic (PLEG): container finished" podID="91cd4a98-ad75-4c0d-942e-3482dcf02e61" containerID="ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2" exitCode=0
Jul 22 16:08:49.998701 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:49.998589    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"91cd4a98-ad75-4c0d-942e-3482dcf02e61","Type":"ContainerDied","Data":"ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2"}
Jul 22 16:08:50.000243 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:50.000176    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-dsckk" event={"ID":"62ca114b-81f3-470f-8f20-65212dfc953e","Type":"ContainerStarted","Data":"31fda13ee3dae1c2285538be0f23d624f59b4f1940bfc35d9d4ed91d5a2b7b64"}
Jul 22 16:08:50.000725 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:50.000708    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-dsckk"
Jul 22 16:08:50.005710 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:50.005689    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-dsckk"
Jul 22 16:08:50.037323 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:50.037271    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.13034568 podStartE2EDuration="7.037253201s" totalImagesPullingTime="3.891346782s" totalInitContainerRuntime="15.560739ms" isStatefulPod=false podCreationTimestamp="2026-07-22 16:08:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:08:49.046032185 +0000 UTC m=+89.176502187" watchObservedRunningTime="2026-07-22 16:08:50.037253201 +0000 UTC m=+90.167723239"
Jul 22 16:08:50.064045 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:50.063993    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-dsckk" podStartSLOduration=2.704587718 podStartE2EDuration="4.063974508s" totalImagesPullingTime="1.35938679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:08:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:08:50.063711165 +0000 UTC m=+90.194181166" watchObservedRunningTime="2026-07-22 16:08:50.063974508 +0000 UTC m=+90.194444510"
Jul 22 16:08:51.534650 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:51.534576    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:51.634245 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:51.634208    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:52.631610 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:52.631550    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:52.726780 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:52.726731    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:54.018450 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:54.018413    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"91cd4a98-ad75-4c0d-942e-3482dcf02e61","Type":"ContainerStarted","Data":"7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69"}
Jul 22 16:08:54.018817 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:54.018494    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"91cd4a98-ad75-4c0d-942e-3482dcf02e61","Type":"ContainerStarted","Data":"f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315"}
Jul 22 16:08:54.610742 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:54.610703    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:54.681771 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:54.681726    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:57.033670 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:57.033628    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"91cd4a98-ad75-4c0d-942e-3482dcf02e61","Type":"ContainerStarted","Data":"906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c"}
Jul 22 16:08:57.034207 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:57.033722    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"91cd4a98-ad75-4c0d-942e-3482dcf02e61","Type":"ContainerStarted","Data":"6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f"}
Jul 22 16:08:57.034207 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:57.033774    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"91cd4a98-ad75-4c0d-942e-3482dcf02e61","Type":"ContainerStarted","Data":"66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc"}
Jul 22 16:08:57.034207 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:57.033818    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"91cd4a98-ad75-4c0d-942e-3482dcf02e61","Type":"ContainerStarted","Data":"3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894"}
Jul 22 16:08:57.071722 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:57.071655    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.866412763 podStartE2EDuration="9.071630066s" totalImagesPullingTime="6.184990801s" totalInitContainerRuntime="20.226502ms" isStatefulPod=false podCreationTimestamp="2026-07-22 16:08:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:08:57.06910695 +0000 UTC m=+97.199576973" watchObservedRunningTime="2026-07-22 16:08:57.071630066 +0000 UTC m=+97.202100069"
Jul 22 16:08:58.680475 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:58.680441    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:08:59.619476 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:59.619444    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:08:59.719532 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:08:59.719497    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:09:00.175831 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:09:00.175787    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:09:00.278012 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:09:00.277966    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:09:00.564285 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:09:00.564252    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:09:00.662574 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:09:00.662537    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:09:01.213305 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:09:01.213277    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:09:01.314863 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:09:01.314830    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:09:03.692374 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:09:03.692336    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:09:03.819193 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:09:03.819159    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:09:04.630215 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:09:04.630186    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:09:04.696278 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:09:04.696242    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:09:05.921768 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:09:05.921742    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kq5l6"
Jul 22 16:09:09.818713 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:09:09.818678    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:09:09.962544 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:09:09.962511    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:09:10.502192 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:09:10.502160    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:09:10.603239 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:09:10.603210    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:09:14.433399 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:09:14.433367    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:09:14.513616 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:09:14.513590    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:09:15.856625 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:09:15.856589    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:09:15.956820 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:09:15.956784    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:09:16.057339 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:09:16.057315    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:09:16.118048 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:09:16.117968    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:09:16.603379 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:09:16.603341    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:09:16.667543 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:09:16.667511    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:09:29.608597 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:09:29.608559    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:09:29.686809 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:09:29.686782    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:09:48.680203 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:09:48.680168    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:09:48.699775 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:09:48.699747    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:09:49.259793 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:09:49.259767    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:01.368569 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:01.368534    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:10:01.670731 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:01.670650    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:10:02.437936 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:02.437897    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 16:10:02.438501 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:02.438446    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6c4b2472-6e15-4d35-94e1-efa2a862db97" containerName="alertmanager" containerID="cri-o://13f04e4b4e865b7184e1b789031a18923fdefffbb4b68c9d36a3aa32b252d8fa" gracePeriod=120
Jul 22 16:10:02.438842 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:02.438789    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6c4b2472-6e15-4d35-94e1-efa2a862db97" containerName="kube-rbac-proxy" containerID="cri-o://589046f23ee2b1388a2fc2a63c6d4edf26ec18272514512837b1babf524072e1" gracePeriod=120
Jul 22 16:10:02.438950 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:02.438906    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6c4b2472-6e15-4d35-94e1-efa2a862db97" containerName="kube-rbac-proxy-web" containerID="cri-o://c5a5260ee3e27fde598c757f0bf5957ca837921e0a8cc14dc966cded406325c6" gracePeriod=120
Jul 22 16:10:02.439026 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:02.439005    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6c4b2472-6e15-4d35-94e1-efa2a862db97" containerName="prom-label-proxy" containerID="cri-o://9ff0c6e9260b9ec367ef07189426973f5b9963bb98f7a99ae2ff9220e4440be5" gracePeriod=120
Jul 22 16:10:02.439074 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:02.439038    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6c4b2472-6e15-4d35-94e1-efa2a862db97" containerName="config-reloader" containerID="cri-o://f367881ec54e59da9b116a8b9f691dc19711c3e935e69f4052a0ead91803d6e0" gracePeriod=120
Jul 22 16:10:02.439143 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:02.439125    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6c4b2472-6e15-4d35-94e1-efa2a862db97" containerName="kube-rbac-proxy-metric" containerID="cri-o://37ec2e044dc90789e462ef5e69362617a7ebbc30f2c03fe143a1174e2e87fb3d" gracePeriod=120
Jul 22 16:10:03.297658 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.297624    2563 generic.go:356] "Generic (PLEG): container finished" podID="6c4b2472-6e15-4d35-94e1-efa2a862db97" containerID="9ff0c6e9260b9ec367ef07189426973f5b9963bb98f7a99ae2ff9220e4440be5" exitCode=0
Jul 22 16:10:03.297658 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.297649    2563 generic.go:356] "Generic (PLEG): container finished" podID="6c4b2472-6e15-4d35-94e1-efa2a862db97" containerID="589046f23ee2b1388a2fc2a63c6d4edf26ec18272514512837b1babf524072e1" exitCode=0
Jul 22 16:10:03.297658 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.297656    2563 generic.go:356] "Generic (PLEG): container finished" podID="6c4b2472-6e15-4d35-94e1-efa2a862db97" containerID="f367881ec54e59da9b116a8b9f691dc19711c3e935e69f4052a0ead91803d6e0" exitCode=0
Jul 22 16:10:03.297658 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.297662    2563 generic.go:356] "Generic (PLEG): container finished" podID="6c4b2472-6e15-4d35-94e1-efa2a862db97" containerID="13f04e4b4e865b7184e1b789031a18923fdefffbb4b68c9d36a3aa32b252d8fa" exitCode=0
Jul 22 16:10:03.297948 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.297695    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c4b2472-6e15-4d35-94e1-efa2a862db97","Type":"ContainerDied","Data":"9ff0c6e9260b9ec367ef07189426973f5b9963bb98f7a99ae2ff9220e4440be5"}
Jul 22 16:10:03.297948 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.297765    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c4b2472-6e15-4d35-94e1-efa2a862db97","Type":"ContainerDied","Data":"589046f23ee2b1388a2fc2a63c6d4edf26ec18272514512837b1babf524072e1"}
Jul 22 16:10:03.297948 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.297792    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c4b2472-6e15-4d35-94e1-efa2a862db97","Type":"ContainerDied","Data":"f367881ec54e59da9b116a8b9f691dc19711c3e935e69f4052a0ead91803d6e0"}
Jul 22 16:10:03.297948 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.297817    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c4b2472-6e15-4d35-94e1-efa2a862db97","Type":"ContainerDied","Data":"13f04e4b4e865b7184e1b789031a18923fdefffbb4b68c9d36a3aa32b252d8fa"}
Jul 22 16:10:03.695102 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.695066    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:10:03.859304 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.859197    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c4b2472-6e15-4d35-94e1-efa2a862db97-alertmanager-trusted-ca-bundle\") pod \"6c4b2472-6e15-4d35-94e1-efa2a862db97\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") "
Jul 22 16:10:03.859304 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.859297    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-web-config\") pod \"6c4b2472-6e15-4d35-94e1-efa2a862db97\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") "
Jul 22 16:10:03.859531 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.859330    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c4b2472-6e15-4d35-94e1-efa2a862db97-metrics-client-ca\") pod \"6c4b2472-6e15-4d35-94e1-efa2a862db97\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") "
Jul 22 16:10:03.859531 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.859355    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-secret-alertmanager-kube-rbac-proxy\") pod \"6c4b2472-6e15-4d35-94e1-efa2a862db97\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") "
Jul 22 16:10:03.859531 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.859385    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-secret-alertmanager-main-tls\") pod \"6c4b2472-6e15-4d35-94e1-efa2a862db97\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") "
Jul 22 16:10:03.859531 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.859413    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6c4b2472-6e15-4d35-94e1-efa2a862db97-alertmanager-main-db\") pod \"6c4b2472-6e15-4d35-94e1-efa2a862db97\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") "
Jul 22 16:10:03.859531 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.859441    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-secret-alertmanager-kube-rbac-proxy-metric\") pod \"6c4b2472-6e15-4d35-94e1-efa2a862db97\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") "
Jul 22 16:10:03.859531 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.859463    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-config-volume\") pod \"6c4b2472-6e15-4d35-94e1-efa2a862db97\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") "
Jul 22 16:10:03.859531 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.859491    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6c4b2472-6e15-4d35-94e1-efa2a862db97-config-out\") pod \"6c4b2472-6e15-4d35-94e1-efa2a862db97\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") "
Jul 22 16:10:03.859531 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.859518    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-secret-alertmanager-kube-rbac-proxy-web\") pod \"6c4b2472-6e15-4d35-94e1-efa2a862db97\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") "
Jul 22 16:10:03.859935 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.859543    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rkp5b\" (UniqueName: \"kubernetes.io/projected/6c4b2472-6e15-4d35-94e1-efa2a862db97-kube-api-access-rkp5b\") pod \"6c4b2472-6e15-4d35-94e1-efa2a862db97\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") "
Jul 22 16:10:03.859935 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.859585    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6c4b2472-6e15-4d35-94e1-efa2a862db97-tls-assets\") pod \"6c4b2472-6e15-4d35-94e1-efa2a862db97\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") "
Jul 22 16:10:03.859935 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.859624    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-cluster-tls-config\") pod \"6c4b2472-6e15-4d35-94e1-efa2a862db97\" (UID: \"6c4b2472-6e15-4d35-94e1-efa2a862db97\") "
Jul 22 16:10:03.859935 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.859730    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c4b2472-6e15-4d35-94e1-efa2a862db97-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "6c4b2472-6e15-4d35-94e1-efa2a862db97" (UID: "6c4b2472-6e15-4d35-94e1-efa2a862db97"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 16:10:03.859935 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.859628    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c4b2472-6e15-4d35-94e1-efa2a862db97-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "6c4b2472-6e15-4d35-94e1-efa2a862db97" (UID: "6c4b2472-6e15-4d35-94e1-efa2a862db97"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 16:10:03.859935 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.859749    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c4b2472-6e15-4d35-94e1-efa2a862db97-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "6c4b2472-6e15-4d35-94e1-efa2a862db97" (UID: "6c4b2472-6e15-4d35-94e1-efa2a862db97"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:10:03.862660 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.862377    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "6c4b2472-6e15-4d35-94e1-efa2a862db97" (UID: "6c4b2472-6e15-4d35-94e1-efa2a862db97"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:10:03.862660 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.862558    2563 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c4b2472-6e15-4d35-94e1-efa2a862db97-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:10:03.862660 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.862576    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c4b2472-6e15-4d35-94e1-efa2a862db97-metrics-client-ca\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:10:03.862660 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.862590    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-secret-alertmanager-main-tls\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:10:03.862660 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.862603    2563 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6c4b2472-6e15-4d35-94e1-efa2a862db97-alertmanager-main-db\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:10:03.863082 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.862687    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c4b2472-6e15-4d35-94e1-efa2a862db97-config-out" (OuterVolumeSpecName: "config-out") pod "6c4b2472-6e15-4d35-94e1-efa2a862db97" (UID: "6c4b2472-6e15-4d35-94e1-efa2a862db97"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:10:03.863082 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.862707    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c4b2472-6e15-4d35-94e1-efa2a862db97-kube-api-access-rkp5b" (OuterVolumeSpecName: "kube-api-access-rkp5b") pod "6c4b2472-6e15-4d35-94e1-efa2a862db97" (UID: "6c4b2472-6e15-4d35-94e1-efa2a862db97"). InnerVolumeSpecName "kube-api-access-rkp5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:10:03.863082 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.862934    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "6c4b2472-6e15-4d35-94e1-efa2a862db97" (UID: "6c4b2472-6e15-4d35-94e1-efa2a862db97"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:10:03.863082 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.862959    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "6c4b2472-6e15-4d35-94e1-efa2a862db97" (UID: "6c4b2472-6e15-4d35-94e1-efa2a862db97"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:10:03.863082 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.863009    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c4b2472-6e15-4d35-94e1-efa2a862db97-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "6c4b2472-6e15-4d35-94e1-efa2a862db97" (UID: "6c4b2472-6e15-4d35-94e1-efa2a862db97"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:10:03.864425 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.864406    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-config-volume" (OuterVolumeSpecName: "config-volume") pod "6c4b2472-6e15-4d35-94e1-efa2a862db97" (UID: "6c4b2472-6e15-4d35-94e1-efa2a862db97"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:10:03.864900 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.864884    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "6c4b2472-6e15-4d35-94e1-efa2a862db97" (UID: "6c4b2472-6e15-4d35-94e1-efa2a862db97"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:10:03.866639 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.866605    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "6c4b2472-6e15-4d35-94e1-efa2a862db97" (UID: "6c4b2472-6e15-4d35-94e1-efa2a862db97"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:10:03.873663 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.873635    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-web-config" (OuterVolumeSpecName: "web-config") pod "6c4b2472-6e15-4d35-94e1-efa2a862db97" (UID: "6c4b2472-6e15-4d35-94e1-efa2a862db97"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:10:03.963273 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.963233    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:10:03.963273 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.963266    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:10:03.963273 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.963276    2563 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-config-volume\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:10:03.963273 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.963285    2563 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6c4b2472-6e15-4d35-94e1-efa2a862db97-config-out\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:10:03.963530 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.963293    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:10:03.963530 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.963303    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkp5b\" (UniqueName: \"kubernetes.io/projected/6c4b2472-6e15-4d35-94e1-efa2a862db97-kube-api-access-rkp5b\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:10:03.963530 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.963311    2563 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6c4b2472-6e15-4d35-94e1-efa2a862db97-tls-assets\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:10:03.963530 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.963320    2563 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-cluster-tls-config\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:10:03.963530 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:03.963328    2563 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6c4b2472-6e15-4d35-94e1-efa2a862db97-web-config\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:10:04.305075 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.305046    2563 generic.go:356] "Generic (PLEG): container finished" podID="6c4b2472-6e15-4d35-94e1-efa2a862db97" containerID="37ec2e044dc90789e462ef5e69362617a7ebbc30f2c03fe143a1174e2e87fb3d" exitCode=0
Jul 22 16:10:04.305075 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.305071    2563 generic.go:356] "Generic (PLEG): container finished" podID="6c4b2472-6e15-4d35-94e1-efa2a862db97" containerID="c5a5260ee3e27fde598c757f0bf5957ca837921e0a8cc14dc966cded406325c6" exitCode=0
Jul 22 16:10:04.305309 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.305109    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c4b2472-6e15-4d35-94e1-efa2a862db97","Type":"ContainerDied","Data":"37ec2e044dc90789e462ef5e69362617a7ebbc30f2c03fe143a1174e2e87fb3d"}
Jul 22 16:10:04.305309 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.305169    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c4b2472-6e15-4d35-94e1-efa2a862db97","Type":"ContainerDied","Data":"c5a5260ee3e27fde598c757f0bf5957ca837921e0a8cc14dc966cded406325c6"}
Jul 22 16:10:04.305309 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.305198    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 22 16:10:04.305309 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.305214    2563 scope.go:117] "RemoveContainer" containerID="9ff0c6e9260b9ec367ef07189426973f5b9963bb98f7a99ae2ff9220e4440be5"
Jul 22 16:10:04.305309 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.305204    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c4b2472-6e15-4d35-94e1-efa2a862db97","Type":"ContainerDied","Data":"898803ce5b23a56085805c71bd4994b07fa53c6d4a8f0bbc60e455de211da4a6"}
Jul 22 16:10:04.313115 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.313091    2563 scope.go:117] "RemoveContainer" containerID="37ec2e044dc90789e462ef5e69362617a7ebbc30f2c03fe143a1174e2e87fb3d"
Jul 22 16:10:04.320669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.320649    2563 scope.go:117] "RemoveContainer" containerID="589046f23ee2b1388a2fc2a63c6d4edf26ec18272514512837b1babf524072e1"
Jul 22 16:10:04.332087 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.332043    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 16:10:04.332403 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.332243    2563 scope.go:117] "RemoveContainer" containerID="c5a5260ee3e27fde598c757f0bf5957ca837921e0a8cc14dc966cded406325c6"
Jul 22 16:10:04.334157 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.334135    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 16:10:04.339693 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.339677    2563 scope.go:117] "RemoveContainer" containerID="f367881ec54e59da9b116a8b9f691dc19711c3e935e69f4052a0ead91803d6e0"
Jul 22 16:10:04.347283 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.347261    2563 scope.go:117] "RemoveContainer" containerID="13f04e4b4e865b7184e1b789031a18923fdefffbb4b68c9d36a3aa32b252d8fa"
Jul 22 16:10:04.355459 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.355439    2563 scope.go:117] "RemoveContainer" containerID="b3fecac0fc5c4bc9f63c73c6a58159f6c919aced4c9910f80aa324bc557e3014"
Jul 22 16:10:04.363119 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.363096    2563 scope.go:117] "RemoveContainer" containerID="9ff0c6e9260b9ec367ef07189426973f5b9963bb98f7a99ae2ff9220e4440be5"
Jul 22 16:10:04.363454 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:10:04.363427    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ff0c6e9260b9ec367ef07189426973f5b9963bb98f7a99ae2ff9220e4440be5\": container with ID starting with 9ff0c6e9260b9ec367ef07189426973f5b9963bb98f7a99ae2ff9220e4440be5 not found: ID does not exist" containerID="9ff0c6e9260b9ec367ef07189426973f5b9963bb98f7a99ae2ff9220e4440be5"
Jul 22 16:10:04.363498 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.363466    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ff0c6e9260b9ec367ef07189426973f5b9963bb98f7a99ae2ff9220e4440be5"} err="failed to get container status \"9ff0c6e9260b9ec367ef07189426973f5b9963bb98f7a99ae2ff9220e4440be5\": rpc error: code = NotFound desc = could not find container \"9ff0c6e9260b9ec367ef07189426973f5b9963bb98f7a99ae2ff9220e4440be5\": container with ID starting with 9ff0c6e9260b9ec367ef07189426973f5b9963bb98f7a99ae2ff9220e4440be5 not found: ID does not exist"
Jul 22 16:10:04.363545 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.363503    2563 scope.go:117] "RemoveContainer" containerID="37ec2e044dc90789e462ef5e69362617a7ebbc30f2c03fe143a1174e2e87fb3d"
Jul 22 16:10:04.363771 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:10:04.363750    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37ec2e044dc90789e462ef5e69362617a7ebbc30f2c03fe143a1174e2e87fb3d\": container with ID starting with 37ec2e044dc90789e462ef5e69362617a7ebbc30f2c03fe143a1174e2e87fb3d not found: ID does not exist" containerID="37ec2e044dc90789e462ef5e69362617a7ebbc30f2c03fe143a1174e2e87fb3d"
Jul 22 16:10:04.363869 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.363783    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37ec2e044dc90789e462ef5e69362617a7ebbc30f2c03fe143a1174e2e87fb3d"} err="failed to get container status \"37ec2e044dc90789e462ef5e69362617a7ebbc30f2c03fe143a1174e2e87fb3d\": rpc error: code = NotFound desc = could not find container \"37ec2e044dc90789e462ef5e69362617a7ebbc30f2c03fe143a1174e2e87fb3d\": container with ID starting with 37ec2e044dc90789e462ef5e69362617a7ebbc30f2c03fe143a1174e2e87fb3d not found: ID does not exist"
Jul 22 16:10:04.363869 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.363809    2563 scope.go:117] "RemoveContainer" containerID="589046f23ee2b1388a2fc2a63c6d4edf26ec18272514512837b1babf524072e1"
Jul 22 16:10:04.364084 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:10:04.364065    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"589046f23ee2b1388a2fc2a63c6d4edf26ec18272514512837b1babf524072e1\": container with ID starting with 589046f23ee2b1388a2fc2a63c6d4edf26ec18272514512837b1babf524072e1 not found: ID does not exist" containerID="589046f23ee2b1388a2fc2a63c6d4edf26ec18272514512837b1babf524072e1"
Jul 22 16:10:04.364125 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.364089    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"589046f23ee2b1388a2fc2a63c6d4edf26ec18272514512837b1babf524072e1"} err="failed to get container status \"589046f23ee2b1388a2fc2a63c6d4edf26ec18272514512837b1babf524072e1\": rpc error: code = NotFound desc = could not find container \"589046f23ee2b1388a2fc2a63c6d4edf26ec18272514512837b1babf524072e1\": container with ID starting with 589046f23ee2b1388a2fc2a63c6d4edf26ec18272514512837b1babf524072e1 not found: ID does not exist"
Jul 22 16:10:04.364125 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.364107    2563 scope.go:117] "RemoveContainer" containerID="c5a5260ee3e27fde598c757f0bf5957ca837921e0a8cc14dc966cded406325c6"
Jul 22 16:10:04.364342 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:10:04.364323    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5a5260ee3e27fde598c757f0bf5957ca837921e0a8cc14dc966cded406325c6\": container with ID starting with c5a5260ee3e27fde598c757f0bf5957ca837921e0a8cc14dc966cded406325c6 not found: ID does not exist" containerID="c5a5260ee3e27fde598c757f0bf5957ca837921e0a8cc14dc966cded406325c6"
Jul 22 16:10:04.364393 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.364347    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5a5260ee3e27fde598c757f0bf5957ca837921e0a8cc14dc966cded406325c6"} err="failed to get container status \"c5a5260ee3e27fde598c757f0bf5957ca837921e0a8cc14dc966cded406325c6\": rpc error: code = NotFound desc = could not find container \"c5a5260ee3e27fde598c757f0bf5957ca837921e0a8cc14dc966cded406325c6\": container with ID starting with c5a5260ee3e27fde598c757f0bf5957ca837921e0a8cc14dc966cded406325c6 not found: ID does not exist"
Jul 22 16:10:04.364393 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.364358    2563 scope.go:117] "RemoveContainer" containerID="f367881ec54e59da9b116a8b9f691dc19711c3e935e69f4052a0ead91803d6e0"
Jul 22 16:10:04.364567 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:10:04.364552    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f367881ec54e59da9b116a8b9f691dc19711c3e935e69f4052a0ead91803d6e0\": container with ID starting with f367881ec54e59da9b116a8b9f691dc19711c3e935e69f4052a0ead91803d6e0 not found: ID does not exist" containerID="f367881ec54e59da9b116a8b9f691dc19711c3e935e69f4052a0ead91803d6e0"
Jul 22 16:10:04.364605 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.364570    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f367881ec54e59da9b116a8b9f691dc19711c3e935e69f4052a0ead91803d6e0"} err="failed to get container status \"f367881ec54e59da9b116a8b9f691dc19711c3e935e69f4052a0ead91803d6e0\": rpc error: code = NotFound desc = could not find container \"f367881ec54e59da9b116a8b9f691dc19711c3e935e69f4052a0ead91803d6e0\": container with ID starting with f367881ec54e59da9b116a8b9f691dc19711c3e935e69f4052a0ead91803d6e0 not found: ID does not exist"
Jul 22 16:10:04.364605 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.364581    2563 scope.go:117] "RemoveContainer" containerID="13f04e4b4e865b7184e1b789031a18923fdefffbb4b68c9d36a3aa32b252d8fa"
Jul 22 16:10:04.364787 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:10:04.364772    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13f04e4b4e865b7184e1b789031a18923fdefffbb4b68c9d36a3aa32b252d8fa\": container with ID starting with 13f04e4b4e865b7184e1b789031a18923fdefffbb4b68c9d36a3aa32b252d8fa not found: ID does not exist" containerID="13f04e4b4e865b7184e1b789031a18923fdefffbb4b68c9d36a3aa32b252d8fa"
Jul 22 16:10:04.364829 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.364788    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13f04e4b4e865b7184e1b789031a18923fdefffbb4b68c9d36a3aa32b252d8fa"} err="failed to get container status \"13f04e4b4e865b7184e1b789031a18923fdefffbb4b68c9d36a3aa32b252d8fa\": rpc error: code = NotFound desc = could not find container \"13f04e4b4e865b7184e1b789031a18923fdefffbb4b68c9d36a3aa32b252d8fa\": container with ID starting with 13f04e4b4e865b7184e1b789031a18923fdefffbb4b68c9d36a3aa32b252d8fa not found: ID does not exist"
Jul 22 16:10:04.364829 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.364800    2563 scope.go:117] "RemoveContainer" containerID="b3fecac0fc5c4bc9f63c73c6a58159f6c919aced4c9910f80aa324bc557e3014"
Jul 22 16:10:04.365054 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:10:04.365039    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3fecac0fc5c4bc9f63c73c6a58159f6c919aced4c9910f80aa324bc557e3014\": container with ID starting with b3fecac0fc5c4bc9f63c73c6a58159f6c919aced4c9910f80aa324bc557e3014 not found: ID does not exist" containerID="b3fecac0fc5c4bc9f63c73c6a58159f6c919aced4c9910f80aa324bc557e3014"
Jul 22 16:10:04.365098 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.365057    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3fecac0fc5c4bc9f63c73c6a58159f6c919aced4c9910f80aa324bc557e3014"} err="failed to get container status \"b3fecac0fc5c4bc9f63c73c6a58159f6c919aced4c9910f80aa324bc557e3014\": rpc error: code = NotFound desc = could not find container \"b3fecac0fc5c4bc9f63c73c6a58159f6c919aced4c9910f80aa324bc557e3014\": container with ID starting with b3fecac0fc5c4bc9f63c73c6a58159f6c919aced4c9910f80aa324bc557e3014 not found: ID does not exist"
Jul 22 16:10:04.365098 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.365070    2563 scope.go:117] "RemoveContainer" containerID="9ff0c6e9260b9ec367ef07189426973f5b9963bb98f7a99ae2ff9220e4440be5"
Jul 22 16:10:04.365277 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.365262    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ff0c6e9260b9ec367ef07189426973f5b9963bb98f7a99ae2ff9220e4440be5"} err="failed to get container status \"9ff0c6e9260b9ec367ef07189426973f5b9963bb98f7a99ae2ff9220e4440be5\": rpc error: code = NotFound desc = could not find container \"9ff0c6e9260b9ec367ef07189426973f5b9963bb98f7a99ae2ff9220e4440be5\": container with ID starting with 9ff0c6e9260b9ec367ef07189426973f5b9963bb98f7a99ae2ff9220e4440be5 not found: ID does not exist"
Jul 22 16:10:04.365277 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.365276    2563 scope.go:117] "RemoveContainer" containerID="37ec2e044dc90789e462ef5e69362617a7ebbc30f2c03fe143a1174e2e87fb3d"
Jul 22 16:10:04.365530 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.365511    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37ec2e044dc90789e462ef5e69362617a7ebbc30f2c03fe143a1174e2e87fb3d"} err="failed to get container status \"37ec2e044dc90789e462ef5e69362617a7ebbc30f2c03fe143a1174e2e87fb3d\": rpc error: code = NotFound desc = could not find container \"37ec2e044dc90789e462ef5e69362617a7ebbc30f2c03fe143a1174e2e87fb3d\": container with ID starting with 37ec2e044dc90789e462ef5e69362617a7ebbc30f2c03fe143a1174e2e87fb3d not found: ID does not exist"
Jul 22 16:10:04.365581 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.365532    2563 scope.go:117] "RemoveContainer" containerID="589046f23ee2b1388a2fc2a63c6d4edf26ec18272514512837b1babf524072e1"
Jul 22 16:10:04.365738 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.365724    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"589046f23ee2b1388a2fc2a63c6d4edf26ec18272514512837b1babf524072e1"} err="failed to get container status \"589046f23ee2b1388a2fc2a63c6d4edf26ec18272514512837b1babf524072e1\": rpc error: code = NotFound desc = could not find container \"589046f23ee2b1388a2fc2a63c6d4edf26ec18272514512837b1babf524072e1\": container with ID starting with 589046f23ee2b1388a2fc2a63c6d4edf26ec18272514512837b1babf524072e1 not found: ID does not exist"
Jul 22 16:10:04.365782 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.365739    2563 scope.go:117] "RemoveContainer" containerID="c5a5260ee3e27fde598c757f0bf5957ca837921e0a8cc14dc966cded406325c6"
Jul 22 16:10:04.365933 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.365912    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5a5260ee3e27fde598c757f0bf5957ca837921e0a8cc14dc966cded406325c6"} err="failed to get container status \"c5a5260ee3e27fde598c757f0bf5957ca837921e0a8cc14dc966cded406325c6\": rpc error: code = NotFound desc = could not find container \"c5a5260ee3e27fde598c757f0bf5957ca837921e0a8cc14dc966cded406325c6\": container with ID starting with c5a5260ee3e27fde598c757f0bf5957ca837921e0a8cc14dc966cded406325c6 not found: ID does not exist"
Jul 22 16:10:04.365979 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.365934    2563 scope.go:117] "RemoveContainer" containerID="f367881ec54e59da9b116a8b9f691dc19711c3e935e69f4052a0ead91803d6e0"
Jul 22 16:10:04.366130 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.366113    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f367881ec54e59da9b116a8b9f691dc19711c3e935e69f4052a0ead91803d6e0"} err="failed to get container status \"f367881ec54e59da9b116a8b9f691dc19711c3e935e69f4052a0ead91803d6e0\": rpc error: code = NotFound desc = could not find container \"f367881ec54e59da9b116a8b9f691dc19711c3e935e69f4052a0ead91803d6e0\": container with ID starting with f367881ec54e59da9b116a8b9f691dc19711c3e935e69f4052a0ead91803d6e0 not found: ID does not exist"
Jul 22 16:10:04.366190 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.366132    2563 scope.go:117] "RemoveContainer" containerID="13f04e4b4e865b7184e1b789031a18923fdefffbb4b68c9d36a3aa32b252d8fa"
Jul 22 16:10:04.366349 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.366333    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13f04e4b4e865b7184e1b789031a18923fdefffbb4b68c9d36a3aa32b252d8fa"} err="failed to get container status \"13f04e4b4e865b7184e1b789031a18923fdefffbb4b68c9d36a3aa32b252d8fa\": rpc error: code = NotFound desc = could not find container \"13f04e4b4e865b7184e1b789031a18923fdefffbb4b68c9d36a3aa32b252d8fa\": container with ID starting with 13f04e4b4e865b7184e1b789031a18923fdefffbb4b68c9d36a3aa32b252d8fa not found: ID does not exist"
Jul 22 16:10:04.366392 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.366350    2563 scope.go:117] "RemoveContainer" containerID="b3fecac0fc5c4bc9f63c73c6a58159f6c919aced4c9910f80aa324bc557e3014"
Jul 22 16:10:04.366568 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.366554    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3fecac0fc5c4bc9f63c73c6a58159f6c919aced4c9910f80aa324bc557e3014"} err="failed to get container status \"b3fecac0fc5c4bc9f63c73c6a58159f6c919aced4c9910f80aa324bc557e3014\": rpc error: code = NotFound desc = could not find container \"b3fecac0fc5c4bc9f63c73c6a58159f6c919aced4c9910f80aa324bc557e3014\": container with ID starting with b3fecac0fc5c4bc9f63c73c6a58159f6c919aced4c9910f80aa324bc557e3014 not found: ID does not exist"
Jul 22 16:10:04.447024 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:04.446994    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c4b2472-6e15-4d35-94e1-efa2a862db97" path="/var/lib/kubelet/pods/6c4b2472-6e15-4d35-94e1-efa2a862db97/volumes"
Jul 22 16:10:06.796404 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:06.796370    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 16:10:06.796926 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:06.796736    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="91cd4a98-ad75-4c0d-942e-3482dcf02e61" containerName="prometheus" containerID="cri-o://f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315" gracePeriod=600
Jul 22 16:10:06.796926 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:06.796784    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="91cd4a98-ad75-4c0d-942e-3482dcf02e61" containerName="kube-rbac-proxy-thanos" containerID="cri-o://906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c" gracePeriod=600
Jul 22 16:10:06.796926 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:06.796816    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="91cd4a98-ad75-4c0d-942e-3482dcf02e61" containerName="kube-rbac-proxy-web" containerID="cri-o://66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc" gracePeriod=600
Jul 22 16:10:06.796926 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:06.796844    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="91cd4a98-ad75-4c0d-942e-3482dcf02e61" containerName="kube-rbac-proxy" containerID="cri-o://6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f" gracePeriod=600
Jul 22 16:10:06.796926 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:06.796894    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="91cd4a98-ad75-4c0d-942e-3482dcf02e61" containerName="thanos-sidecar" containerID="cri-o://3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894" gracePeriod=600
Jul 22 16:10:06.797181 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:06.797082    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="91cd4a98-ad75-4c0d-942e-3482dcf02e61" containerName="config-reloader" containerID="cri-o://7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69" gracePeriod=600
Jul 22 16:10:07.028623 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.028599    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.184350 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.184317    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-configmap-serving-certs-ca-bundle\") pod \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") "
Jul 22 16:10:07.184529 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.184360    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-grpc-tls\") pod \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") "
Jul 22 16:10:07.184529 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.184376    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-prometheus-k8s-tls\") pod \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") "
Jul 22 16:10:07.184529 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.184402    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/91cd4a98-ad75-4c0d-942e-3482dcf02e61-prometheus-k8s-db\") pod \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") "
Jul 22 16:10:07.184529 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.184424    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") "
Jul 22 16:10:07.184529 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.184440    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-prometheus-k8s-rulefiles-1\") pod \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") "
Jul 22 16:10:07.184529 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.184460    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") "
Jul 22 16:10:07.184529 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.184483    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-prometheus-k8s-rulefiles-2\") pod \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") "
Jul 22 16:10:07.184924 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.184896    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "91cd4a98-ad75-4c0d-942e-3482dcf02e61" (UID: "91cd4a98-ad75-4c0d-942e-3482dcf02e61"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 16:10:07.184987 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.184957    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-configmap-kubelet-serving-ca-bundle\") pod \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") "
Jul 22 16:10:07.185036 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.184973    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "91cd4a98-ad75-4c0d-942e-3482dcf02e61" (UID: "91cd4a98-ad75-4c0d-942e-3482dcf02e61"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 16:10:07.185036 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.184999    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/91cd4a98-ad75-4c0d-942e-3482dcf02e61-config-out\") pod \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") "
Jul 22 16:10:07.185036 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.185028    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-config\") pod \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") "
Jul 22 16:10:07.185177 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.185054    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-prometheus-k8s-rulefiles-0\") pod \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") "
Jul 22 16:10:07.185177 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.185078    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-configmap-metrics-client-ca\") pod \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") "
Jul 22 16:10:07.185177 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.185101    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-web-config\") pod \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") "
Jul 22 16:10:07.185364 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.185232    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7mvl7\" (UniqueName: \"kubernetes.io/projected/91cd4a98-ad75-4c0d-942e-3482dcf02e61-kube-api-access-7mvl7\") pod \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") "
Jul 22 16:10:07.185364 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.185267    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-kube-rbac-proxy\") pod \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") "
Jul 22 16:10:07.185364 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.185316    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-metrics-client-certs\") pod \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") "
Jul 22 16:10:07.185364 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.185325    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "91cd4a98-ad75-4c0d-942e-3482dcf02e61" (UID: "91cd4a98-ad75-4c0d-942e-3482dcf02e61"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 16:10:07.185364 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.185346    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-prometheus-trusted-ca-bundle\") pod \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") "
Jul 22 16:10:07.185595 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.185408    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-thanos-prometheus-http-client-file\") pod \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") "
Jul 22 16:10:07.185595 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.185425    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "91cd4a98-ad75-4c0d-942e-3482dcf02e61" (UID: "91cd4a98-ad75-4c0d-942e-3482dcf02e61"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 16:10:07.185595 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.185443    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/91cd4a98-ad75-4c0d-942e-3482dcf02e61-tls-assets\") pod \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\" (UID: \"91cd4a98-ad75-4c0d-942e-3482dcf02e61\") "
Jul 22 16:10:07.185728 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.185667    2563 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:10:07.185728 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.185685    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:10:07.185728 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.185699    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:10:07.185728 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.185696    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "91cd4a98-ad75-4c0d-942e-3482dcf02e61" (UID: "91cd4a98-ad75-4c0d-942e-3482dcf02e61"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 16:10:07.185728 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.185713    2563 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:10:07.186012 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.185787    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91cd4a98-ad75-4c0d-942e-3482dcf02e61-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "91cd4a98-ad75-4c0d-942e-3482dcf02e61" (UID: "91cd4a98-ad75-4c0d-942e-3482dcf02e61"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:10:07.186482 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.186243    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "91cd4a98-ad75-4c0d-942e-3482dcf02e61" (UID: "91cd4a98-ad75-4c0d-942e-3482dcf02e61"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 16:10:07.187083 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.187055    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "91cd4a98-ad75-4c0d-942e-3482dcf02e61" (UID: "91cd4a98-ad75-4c0d-942e-3482dcf02e61"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:10:07.187347 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.187320    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "91cd4a98-ad75-4c0d-942e-3482dcf02e61" (UID: "91cd4a98-ad75-4c0d-942e-3482dcf02e61"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:10:07.187553 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.187529    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "91cd4a98-ad75-4c0d-942e-3482dcf02e61" (UID: "91cd4a98-ad75-4c0d-942e-3482dcf02e61"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:10:07.187815 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.187786    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "91cd4a98-ad75-4c0d-942e-3482dcf02e61" (UID: "91cd4a98-ad75-4c0d-942e-3482dcf02e61"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:10:07.187942 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.187820    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "91cd4a98-ad75-4c0d-942e-3482dcf02e61" (UID: "91cd4a98-ad75-4c0d-942e-3482dcf02e61"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 16:10:07.188086 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.188059    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91cd4a98-ad75-4c0d-942e-3482dcf02e61-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "91cd4a98-ad75-4c0d-942e-3482dcf02e61" (UID: "91cd4a98-ad75-4c0d-942e-3482dcf02e61"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:10:07.188481 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.188449    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91cd4a98-ad75-4c0d-942e-3482dcf02e61-config-out" (OuterVolumeSpecName: "config-out") pod "91cd4a98-ad75-4c0d-942e-3482dcf02e61" (UID: "91cd4a98-ad75-4c0d-942e-3482dcf02e61"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:10:07.189258 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.189231    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-config" (OuterVolumeSpecName: "config") pod "91cd4a98-ad75-4c0d-942e-3482dcf02e61" (UID: "91cd4a98-ad75-4c0d-942e-3482dcf02e61"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:10:07.189258 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.189250    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "91cd4a98-ad75-4c0d-942e-3482dcf02e61" (UID: "91cd4a98-ad75-4c0d-942e-3482dcf02e61"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:10:07.189426 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.189411    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "91cd4a98-ad75-4c0d-942e-3482dcf02e61" (UID: "91cd4a98-ad75-4c0d-942e-3482dcf02e61"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:10:07.189877 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.189844    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91cd4a98-ad75-4c0d-942e-3482dcf02e61-kube-api-access-7mvl7" (OuterVolumeSpecName: "kube-api-access-7mvl7") pod "91cd4a98-ad75-4c0d-942e-3482dcf02e61" (UID: "91cd4a98-ad75-4c0d-942e-3482dcf02e61"). InnerVolumeSpecName "kube-api-access-7mvl7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:10:07.189939 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.189926    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "91cd4a98-ad75-4c0d-942e-3482dcf02e61" (UID: "91cd4a98-ad75-4c0d-942e-3482dcf02e61"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:10:07.198210 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.198188    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-web-config" (OuterVolumeSpecName: "web-config") pod "91cd4a98-ad75-4c0d-942e-3482dcf02e61" (UID: "91cd4a98-ad75-4c0d-942e-3482dcf02e61"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:10:07.286584 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.286548    2563 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-grpc-tls\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:10:07.286584 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.286579    2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-prometheus-k8s-tls\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:10:07.286584 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.286590    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/91cd4a98-ad75-4c0d-942e-3482dcf02e61-prometheus-k8s-db\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:10:07.286816 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.286599    2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:10:07.286816 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.286610    2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:10:07.286816 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.286619    2563 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/91cd4a98-ad75-4c0d-942e-3482dcf02e61-config-out\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:10:07.286816 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.286627    2563 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-config\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:10:07.286816 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.286635    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:10:07.286816 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.286645    2563 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-configmap-metrics-client-ca\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:10:07.286816 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.286653    2563 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-web-config\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:10:07.286816 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.286662    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mvl7\" (UniqueName: \"kubernetes.io/projected/91cd4a98-ad75-4c0d-942e-3482dcf02e61-kube-api-access-7mvl7\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:10:07.286816 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.286670    2563 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-kube-rbac-proxy\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:10:07.286816 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.286678    2563 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-secret-metrics-client-certs\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:10:07.286816 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.286686    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91cd4a98-ad75-4c0d-942e-3482dcf02e61-prometheus-trusted-ca-bundle\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:10:07.286816 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.286695    2563 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/91cd4a98-ad75-4c0d-942e-3482dcf02e61-thanos-prometheus-http-client-file\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:10:07.286816 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.286703    2563 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/91cd4a98-ad75-4c0d-942e-3482dcf02e61-tls-assets\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:10:07.324576 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.324540    2563 generic.go:356] "Generic (PLEG): container finished" podID="91cd4a98-ad75-4c0d-942e-3482dcf02e61" containerID="906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c" exitCode=0
Jul 22 16:10:07.324576 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.324571    2563 generic.go:356] "Generic (PLEG): container finished" podID="91cd4a98-ad75-4c0d-942e-3482dcf02e61" containerID="6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f" exitCode=0
Jul 22 16:10:07.324576 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.324581    2563 generic.go:356] "Generic (PLEG): container finished" podID="91cd4a98-ad75-4c0d-942e-3482dcf02e61" containerID="66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc" exitCode=0
Jul 22 16:10:07.324576 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.324587    2563 generic.go:356] "Generic (PLEG): container finished" podID="91cd4a98-ad75-4c0d-942e-3482dcf02e61" containerID="3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894" exitCode=0
Jul 22 16:10:07.324914 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.324592    2563 generic.go:356] "Generic (PLEG): container finished" podID="91cd4a98-ad75-4c0d-942e-3482dcf02e61" containerID="7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69" exitCode=0
Jul 22 16:10:07.324914 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.324599    2563 generic.go:356] "Generic (PLEG): container finished" podID="91cd4a98-ad75-4c0d-942e-3482dcf02e61" containerID="f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315" exitCode=0
Jul 22 16:10:07.324914 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.324621    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.324914 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.324632    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"91cd4a98-ad75-4c0d-942e-3482dcf02e61","Type":"ContainerDied","Data":"906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c"}
Jul 22 16:10:07.324914 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.324701    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"91cd4a98-ad75-4c0d-942e-3482dcf02e61","Type":"ContainerDied","Data":"6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f"}
Jul 22 16:10:07.324914 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.324730    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"91cd4a98-ad75-4c0d-942e-3482dcf02e61","Type":"ContainerDied","Data":"66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc"}
Jul 22 16:10:07.324914 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.324760    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"91cd4a98-ad75-4c0d-942e-3482dcf02e61","Type":"ContainerDied","Data":"3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894"}
Jul 22 16:10:07.324914 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.324769    2563 scope.go:117] "RemoveContainer" containerID="906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c"
Jul 22 16:10:07.324914 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.324798    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"91cd4a98-ad75-4c0d-942e-3482dcf02e61","Type":"ContainerDied","Data":"7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69"}
Jul 22 16:10:07.324914 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.324839    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"91cd4a98-ad75-4c0d-942e-3482dcf02e61","Type":"ContainerDied","Data":"f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315"}
Jul 22 16:10:07.324914 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.324898    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"91cd4a98-ad75-4c0d-942e-3482dcf02e61","Type":"ContainerDied","Data":"56ff8ac9a581be0aa06dec1780a4835e3df6d330bb4ed6c2786de5b7084e6ae9"}
Jul 22 16:10:07.333161 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.333144    2563 scope.go:117] "RemoveContainer" containerID="6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f"
Jul 22 16:10:07.340176 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.340156    2563 scope.go:117] "RemoveContainer" containerID="66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc"
Jul 22 16:10:07.346624 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.346607    2563 scope.go:117] "RemoveContainer" containerID="3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894"
Jul 22 16:10:07.350501 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.350480    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 16:10:07.354492 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.354471    2563 scope.go:117] "RemoveContainer" containerID="7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69"
Jul 22 16:10:07.354721 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.354692    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 16:10:07.361048 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.361025    2563 scope.go:117] "RemoveContainer" containerID="f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315"
Jul 22 16:10:07.367906 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.367890    2563 scope.go:117] "RemoveContainer" containerID="ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2"
Jul 22 16:10:07.374381 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.374365    2563 scope.go:117] "RemoveContainer" containerID="906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c"
Jul 22 16:10:07.374622 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:10:07.374604    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c\": container with ID starting with 906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c not found: ID does not exist" containerID="906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c"
Jul 22 16:10:07.374673 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.374631    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c"} err="failed to get container status \"906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c\": rpc error: code = NotFound desc = could not find container \"906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c\": container with ID starting with 906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c not found: ID does not exist"
Jul 22 16:10:07.374673 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.374650    2563 scope.go:117] "RemoveContainer" containerID="6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f"
Jul 22 16:10:07.374919 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:10:07.374899    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f\": container with ID starting with 6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f not found: ID does not exist" containerID="6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f"
Jul 22 16:10:07.375015 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.374922    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f"} err="failed to get container status \"6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f\": rpc error: code = NotFound desc = could not find container \"6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f\": container with ID starting with 6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f not found: ID does not exist"
Jul 22 16:10:07.375015 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.374934    2563 scope.go:117] "RemoveContainer" containerID="66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc"
Jul 22 16:10:07.375160 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:10:07.375143    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc\": container with ID starting with 66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc not found: ID does not exist" containerID="66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc"
Jul 22 16:10:07.375197 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.375166    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc"} err="failed to get container status \"66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc\": rpc error: code = NotFound desc = could not find container \"66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc\": container with ID starting with 66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc not found: ID does not exist"
Jul 22 16:10:07.375197 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.375179    2563 scope.go:117] "RemoveContainer" containerID="3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894"
Jul 22 16:10:07.375437 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:10:07.375419    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894\": container with ID starting with 3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894 not found: ID does not exist" containerID="3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894"
Jul 22 16:10:07.375476 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.375450    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894"} err="failed to get container status \"3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894\": rpc error: code = NotFound desc = could not find container \"3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894\": container with ID starting with 3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894 not found: ID does not exist"
Jul 22 16:10:07.375476 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.375468    2563 scope.go:117] "RemoveContainer" containerID="7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69"
Jul 22 16:10:07.375677 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:10:07.375662    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69\": container with ID starting with 7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69 not found: ID does not exist" containerID="7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69"
Jul 22 16:10:07.375715 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.375682    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69"} err="failed to get container status \"7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69\": rpc error: code = NotFound desc = could not find container \"7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69\": container with ID starting with 7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69 not found: ID does not exist"
Jul 22 16:10:07.375715 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.375694    2563 scope.go:117] "RemoveContainer" containerID="f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315"
Jul 22 16:10:07.375982 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:10:07.375966    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315\": container with ID starting with f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315 not found: ID does not exist" containerID="f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315"
Jul 22 16:10:07.376035 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.375988    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315"} err="failed to get container status \"f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315\": rpc error: code = NotFound desc = could not find container \"f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315\": container with ID starting with f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315 not found: ID does not exist"
Jul 22 16:10:07.376035 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.376001    2563 scope.go:117] "RemoveContainer" containerID="ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2"
Jul 22 16:10:07.376193 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:10:07.376180    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2\": container with ID starting with ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2 not found: ID does not exist" containerID="ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2"
Jul 22 16:10:07.376232 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.376196    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2"} err="failed to get container status \"ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2\": rpc error: code = NotFound desc = could not find container \"ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2\": container with ID starting with ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2 not found: ID does not exist"
Jul 22 16:10:07.376232 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.376207    2563 scope.go:117] "RemoveContainer" containerID="906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c"
Jul 22 16:10:07.376369 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.376353    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c"} err="failed to get container status \"906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c\": rpc error: code = NotFound desc = could not find container \"906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c\": container with ID starting with 906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c not found: ID does not exist"
Jul 22 16:10:07.376415 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.376368    2563 scope.go:117] "RemoveContainer" containerID="6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f"
Jul 22 16:10:07.376571 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.376554    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f"} err="failed to get container status \"6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f\": rpc error: code = NotFound desc = could not find container \"6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f\": container with ID starting with 6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f not found: ID does not exist"
Jul 22 16:10:07.376571 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.376570    2563 scope.go:117] "RemoveContainer" containerID="66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc"
Jul 22 16:10:07.376760 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.376745    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc"} err="failed to get container status \"66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc\": rpc error: code = NotFound desc = could not find container \"66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc\": container with ID starting with 66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc not found: ID does not exist"
Jul 22 16:10:07.376760 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.376759    2563 scope.go:117] "RemoveContainer" containerID="3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894"
Jul 22 16:10:07.376938 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.376923    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894"} err="failed to get container status \"3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894\": rpc error: code = NotFound desc = could not find container \"3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894\": container with ID starting with 3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894 not found: ID does not exist"
Jul 22 16:10:07.376975 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.376938    2563 scope.go:117] "RemoveContainer" containerID="7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69"
Jul 22 16:10:07.377111 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.377095    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69"} err="failed to get container status \"7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69\": rpc error: code = NotFound desc = could not find container \"7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69\": container with ID starting with 7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69 not found: ID does not exist"
Jul 22 16:10:07.377153 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.377111    2563 scope.go:117] "RemoveContainer" containerID="f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315"
Jul 22 16:10:07.377294 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.377280    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315"} err="failed to get container status \"f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315\": rpc error: code = NotFound desc = could not find container \"f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315\": container with ID starting with f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315 not found: ID does not exist"
Jul 22 16:10:07.377334 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.377294    2563 scope.go:117] "RemoveContainer" containerID="ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2"
Jul 22 16:10:07.377486 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.377472    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2"} err="failed to get container status \"ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2\": rpc error: code = NotFound desc = could not find container \"ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2\": container with ID starting with ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2 not found: ID does not exist"
Jul 22 16:10:07.377522 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.377486    2563 scope.go:117] "RemoveContainer" containerID="906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c"
Jul 22 16:10:07.377706 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.377690    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c"} err="failed to get container status \"906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c\": rpc error: code = NotFound desc = could not find container \"906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c\": container with ID starting with 906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c not found: ID does not exist"
Jul 22 16:10:07.377746 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.377705    2563 scope.go:117] "RemoveContainer" containerID="6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f"
Jul 22 16:10:07.377913 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.377898    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f"} err="failed to get container status \"6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f\": rpc error: code = NotFound desc = could not find container \"6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f\": container with ID starting with 6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f not found: ID does not exist"
Jul 22 16:10:07.377959 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.377913    2563 scope.go:117] "RemoveContainer" containerID="66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc"
Jul 22 16:10:07.378118 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.378098    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc"} err="failed to get container status \"66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc\": rpc error: code = NotFound desc = could not find container \"66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc\": container with ID starting with 66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc not found: ID does not exist"
Jul 22 16:10:07.378156 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.378119    2563 scope.go:117] "RemoveContainer" containerID="3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894"
Jul 22 16:10:07.378310 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.378293    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894"} err="failed to get container status \"3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894\": rpc error: code = NotFound desc = could not find container \"3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894\": container with ID starting with 3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894 not found: ID does not exist"
Jul 22 16:10:07.378349 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.378310    2563 scope.go:117] "RemoveContainer" containerID="7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69"
Jul 22 16:10:07.378518 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.378495    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69"} err="failed to get container status \"7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69\": rpc error: code = NotFound desc = could not find container \"7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69\": container with ID starting with 7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69 not found: ID does not exist"
Jul 22 16:10:07.378584 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.378521    2563 scope.go:117] "RemoveContainer" containerID="f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315"
Jul 22 16:10:07.378777 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.378761    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315"} err="failed to get container status \"f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315\": rpc error: code = NotFound desc = could not find container \"f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315\": container with ID starting with f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315 not found: ID does not exist"
Jul 22 16:10:07.378820 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.378777    2563 scope.go:117] "RemoveContainer" containerID="ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2"
Jul 22 16:10:07.378992 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.378974    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2"} err="failed to get container status \"ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2\": rpc error: code = NotFound desc = could not find container \"ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2\": container with ID starting with ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2 not found: ID does not exist"
Jul 22 16:10:07.379062 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.378994    2563 scope.go:117] "RemoveContainer" containerID="906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c"
Jul 22 16:10:07.379208 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.379190    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c"} err="failed to get container status \"906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c\": rpc error: code = NotFound desc = could not find container \"906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c\": container with ID starting with 906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c not found: ID does not exist"
Jul 22 16:10:07.379249 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.379209    2563 scope.go:117] "RemoveContainer" containerID="6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f"
Jul 22 16:10:07.379402 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.379385    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f"} err="failed to get container status \"6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f\": rpc error: code = NotFound desc = could not find container \"6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f\": container with ID starting with 6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f not found: ID does not exist"
Jul 22 16:10:07.379468 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.379403    2563 scope.go:117] "RemoveContainer" containerID="66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc"
Jul 22 16:10:07.379626 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.379607    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc"} err="failed to get container status \"66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc\": rpc error: code = NotFound desc = could not find container \"66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc\": container with ID starting with 66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc not found: ID does not exist"
Jul 22 16:10:07.379669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.379628    2563 scope.go:117] "RemoveContainer" containerID="3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894"
Jul 22 16:10:07.379835 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.379818    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894"} err="failed to get container status \"3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894\": rpc error: code = NotFound desc = could not find container \"3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894\": container with ID starting with 3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894 not found: ID does not exist"
Jul 22 16:10:07.379941 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.379834    2563 scope.go:117] "RemoveContainer" containerID="7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69"
Jul 22 16:10:07.380088 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.380069    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69"} err="failed to get container status \"7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69\": rpc error: code = NotFound desc = could not find container \"7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69\": container with ID starting with 7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69 not found: ID does not exist"
Jul 22 16:10:07.380128 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.380088    2563 scope.go:117] "RemoveContainer" containerID="f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315"
Jul 22 16:10:07.380287 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.380272    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315"} err="failed to get container status \"f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315\": rpc error: code = NotFound desc = could not find container \"f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315\": container with ID starting with f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315 not found: ID does not exist"
Jul 22 16:10:07.380287 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.380287    2563 scope.go:117] "RemoveContainer" containerID="ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2"
Jul 22 16:10:07.380498 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.380483    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2"} err="failed to get container status \"ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2\": rpc error: code = NotFound desc = could not find container \"ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2\": container with ID starting with ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2 not found: ID does not exist"
Jul 22 16:10:07.380498 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.380496    2563 scope.go:117] "RemoveContainer" containerID="906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c"
Jul 22 16:10:07.380729 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.380709    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c"} err="failed to get container status \"906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c\": rpc error: code = NotFound desc = could not find container \"906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c\": container with ID starting with 906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c not found: ID does not exist"
Jul 22 16:10:07.380768 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.380733    2563 scope.go:117] "RemoveContainer" containerID="6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f"
Jul 22 16:10:07.380977 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.380958    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f"} err="failed to get container status \"6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f\": rpc error: code = NotFound desc = could not find container \"6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f\": container with ID starting with 6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f not found: ID does not exist"
Jul 22 16:10:07.381023 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.380978    2563 scope.go:117] "RemoveContainer" containerID="66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc"
Jul 22 16:10:07.381180 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.381164    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc"} err="failed to get container status \"66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc\": rpc error: code = NotFound desc = could not find container \"66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc\": container with ID starting with 66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc not found: ID does not exist"
Jul 22 16:10:07.381217 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.381180    2563 scope.go:117] "RemoveContainer" containerID="3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894"
Jul 22 16:10:07.381395 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.381378    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894"} err="failed to get container status \"3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894\": rpc error: code = NotFound desc = could not find container \"3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894\": container with ID starting with 3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894 not found: ID does not exist"
Jul 22 16:10:07.381439 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.381395    2563 scope.go:117] "RemoveContainer" containerID="7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69"
Jul 22 16:10:07.381610 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.381595    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69"} err="failed to get container status \"7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69\": rpc error: code = NotFound desc = could not find container \"7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69\": container with ID starting with 7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69 not found: ID does not exist"
Jul 22 16:10:07.381650 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.381610    2563 scope.go:117] "RemoveContainer" containerID="f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315"
Jul 22 16:10:07.381821 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.381806    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315"} err="failed to get container status \"f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315\": rpc error: code = NotFound desc = could not find container \"f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315\": container with ID starting with f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315 not found: ID does not exist"
Jul 22 16:10:07.381821 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.381819    2563 scope.go:117] "RemoveContainer" containerID="ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2"
Jul 22 16:10:07.382034 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.382017    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2"} err="failed to get container status \"ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2\": rpc error: code = NotFound desc = could not find container \"ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2\": container with ID starting with ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2 not found: ID does not exist"
Jul 22 16:10:07.382080 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.382035    2563 scope.go:117] "RemoveContainer" containerID="906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c"
Jul 22 16:10:07.382225 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.382210    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c"} err="failed to get container status \"906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c\": rpc error: code = NotFound desc = could not find container \"906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c\": container with ID starting with 906a80ff08e5e7b852adad1a10a1d7db74f976b733ccfcc91c58f599599bd11c not found: ID does not exist"
Jul 22 16:10:07.382267 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.382225    2563 scope.go:117] "RemoveContainer" containerID="6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f"
Jul 22 16:10:07.382555 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.382522    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f"} err="failed to get container status \"6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f\": rpc error: code = NotFound desc = could not find container \"6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f\": container with ID starting with 6231982a841efcd0dae136d21aef24994c335f66466c881a455cb1269f96690f not found: ID does not exist"
Jul 22 16:10:07.382555 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.382552    2563 scope.go:117] "RemoveContainer" containerID="66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc"
Jul 22 16:10:07.382877 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.382823    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc"} err="failed to get container status \"66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc\": rpc error: code = NotFound desc = could not find container \"66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc\": container with ID starting with 66b86ed3a8a2703555a3c33bf56499f821c23e078baae2a1207a926ed4fae8bc not found: ID does not exist"
Jul 22 16:10:07.382877 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.382848    2563 scope.go:117] "RemoveContainer" containerID="3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894"
Jul 22 16:10:07.383359 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.383334    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894"} err="failed to get container status \"3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894\": rpc error: code = NotFound desc = could not find container \"3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894\": container with ID starting with 3ed949c7f53a8455670e892c79de3073868ce35cb7fe2c6ebad9ecf657e3f894 not found: ID does not exist"
Jul 22 16:10:07.383359 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.383362    2563 scope.go:117] "RemoveContainer" containerID="7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69"
Jul 22 16:10:07.383604 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.383576    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69"} err="failed to get container status \"7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69\": rpc error: code = NotFound desc = could not find container \"7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69\": container with ID starting with 7bf0b7de42de4dd24ab698c39502b73845d0c52fc07217710bc1f8189dfe0a69 not found: ID does not exist"
Jul 22 16:10:07.383669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.383605    2563 scope.go:117] "RemoveContainer" containerID="f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315"
Jul 22 16:10:07.383846 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.383827    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315"} err="failed to get container status \"f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315\": rpc error: code = NotFound desc = could not find container \"f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315\": container with ID starting with f69fe44ed25fd267ae725521b77497b12ad52bb3c12ebfc48e7d46707a0c5315 not found: ID does not exist"
Jul 22 16:10:07.383950 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.383845    2563 scope.go:117] "RemoveContainer" containerID="ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2"
Jul 22 16:10:07.384138 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.384119    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2"} err="failed to get container status \"ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2\": rpc error: code = NotFound desc = could not find container \"ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2\": container with ID starting with ba0d652ceb0d99179c53d8d1ea9dbcd49972b5c58d5840b4f550c51dd1a7cea2 not found: ID does not exist"
Jul 22 16:10:07.384488 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.384471    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 16:10:07.384764 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.384749    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c4b2472-6e15-4d35-94e1-efa2a862db97" containerName="alertmanager"
Jul 22 16:10:07.384835 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.384767    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c4b2472-6e15-4d35-94e1-efa2a862db97" containerName="alertmanager"
Jul 22 16:10:07.384835 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.384782    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c4b2472-6e15-4d35-94e1-efa2a862db97" containerName="kube-rbac-proxy"
Jul 22 16:10:07.384835 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.384791    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c4b2472-6e15-4d35-94e1-efa2a862db97" containerName="kube-rbac-proxy"
Jul 22 16:10:07.384835 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.384804    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c4b2472-6e15-4d35-94e1-efa2a862db97" containerName="prom-label-proxy"
Jul 22 16:10:07.384835 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.384813    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c4b2472-6e15-4d35-94e1-efa2a862db97" containerName="prom-label-proxy"
Jul 22 16:10:07.384835 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.384822    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91cd4a98-ad75-4c0d-942e-3482dcf02e61" containerName="config-reloader"
Jul 22 16:10:07.384835 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.384830    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="91cd4a98-ad75-4c0d-942e-3482dcf02e61" containerName="config-reloader"
Jul 22 16:10:07.385179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.384840    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91cd4a98-ad75-4c0d-942e-3482dcf02e61" containerName="kube-rbac-proxy-web"
Jul 22 16:10:07.385179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.384866    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="91cd4a98-ad75-4c0d-942e-3482dcf02e61" containerName="kube-rbac-proxy-web"
Jul 22 16:10:07.385179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.384881    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c4b2472-6e15-4d35-94e1-efa2a862db97" containerName="init-config-reloader"
Jul 22 16:10:07.385179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.384890    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c4b2472-6e15-4d35-94e1-efa2a862db97" containerName="init-config-reloader"
Jul 22 16:10:07.385179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.384902    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91cd4a98-ad75-4c0d-942e-3482dcf02e61" containerName="prometheus"
Jul 22 16:10:07.385179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.384910    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="91cd4a98-ad75-4c0d-942e-3482dcf02e61" containerName="prometheus"
Jul 22 16:10:07.385179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.384919    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91cd4a98-ad75-4c0d-942e-3482dcf02e61" containerName="kube-rbac-proxy"
Jul 22 16:10:07.385179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.384928    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="91cd4a98-ad75-4c0d-942e-3482dcf02e61" containerName="kube-rbac-proxy"
Jul 22 16:10:07.385179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.384938    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91cd4a98-ad75-4c0d-942e-3482dcf02e61" containerName="init-config-reloader"
Jul 22 16:10:07.385179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.384947    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="91cd4a98-ad75-4c0d-942e-3482dcf02e61" containerName="init-config-reloader"
Jul 22 16:10:07.385179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.384961    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c4b2472-6e15-4d35-94e1-efa2a862db97" containerName="config-reloader"
Jul 22 16:10:07.385179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.384969    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c4b2472-6e15-4d35-94e1-efa2a862db97" containerName="config-reloader"
Jul 22 16:10:07.385179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.384977    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c4b2472-6e15-4d35-94e1-efa2a862db97" containerName="kube-rbac-proxy-metric"
Jul 22 16:10:07.385179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.384986    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c4b2472-6e15-4d35-94e1-efa2a862db97" containerName="kube-rbac-proxy-metric"
Jul 22 16:10:07.385179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.384998    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c4b2472-6e15-4d35-94e1-efa2a862db97" containerName="kube-rbac-proxy-web"
Jul 22 16:10:07.385179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.385005    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c4b2472-6e15-4d35-94e1-efa2a862db97" containerName="kube-rbac-proxy-web"
Jul 22 16:10:07.385179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.385014    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91cd4a98-ad75-4c0d-942e-3482dcf02e61" containerName="thanos-sidecar"
Jul 22 16:10:07.385179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.385022    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="91cd4a98-ad75-4c0d-942e-3482dcf02e61" containerName="thanos-sidecar"
Jul 22 16:10:07.385179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.385032    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91cd4a98-ad75-4c0d-942e-3482dcf02e61" containerName="kube-rbac-proxy-thanos"
Jul 22 16:10:07.385179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.385040    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="91cd4a98-ad75-4c0d-942e-3482dcf02e61" containerName="kube-rbac-proxy-thanos"
Jul 22 16:10:07.385179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.385107    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="91cd4a98-ad75-4c0d-942e-3482dcf02e61" containerName="config-reloader"
Jul 22 16:10:07.385179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.385121    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c4b2472-6e15-4d35-94e1-efa2a862db97" containerName="kube-rbac-proxy"
Jul 22 16:10:07.385179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.385131    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c4b2472-6e15-4d35-94e1-efa2a862db97" containerName="kube-rbac-proxy-metric"
Jul 22 16:10:07.385179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.385140    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="91cd4a98-ad75-4c0d-942e-3482dcf02e61" containerName="prometheus"
Jul 22 16:10:07.385179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.385149    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="91cd4a98-ad75-4c0d-942e-3482dcf02e61" containerName="kube-rbac-proxy-web"
Jul 22 16:10:07.385179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.385157    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="91cd4a98-ad75-4c0d-942e-3482dcf02e61" containerName="kube-rbac-proxy"
Jul 22 16:10:07.385179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.385164    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="91cd4a98-ad75-4c0d-942e-3482dcf02e61" containerName="kube-rbac-proxy-thanos"
Jul 22 16:10:07.385179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.385172    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c4b2472-6e15-4d35-94e1-efa2a862db97" containerName="alertmanager"
Jul 22 16:10:07.385179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.385180    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="91cd4a98-ad75-4c0d-942e-3482dcf02e61" containerName="thanos-sidecar"
Jul 22 16:10:07.385179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.385190    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c4b2472-6e15-4d35-94e1-efa2a862db97" containerName="config-reloader"
Jul 22 16:10:07.386488 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.385197    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c4b2472-6e15-4d35-94e1-efa2a862db97" containerName="kube-rbac-proxy-web"
Jul 22 16:10:07.386488 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.385204    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c4b2472-6e15-4d35-94e1-efa2a862db97" containerName="prom-label-proxy"
Jul 22 16:10:07.389178 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.389160    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.392205 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.392181    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 22 16:10:07.392315 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.392208    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 22 16:10:07.392315 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.392251    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 22 16:10:07.392504 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.392480    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-6676b\""
Jul 22 16:10:07.392504 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.392492    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 22 16:10:07.392632 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.392512    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 22 16:10:07.392632 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.392487    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 22 16:10:07.392715 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.392651    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 22 16:10:07.392841 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.392824    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-14st95tns06dl\""
Jul 22 16:10:07.392969 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.392937    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 22 16:10:07.393089 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.393076    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 22 16:10:07.393152 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.393137    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 22 16:10:07.393762 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.393702    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 22 16:10:07.393762 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.393752    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 22 16:10:07.393928 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.393837    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 22 16:10:07.396058 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.396038    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 22 16:10:07.398696 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.398679    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 22 16:10:07.412496 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.408089    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 16:10:07.488843 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.488750    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c800f5b3-a21b-4128-baf5-09b8819c1a16-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.488843 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.488797    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2b9zw\" (UniqueName: \"kubernetes.io/projected/c800f5b3-a21b-4128-baf5-09b8819c1a16-kube-api-access-2b9zw\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.488843 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.488817    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c800f5b3-a21b-4128-baf5-09b8819c1a16-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.489073 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.488913    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c800f5b3-a21b-4128-baf5-09b8819c1a16-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.489073 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.488955    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c800f5b3-a21b-4128-baf5-09b8819c1a16-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.489073 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.488977    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c800f5b3-a21b-4128-baf5-09b8819c1a16-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.489073 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.488992    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c800f5b3-a21b-4128-baf5-09b8819c1a16-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.489073 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.489014    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c800f5b3-a21b-4128-baf5-09b8819c1a16-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.489073 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.489054    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c800f5b3-a21b-4128-baf5-09b8819c1a16-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.489247 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.489079    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c800f5b3-a21b-4128-baf5-09b8819c1a16-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.489247 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.489100    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c800f5b3-a21b-4128-baf5-09b8819c1a16-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.489247 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.489122    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c800f5b3-a21b-4128-baf5-09b8819c1a16-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.489247 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.489139    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c800f5b3-a21b-4128-baf5-09b8819c1a16-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.489247 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.489174    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c800f5b3-a21b-4128-baf5-09b8819c1a16-config\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.489247 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.489199    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c800f5b3-a21b-4128-baf5-09b8819c1a16-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.489247 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.489213    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c800f5b3-a21b-4128-baf5-09b8819c1a16-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.489247 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.489237    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c800f5b3-a21b-4128-baf5-09b8819c1a16-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.489472 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.489258    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c800f5b3-a21b-4128-baf5-09b8819c1a16-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.489472 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.489274    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c800f5b3-a21b-4128-baf5-09b8819c1a16-config-out\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.489472 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.489289    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c800f5b3-a21b-4128-baf5-09b8819c1a16-web-config\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.590171 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.590127    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c800f5b3-a21b-4128-baf5-09b8819c1a16-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.590171 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.590177    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c800f5b3-a21b-4128-baf5-09b8819c1a16-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.590280 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.590270    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c800f5b3-a21b-4128-baf5-09b8819c1a16-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.590312 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.590287    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c800f5b3-a21b-4128-baf5-09b8819c1a16-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.590312 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.590305    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c800f5b3-a21b-4128-baf5-09b8819c1a16-config-out\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.590378 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.590321    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c800f5b3-a21b-4128-baf5-09b8819c1a16-web-config\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.590378 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.590356    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c800f5b3-a21b-4128-baf5-09b8819c1a16-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.591395 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.590533    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2b9zw\" (UniqueName: \"kubernetes.io/projected/c800f5b3-a21b-4128-baf5-09b8819c1a16-kube-api-access-2b9zw\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.591395 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.590583    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c800f5b3-a21b-4128-baf5-09b8819c1a16-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.591395 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.590628    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c800f5b3-a21b-4128-baf5-09b8819c1a16-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.591395 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.590662    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c800f5b3-a21b-4128-baf5-09b8819c1a16-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.591395 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.590688    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c800f5b3-a21b-4128-baf5-09b8819c1a16-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.591395 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.590713    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c800f5b3-a21b-4128-baf5-09b8819c1a16-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.591395 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.590743    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c800f5b3-a21b-4128-baf5-09b8819c1a16-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.591395 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.590751    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c800f5b3-a21b-4128-baf5-09b8819c1a16-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.591395 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.590822    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c800f5b3-a21b-4128-baf5-09b8819c1a16-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.591395 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.590892    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c800f5b3-a21b-4128-baf5-09b8819c1a16-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.591395 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.590900    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c800f5b3-a21b-4128-baf5-09b8819c1a16-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.591395 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.590944    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c800f5b3-a21b-4128-baf5-09b8819c1a16-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.591395 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.590968    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c800f5b3-a21b-4128-baf5-09b8819c1a16-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.591395 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.590981    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c800f5b3-a21b-4128-baf5-09b8819c1a16-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.592194 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.590989    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c800f5b3-a21b-4128-baf5-09b8819c1a16-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.592194 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.591242    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c800f5b3-a21b-4128-baf5-09b8819c1a16-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.592194 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.591425    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c800f5b3-a21b-4128-baf5-09b8819c1a16-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.592194 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.591638    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c800f5b3-a21b-4128-baf5-09b8819c1a16-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.592194 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.591751    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c800f5b3-a21b-4128-baf5-09b8819c1a16-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.594096 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.593511    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c800f5b3-a21b-4128-baf5-09b8819c1a16-web-config\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.594096 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.593721    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c800f5b3-a21b-4128-baf5-09b8819c1a16-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.594096 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.593768    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c800f5b3-a21b-4128-baf5-09b8819c1a16-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.594303 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.594249    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c800f5b3-a21b-4128-baf5-09b8819c1a16-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.594362 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.594301    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c800f5b3-a21b-4128-baf5-09b8819c1a16-config\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.594586 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.594545    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c800f5b3-a21b-4128-baf5-09b8819c1a16-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.594659 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.594632    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c800f5b3-a21b-4128-baf5-09b8819c1a16-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.594842 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.594804    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c800f5b3-a21b-4128-baf5-09b8819c1a16-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.594842 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.594829    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c800f5b3-a21b-4128-baf5-09b8819c1a16-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.596249 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.596221    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c800f5b3-a21b-4128-baf5-09b8819c1a16-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.596249 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.596234    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c800f5b3-a21b-4128-baf5-09b8819c1a16-config-out\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.596370 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.596283    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c800f5b3-a21b-4128-baf5-09b8819c1a16-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.596370 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.596330    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/c800f5b3-a21b-4128-baf5-09b8819c1a16-config\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.602003 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.601985    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2b9zw\" (UniqueName: \"kubernetes.io/projected/c800f5b3-a21b-4128-baf5-09b8819c1a16-kube-api-access-2b9zw\") pod \"prometheus-k8s-0\" (UID: \"c800f5b3-a21b-4128-baf5-09b8819c1a16\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.699436 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.699402    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:07.836088 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:07.836052    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 16:10:07.839182 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:10:07.839149    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc800f5b3_a21b_4128_baf5_09b8819c1a16.slice/crio-c685cf448b6a3037581e2616ba8539af496790b254031c3c84279621889f9ca0 WatchSource:0}: Error finding container c685cf448b6a3037581e2616ba8539af496790b254031c3c84279621889f9ca0: Status 404 returned error can't find the container with id c685cf448b6a3037581e2616ba8539af496790b254031c3c84279621889f9ca0
Jul 22 16:10:08.329464 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:08.329431    2563 generic.go:356] "Generic (PLEG): container finished" podID="c800f5b3-a21b-4128-baf5-09b8819c1a16" containerID="4cc52eaf9bd5089932c5e9a5e81a5f4fd8978375f5ddb63533ca22013106cfd1" exitCode=0
Jul 22 16:10:08.329634 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:08.329526    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c800f5b3-a21b-4128-baf5-09b8819c1a16","Type":"ContainerDied","Data":"4cc52eaf9bd5089932c5e9a5e81a5f4fd8978375f5ddb63533ca22013106cfd1"}
Jul 22 16:10:08.329634 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:08.329619    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c800f5b3-a21b-4128-baf5-09b8819c1a16","Type":"ContainerStarted","Data":"c685cf448b6a3037581e2616ba8539af496790b254031c3c84279621889f9ca0"}
Jul 22 16:10:08.448437 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:08.448407    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91cd4a98-ad75-4c0d-942e-3482dcf02e61" path="/var/lib/kubelet/pods/91cd4a98-ad75-4c0d-942e-3482dcf02e61/volumes"
Jul 22 16:10:09.337559 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:09.337524    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c800f5b3-a21b-4128-baf5-09b8819c1a16","Type":"ContainerStarted","Data":"a5f5824c211b5812a6b71a7f03ae737120e054ecefa6a6d40153e5593bae9e94"}
Jul 22 16:10:09.338012 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:09.337586    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c800f5b3-a21b-4128-baf5-09b8819c1a16","Type":"ContainerStarted","Data":"5d957e7b6479e305ed58dce9b6b8f1c085482423f8a118a3bd54f232086ed77d"}
Jul 22 16:10:09.338012 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:09.337615    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c800f5b3-a21b-4128-baf5-09b8819c1a16","Type":"ContainerStarted","Data":"8482f82cebe391b8ddb849c7be78749c58a029220fca812fc69a560bf4f91482"}
Jul 22 16:10:09.338012 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:09.337639    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c800f5b3-a21b-4128-baf5-09b8819c1a16","Type":"ContainerStarted","Data":"39661846eb165bc54288212ea317f0102b604f501832520007c07fcde602cc58"}
Jul 22 16:10:09.338012 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:09.337661    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c800f5b3-a21b-4128-baf5-09b8819c1a16","Type":"ContainerStarted","Data":"bc43868e655aace2e73b6867597dc4b40640090020ad53caf353d3a712f76190"}
Jul 22 16:10:09.338012 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:09.337685    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c800f5b3-a21b-4128-baf5-09b8819c1a16","Type":"ContainerStarted","Data":"d23d2010060f02e2a1a66ebbfc306030fa7ed751126c93420dd4b78059526e68"}
Jul 22 16:10:09.374239 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:09.374182    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.350531967 podStartE2EDuration="2.374165478s" totalImagesPullingTime="0s" totalInitContainerRuntime="23.633511ms" isStatefulPod=false podCreationTimestamp="2026-07-22 16:10:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:10:09.372302289 +0000 UTC m=+169.502772310" watchObservedRunningTime="2026-07-22 16:10:09.374165478 +0000 UTC m=+169.504635479"
Jul 22 16:10:11.573445 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:11.573339    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:10:11.674174 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:11.674131    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:10:12.700254 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:12.700211    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:10:22.112465 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:22.112411    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:10:22.216428 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:22.216386    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:10:22.281847 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:22.281815    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:10:56.566283 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:56.566241    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:10:56.655200 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:10:56.655169    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:11:07.700362 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:11:07.700326    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:11:07.715652 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:11:07.715625    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:11:08.556818 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:11:08.556788    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 16:11:08.635753 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:11:08.635718    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:11:16.659684 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:11:16.659646    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:11:16.762482 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:11:16.762447    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:11:22.848973 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:11:22.848930    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:11:22.925118 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:11:22.925073    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:11:28.617579 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:11:28.617535    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:11:28.693319 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:11:28.693282    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:11:33.443777 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:11:33.443746    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:11:33.540875 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:11:33.540830    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:11:33.611731 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:11:33.611696    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:11:36.702184 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:11:36.702147    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:11:44.658034 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:11:44.657957    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:11:44.779706 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:11:44.779671    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:11:44.882827 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:11:44.882790    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:12:20.249842 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:12:20.249809    2563 kubelet.go:1648] "Image garbage collection succeeded"
Jul 22 16:12:23.540045 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:12:23.540010    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:12:25.449230 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:12:25.449195    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:12:52.679544 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:12:52.679506    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:12:52.791200 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:12:52.791167    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:12:57.138559 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:12:57.138522    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:12:57.365219 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:12:57.365180    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:12:59.504295 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:12:59.504257    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:12:59.809419 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:12:59.809331    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:00.211529 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:00.211495    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:00.468052 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:00.467957    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:02.146095 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:02.146042    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:02.261311 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:02.261275    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:03.444022 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:03.443983    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:03.651643 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:03.651611    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:08.152059 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:08.152028    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:08.547206 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:08.547173    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:09.232077 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:09.232043    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:12.659782 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:12.659702    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:12.884162 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:12.884134    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:19.602893 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:19.602836    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:19.773267 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:19.773228    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:21.956566 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:21.956528    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:32.199230 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:32.199197    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:33.142218 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:33.142174    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:33.292675 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:33.292644    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:33.694523 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:33.694492    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:33.877900 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:33.877847    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:34.120277 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:34.120184    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:34.759914 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:34.759845    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:34.962002 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:34.961972    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:34.997066 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:34.997036    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-587bf8b54-52jlh"]
Jul 22 16:13:34.999213 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:34.999198    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-52jlh"
Jul 22 16:13:35.021604 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:35.021536    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 22 16:13:35.033224 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:35.033201    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 22 16:13:35.033376 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:35.033207    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 22 16:13:35.033376 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:35.033207    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-x4fjv\""
Jul 22 16:13:35.033376 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:35.033264    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 22 16:13:35.068038 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:35.068011    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-587bf8b54-52jlh"]
Jul 22 16:13:35.080914 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:35.080887    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/15b90659-90dd-4fd8-aaf5-1e97d6aad202-tmp\") pod \"opendatahub-operator-controller-manager-587bf8b54-52jlh\" (UID: \"15b90659-90dd-4fd8-aaf5-1e97d6aad202\") " pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-52jlh"
Jul 22 16:13:35.081056 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:35.080924    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppktr\" (UniqueName: \"kubernetes.io/projected/15b90659-90dd-4fd8-aaf5-1e97d6aad202-kube-api-access-ppktr\") pod \"opendatahub-operator-controller-manager-587bf8b54-52jlh\" (UID: \"15b90659-90dd-4fd8-aaf5-1e97d6aad202\") " pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-52jlh"
Jul 22 16:13:35.081056 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:35.080951    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/15b90659-90dd-4fd8-aaf5-1e97d6aad202-manifests\") pod \"opendatahub-operator-controller-manager-587bf8b54-52jlh\" (UID: \"15b90659-90dd-4fd8-aaf5-1e97d6aad202\") " pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-52jlh"
Jul 22 16:13:35.081056 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:35.081028    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/15b90659-90dd-4fd8-aaf5-1e97d6aad202-webhook-cert\") pod \"opendatahub-operator-controller-manager-587bf8b54-52jlh\" (UID: \"15b90659-90dd-4fd8-aaf5-1e97d6aad202\") " pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-52jlh"
Jul 22 16:13:35.081259 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:35.081125    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/15b90659-90dd-4fd8-aaf5-1e97d6aad202-apiservice-cert\") pod \"opendatahub-operator-controller-manager-587bf8b54-52jlh\" (UID: \"15b90659-90dd-4fd8-aaf5-1e97d6aad202\") " pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-52jlh"
Jul 22 16:13:35.181653 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:35.181616    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/15b90659-90dd-4fd8-aaf5-1e97d6aad202-tmp\") pod \"opendatahub-operator-controller-manager-587bf8b54-52jlh\" (UID: \"15b90659-90dd-4fd8-aaf5-1e97d6aad202\") " pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-52jlh"
Jul 22 16:13:35.181653 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:35.181656    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppktr\" (UniqueName: \"kubernetes.io/projected/15b90659-90dd-4fd8-aaf5-1e97d6aad202-kube-api-access-ppktr\") pod \"opendatahub-operator-controller-manager-587bf8b54-52jlh\" (UID: \"15b90659-90dd-4fd8-aaf5-1e97d6aad202\") " pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-52jlh"
Jul 22 16:13:35.181875 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:35.181762    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/15b90659-90dd-4fd8-aaf5-1e97d6aad202-manifests\") pod \"opendatahub-operator-controller-manager-587bf8b54-52jlh\" (UID: \"15b90659-90dd-4fd8-aaf5-1e97d6aad202\") " pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-52jlh"
Jul 22 16:13:35.181875 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:35.181814    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/15b90659-90dd-4fd8-aaf5-1e97d6aad202-webhook-cert\") pod \"opendatahub-operator-controller-manager-587bf8b54-52jlh\" (UID: \"15b90659-90dd-4fd8-aaf5-1e97d6aad202\") " pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-52jlh"
Jul 22 16:13:35.181989 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:35.181918    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/15b90659-90dd-4fd8-aaf5-1e97d6aad202-apiservice-cert\") pod \"opendatahub-operator-controller-manager-587bf8b54-52jlh\" (UID: \"15b90659-90dd-4fd8-aaf5-1e97d6aad202\") " pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-52jlh"
Jul 22 16:13:35.182124 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:35.182102    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/15b90659-90dd-4fd8-aaf5-1e97d6aad202-manifests\") pod \"opendatahub-operator-controller-manager-587bf8b54-52jlh\" (UID: \"15b90659-90dd-4fd8-aaf5-1e97d6aad202\") " pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-52jlh"
Jul 22 16:13:35.184031 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:35.184000    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/15b90659-90dd-4fd8-aaf5-1e97d6aad202-tmp\") pod \"opendatahub-operator-controller-manager-587bf8b54-52jlh\" (UID: \"15b90659-90dd-4fd8-aaf5-1e97d6aad202\") " pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-52jlh"
Jul 22 16:13:35.184278 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:35.184259    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/15b90659-90dd-4fd8-aaf5-1e97d6aad202-webhook-cert\") pod \"opendatahub-operator-controller-manager-587bf8b54-52jlh\" (UID: \"15b90659-90dd-4fd8-aaf5-1e97d6aad202\") " pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-52jlh"
Jul 22 16:13:35.184346 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:35.184260    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/15b90659-90dd-4fd8-aaf5-1e97d6aad202-apiservice-cert\") pod \"opendatahub-operator-controller-manager-587bf8b54-52jlh\" (UID: \"15b90659-90dd-4fd8-aaf5-1e97d6aad202\") " pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-52jlh"
Jul 22 16:13:35.204530 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:35.204504    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppktr\" (UniqueName: \"kubernetes.io/projected/15b90659-90dd-4fd8-aaf5-1e97d6aad202-kube-api-access-ppktr\") pod \"opendatahub-operator-controller-manager-587bf8b54-52jlh\" (UID: \"15b90659-90dd-4fd8-aaf5-1e97d6aad202\") " pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-52jlh"
Jul 22 16:13:35.309438 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:35.309350    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-52jlh"
Jul 22 16:13:35.406727 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:35.406682    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:35.510983 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:35.510948    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-587bf8b54-52jlh"]
Jul 22 16:13:35.513284 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:13:35.513259    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15b90659_90dd_4fd8_aaf5_1e97d6aad202.slice/crio-d4180669cdab8bf5c2b393ff102b675fd5e8bcdefa3974a7db5619cc1de440cf WatchSource:0}: Error finding container d4180669cdab8bf5c2b393ff102b675fd5e8bcdefa3974a7db5619cc1de440cf: Status 404 returned error can't find the container with id d4180669cdab8bf5c2b393ff102b675fd5e8bcdefa3974a7db5619cc1de440cf
Jul 22 16:13:35.515193 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:35.515173    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 16:13:35.670422 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:35.670333    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:35.796092 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:35.796058    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:36.025138 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:36.025105    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:36.052024 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:36.051985    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-52jlh" event={"ID":"15b90659-90dd-4fd8-aaf5-1e97d6aad202","Type":"ContainerStarted","Data":"d4180669cdab8bf5c2b393ff102b675fd5e8bcdefa3974a7db5619cc1de440cf"}
Jul 22 16:13:36.157539 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:36.157491    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:36.359134 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:36.358953    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-b9d69d95c-hjlf7"]
Jul 22 16:13:36.362158 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:36.361622    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-b9d69d95c-hjlf7"
Jul 22 16:13:36.367612 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:36.367566    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 22 16:13:36.369127 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:36.369102    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 22 16:13:36.369250 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:36.369229    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 22 16:13:36.369324 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:36.369268    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 22 16:13:36.369380 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:36.369343    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-2tkjq\""
Jul 22 16:13:36.375029 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:36.375007    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 22 16:13:36.438186 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:36.438148    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:36.493549 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:36.493521    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c829e2d9-e402-4105-a098-64dc65ca6574-metrics-cert\") pod \"lws-controller-manager-b9d69d95c-hjlf7\" (UID: \"c829e2d9-e402-4105-a098-64dc65ca6574\") " pod="openshift-lws-operator/lws-controller-manager-b9d69d95c-hjlf7"
Jul 22 16:13:36.493728 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:36.493564    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c829e2d9-e402-4105-a098-64dc65ca6574-cert\") pod \"lws-controller-manager-b9d69d95c-hjlf7\" (UID: \"c829e2d9-e402-4105-a098-64dc65ca6574\") " pod="openshift-lws-operator/lws-controller-manager-b9d69d95c-hjlf7"
Jul 22 16:13:36.493728 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:36.493608    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnvsf\" (UniqueName: \"kubernetes.io/projected/c829e2d9-e402-4105-a098-64dc65ca6574-kube-api-access-nnvsf\") pod \"lws-controller-manager-b9d69d95c-hjlf7\" (UID: \"c829e2d9-e402-4105-a098-64dc65ca6574\") " pod="openshift-lws-operator/lws-controller-manager-b9d69d95c-hjlf7"
Jul 22 16:13:36.493728 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:36.493647    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c829e2d9-e402-4105-a098-64dc65ca6574-manager-config\") pod \"lws-controller-manager-b9d69d95c-hjlf7\" (UID: \"c829e2d9-e402-4105-a098-64dc65ca6574\") " pod="openshift-lws-operator/lws-controller-manager-b9d69d95c-hjlf7"
Jul 22 16:13:36.515115 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:36.515045    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-b9d69d95c-hjlf7"]
Jul 22 16:13:36.594755 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:36.594715    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c829e2d9-e402-4105-a098-64dc65ca6574-metrics-cert\") pod \"lws-controller-manager-b9d69d95c-hjlf7\" (UID: \"c829e2d9-e402-4105-a098-64dc65ca6574\") " pod="openshift-lws-operator/lws-controller-manager-b9d69d95c-hjlf7"
Jul 22 16:13:36.594968 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:36.594793    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c829e2d9-e402-4105-a098-64dc65ca6574-cert\") pod \"lws-controller-manager-b9d69d95c-hjlf7\" (UID: \"c829e2d9-e402-4105-a098-64dc65ca6574\") " pod="openshift-lws-operator/lws-controller-manager-b9d69d95c-hjlf7"
Jul 22 16:13:36.595220 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:36.595182    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnvsf\" (UniqueName: \"kubernetes.io/projected/c829e2d9-e402-4105-a098-64dc65ca6574-kube-api-access-nnvsf\") pod \"lws-controller-manager-b9d69d95c-hjlf7\" (UID: \"c829e2d9-e402-4105-a098-64dc65ca6574\") " pod="openshift-lws-operator/lws-controller-manager-b9d69d95c-hjlf7"
Jul 22 16:13:36.595319 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:36.595279    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c829e2d9-e402-4105-a098-64dc65ca6574-manager-config\") pod \"lws-controller-manager-b9d69d95c-hjlf7\" (UID: \"c829e2d9-e402-4105-a098-64dc65ca6574\") " pod="openshift-lws-operator/lws-controller-manager-b9d69d95c-hjlf7"
Jul 22 16:13:36.596128 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:36.596085    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c829e2d9-e402-4105-a098-64dc65ca6574-manager-config\") pod \"lws-controller-manager-b9d69d95c-hjlf7\" (UID: \"c829e2d9-e402-4105-a098-64dc65ca6574\") " pod="openshift-lws-operator/lws-controller-manager-b9d69d95c-hjlf7"
Jul 22 16:13:36.598396 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:36.598366    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c829e2d9-e402-4105-a098-64dc65ca6574-metrics-cert\") pod \"lws-controller-manager-b9d69d95c-hjlf7\" (UID: \"c829e2d9-e402-4105-a098-64dc65ca6574\") " pod="openshift-lws-operator/lws-controller-manager-b9d69d95c-hjlf7"
Jul 22 16:13:36.598553 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:36.598528    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c829e2d9-e402-4105-a098-64dc65ca6574-cert\") pod \"lws-controller-manager-b9d69d95c-hjlf7\" (UID: \"c829e2d9-e402-4105-a098-64dc65ca6574\") " pod="openshift-lws-operator/lws-controller-manager-b9d69d95c-hjlf7"
Jul 22 16:13:36.621095 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:36.621010    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnvsf\" (UniqueName: \"kubernetes.io/projected/c829e2d9-e402-4105-a098-64dc65ca6574-kube-api-access-nnvsf\") pod \"lws-controller-manager-b9d69d95c-hjlf7\" (UID: \"c829e2d9-e402-4105-a098-64dc65ca6574\") " pod="openshift-lws-operator/lws-controller-manager-b9d69d95c-hjlf7"
Jul 22 16:13:36.675049 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:36.675013    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-b9d69d95c-hjlf7"
Jul 22 16:13:36.856364 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:36.856322    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-b9d69d95c-hjlf7"]
Jul 22 16:13:36.860281 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:13:36.860253    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc829e2d9_e402_4105_a098_64dc65ca6574.slice/crio-0b4d45c9d78d177555bcb63408d347b510828081806f31b527b43e823bb38b41 WatchSource:0}: Error finding container 0b4d45c9d78d177555bcb63408d347b510828081806f31b527b43e823bb38b41: Status 404 returned error can't find the container with id 0b4d45c9d78d177555bcb63408d347b510828081806f31b527b43e823bb38b41
Jul 22 16:13:37.058155 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:37.058117    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-b9d69d95c-hjlf7" event={"ID":"c829e2d9-e402-4105-a098-64dc65ca6574","Type":"ContainerStarted","Data":"0b4d45c9d78d177555bcb63408d347b510828081806f31b527b43e823bb38b41"}
Jul 22 16:13:39.069197 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:39.069150    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-52jlh" event={"ID":"15b90659-90dd-4fd8-aaf5-1e97d6aad202","Type":"ContainerStarted","Data":"6b5a8a7fbd3a7ceb164bcd19f879324e67bb8da686890b86906db078d7f1fb6b"}
Jul 22 16:13:41.077412 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:41.077378    2563 generic.go:356] "Generic (PLEG): container finished" podID="15b90659-90dd-4fd8-aaf5-1e97d6aad202" containerID="6b5a8a7fbd3a7ceb164bcd19f879324e67bb8da686890b86906db078d7f1fb6b" exitCode=0
Jul 22 16:13:41.077838 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:41.077452    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-52jlh" event={"ID":"15b90659-90dd-4fd8-aaf5-1e97d6aad202","Type":"ContainerDied","Data":"6b5a8a7fbd3a7ceb164bcd19f879324e67bb8da686890b86906db078d7f1fb6b"}
Jul 22 16:13:41.078995 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:41.078973    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-b9d69d95c-hjlf7" event={"ID":"c829e2d9-e402-4105-a098-64dc65ca6574","Type":"ContainerStarted","Data":"389ce6f141660c116f74c449702d952979fb45dd64f2d6f1d5f35e92eb8dfacd"}
Jul 22 16:13:41.079071 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:41.079046    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-b9d69d95c-hjlf7"
Jul 22 16:13:41.195576 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:41.195520    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-b9d69d95c-hjlf7" podStartSLOduration=1.589692747 podStartE2EDuration="5.195501438s" totalImagesPullingTime="3.605808691s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:13:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:13:41.193604022 +0000 UTC m=+381.324074057" watchObservedRunningTime="2026-07-22 16:13:41.195501438 +0000 UTC m=+381.325971440"
Jul 22 16:13:42.084565 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:42.084476    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-52jlh" event={"ID":"15b90659-90dd-4fd8-aaf5-1e97d6aad202","Type":"ContainerStarted","Data":"604c03f94573fb06bb6cb6037fb1d879e9298286362a20cf3f498354b981cda0"}
Jul 22 16:13:42.084981 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:42.084711    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-52jlh"
Jul 22 16:13:42.125461 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:42.125402    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-52jlh" podStartSLOduration=2.967669818 podStartE2EDuration="8.125383428s" totalImagesPullingTime="3.81371958s" totalInitContainerRuntime="1.34399403s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:13:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:13:42.123504727 +0000 UTC m=+382.253974742" watchObservedRunningTime="2026-07-22 16:13:42.125383428 +0000 UTC m=+382.255853488"
Jul 22 16:13:50.601191 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:50.601158    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:50.708592 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:50.708559    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:52.086999 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:52.086972    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-b9d69d95c-hjlf7"
Jul 22 16:13:53.090585 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:53.090559    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-52jlh"
Jul 22 16:13:55.118500 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:55.118467    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:55.524011 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:55.523979    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:56.357660 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:56.357630    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:56.621998 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:56.621922    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-9b87bf6fb-g6x9x"]
Jul 22 16:13:56.635869 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:56.635833    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-9b87bf6fb-g6x9x"
Jul 22 16:13:56.641263 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:56.641233    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2692t\" (UniqueName: \"kubernetes.io/projected/79491bd3-166f-4502-b60d-ec1eb758d07a-kube-api-access-2692t\") pod \"kube-auth-proxy-9b87bf6fb-g6x9x\" (UID: \"79491bd3-166f-4502-b60d-ec1eb758d07a\") " pod="openshift-ingress/kube-auth-proxy-9b87bf6fb-g6x9x"
Jul 22 16:13:56.641401 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:56.641293    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/79491bd3-166f-4502-b60d-ec1eb758d07a-tmp\") pod \"kube-auth-proxy-9b87bf6fb-g6x9x\" (UID: \"79491bd3-166f-4502-b60d-ec1eb758d07a\") " pod="openshift-ingress/kube-auth-proxy-9b87bf6fb-g6x9x"
Jul 22 16:13:56.641401 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:56.641385    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79491bd3-166f-4502-b60d-ec1eb758d07a-tls-certs\") pod \"kube-auth-proxy-9b87bf6fb-g6x9x\" (UID: \"79491bd3-166f-4502-b60d-ec1eb758d07a\") " pod="openshift-ingress/kube-auth-proxy-9b87bf6fb-g6x9x"
Jul 22 16:13:56.644480 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:56.644457    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 22 16:13:56.645437 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:56.645420    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 22 16:13:56.646710 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:56.646697    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 22 16:13:56.650148 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:56.650133    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-rrr2z\""
Jul 22 16:13:56.656311 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:56.656296    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 22 16:13:56.677141 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:56.677116    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-9b87bf6fb-g6x9x"]
Jul 22 16:13:56.742512 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:56.742480    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2692t\" (UniqueName: \"kubernetes.io/projected/79491bd3-166f-4502-b60d-ec1eb758d07a-kube-api-access-2692t\") pod \"kube-auth-proxy-9b87bf6fb-g6x9x\" (UID: \"79491bd3-166f-4502-b60d-ec1eb758d07a\") " pod="openshift-ingress/kube-auth-proxy-9b87bf6fb-g6x9x"
Jul 22 16:13:56.742678 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:56.742530    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/79491bd3-166f-4502-b60d-ec1eb758d07a-tmp\") pod \"kube-auth-proxy-9b87bf6fb-g6x9x\" (UID: \"79491bd3-166f-4502-b60d-ec1eb758d07a\") " pod="openshift-ingress/kube-auth-proxy-9b87bf6fb-g6x9x"
Jul 22 16:13:56.742678 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:56.742554    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79491bd3-166f-4502-b60d-ec1eb758d07a-tls-certs\") pod \"kube-auth-proxy-9b87bf6fb-g6x9x\" (UID: \"79491bd3-166f-4502-b60d-ec1eb758d07a\") " pod="openshift-ingress/kube-auth-proxy-9b87bf6fb-g6x9x"
Jul 22 16:13:56.744823 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:56.744798    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/79491bd3-166f-4502-b60d-ec1eb758d07a-tmp\") pod \"kube-auth-proxy-9b87bf6fb-g6x9x\" (UID: \"79491bd3-166f-4502-b60d-ec1eb758d07a\") " pod="openshift-ingress/kube-auth-proxy-9b87bf6fb-g6x9x"
Jul 22 16:13:56.745043 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:56.745026    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79491bd3-166f-4502-b60d-ec1eb758d07a-tls-certs\") pod \"kube-auth-proxy-9b87bf6fb-g6x9x\" (UID: \"79491bd3-166f-4502-b60d-ec1eb758d07a\") " pod="openshift-ingress/kube-auth-proxy-9b87bf6fb-g6x9x"
Jul 22 16:13:56.746686 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:56.746663    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:56.761814 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:56.761791    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2692t\" (UniqueName: \"kubernetes.io/projected/79491bd3-166f-4502-b60d-ec1eb758d07a-kube-api-access-2692t\") pod \"kube-auth-proxy-9b87bf6fb-g6x9x\" (UID: \"79491bd3-166f-4502-b60d-ec1eb758d07a\") " pod="openshift-ingress/kube-auth-proxy-9b87bf6fb-g6x9x"
Jul 22 16:13:56.945738 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:56.945710    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-9b87bf6fb-g6x9x"
Jul 22 16:13:57.041101 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:57.041069    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:57.226427 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:57.226351    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-9b87bf6fb-g6x9x"]
Jul 22 16:13:57.258940 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:13:57.258905    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79491bd3_166f_4502_b60d_ec1eb758d07a.slice/crio-5e8326d181e2d993f445f4a120e022d3a3a1ab296452a3d4307886d7359b7b3f WatchSource:0}: Error finding container 5e8326d181e2d993f445f4a120e022d3a3a1ab296452a3d4307886d7359b7b3f: Status 404 returned error can't find the container with id 5e8326d181e2d993f445f4a120e022d3a3a1ab296452a3d4307886d7359b7b3f
Jul 22 16:13:57.360194 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:57.360159    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:57.675372 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:57.675289    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:58.145142 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:58.145104    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-9b87bf6fb-g6x9x" event={"ID":"79491bd3-166f-4502-b60d-ec1eb758d07a","Type":"ContainerStarted","Data":"5e8326d181e2d993f445f4a120e022d3a3a1ab296452a3d4307886d7359b7b3f"}
Jul 22 16:13:59.123929 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:59.123895    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:13:59.592058 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:13:59.592021    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:14:00.439170 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:14:00.439135    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:14:00.758291 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:14:00.758244    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:14:01.158704 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:14:01.158623    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-9b87bf6fb-g6x9x" event={"ID":"79491bd3-166f-4502-b60d-ec1eb758d07a","Type":"ContainerStarted","Data":"ddcc5609379dc231faf90f87bd1c13a6ebd3fb62c535a53d89e25ac7a87bf28c"}
Jul 22 16:14:01.216023 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:14:01.215963    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-9b87bf6fb-g6x9x" podStartSLOduration=2.201728624 podStartE2EDuration="5.215945331s" totalImagesPullingTime="3.014216707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:13:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:14:01.210960506 +0000 UTC m=+401.341430505" watchObservedRunningTime="2026-07-22 16:14:01.215945331 +0000 UTC m=+401.346415332"
Jul 22 16:14:04.540521 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:14:04.540490    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:14:06.082468 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:14:06.082433    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:14:06.266544 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:14:06.266511    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:14:08.191423 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:14:08.191385    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:14:08.436633 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:14:08.436602    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:14:08.778320 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:14:08.778285    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:14:13.840905 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:14:13.840865    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:14:14.078684 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:14:14.078652    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:14:14.269986 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:14:14.269953    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:14:15.427242 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:14:15.427204    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:14:15.644569 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:14:15.644536    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:14:16.037612 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:14:16.037581    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:14:31.523993 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:14:31.523955    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:14:37.203070 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:14:37.203039    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:14:56.987836 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:14:56.987751    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:14:57.131707 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:14:57.131672    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:14:57.246660 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:14:57.246567    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:15:11.685017 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:11.684977    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:15:11.748482 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:11.748446    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:15:11.832321 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:11.832289    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:15:11.903037 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:11.902999    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:15:12.415810 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:12.415762    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:15:22.516179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:22.516146    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:15:22.614135 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:22.614104    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:15:22.713814 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:22.713785    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:15:25.629385 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:25.629354    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:15:26.415567 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:26.415539    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:15:26.722056 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:26.722024    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:15:26.824960 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:26.824923    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:15:30.646375 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:30.646338    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:15:33.048737 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:33.048703    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:15:33.298829 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:33.298743    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:15:35.540497 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:35.540465    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:15:35.717152 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:35.717117    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:15:38.333531 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:38.333496    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:15:40.433524 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:40.433492    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:15:40.535333 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:40.535295    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:15:40.828453 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:40.828364    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:15:40.929487 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:40.929451    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:15:44.702225 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:44.702187    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:15:44.864480 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:44.864440    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:15:47.665337 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:47.665307    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:15:47.763121 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:47.763090    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:15:48.063738 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:48.063706    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:15:48.168089 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:48.168050    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:15:52.152057 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:52.152021    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:15:52.250054 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:52.250022    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:15:58.629117 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:58.629085    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-87f9g"]
Jul 22 16:15:58.631267 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:58.631251    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-87f9g"
Jul 22 16:15:58.636720 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:58.636686    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 22 16:15:58.636892 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:58.636757    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-vrg4s\""
Jul 22 16:15:58.637031 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:58.637011    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 22 16:15:58.654541 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:58.654514    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-87f9g"]
Jul 22 16:15:58.729934 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:58.729897    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxvs6\" (UniqueName: \"kubernetes.io/projected/c0a0cd66-4510-442e-8717-b9be93deafac-kube-api-access-cxvs6\") pod \"limitador-operator-controller-manager-859d7b78bd-87f9g\" (UID: \"c0a0cd66-4510-442e-8717-b9be93deafac\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-87f9g"
Jul 22 16:15:58.830737 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:58.830706    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxvs6\" (UniqueName: \"kubernetes.io/projected/c0a0cd66-4510-442e-8717-b9be93deafac-kube-api-access-cxvs6\") pod \"limitador-operator-controller-manager-859d7b78bd-87f9g\" (UID: \"c0a0cd66-4510-442e-8717-b9be93deafac\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-87f9g"
Jul 22 16:15:58.841208 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:58.841176    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxvs6\" (UniqueName: \"kubernetes.io/projected/c0a0cd66-4510-442e-8717-b9be93deafac-kube-api-access-cxvs6\") pod \"limitador-operator-controller-manager-859d7b78bd-87f9g\" (UID: \"c0a0cd66-4510-442e-8717-b9be93deafac\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-87f9g"
Jul 22 16:15:58.943200 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:58.943158    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-87f9g"
Jul 22 16:15:59.079232 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:59.079200    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-87f9g"]
Jul 22 16:15:59.085641 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:15:59.085612    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0a0cd66_4510_442e_8717_b9be93deafac.slice/crio-f5f0bdd686ac31887ae3feffe615babb4381a31055247c5fe562bb53f00453e9 WatchSource:0}: Error finding container f5f0bdd686ac31887ae3feffe615babb4381a31055247c5fe562bb53f00453e9: Status 404 returned error can't find the container with id f5f0bdd686ac31887ae3feffe615babb4381a31055247c5fe562bb53f00453e9
Jul 22 16:15:59.595038 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:15:59.595002    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-87f9g" event={"ID":"c0a0cd66-4510-442e-8717-b9be93deafac","Type":"ContainerStarted","Data":"f5f0bdd686ac31887ae3feffe615babb4381a31055247c5fe562bb53f00453e9"}
Jul 22 16:16:00.210089 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:00.210059    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:16:00.321825 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:00.321794    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:16:00.820399 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:00.820367    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:16:00.919468 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:00.919438    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:16:01.605513 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:01.605479    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-87f9g" event={"ID":"c0a0cd66-4510-442e-8717-b9be93deafac","Type":"ContainerStarted","Data":"e9a37f46548b22ca55d0700d14cb5cdbea3f68b276ad304608e09a05dd0a5d42"}
Jul 22 16:16:01.605952 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:01.605674    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-87f9g"
Jul 22 16:16:01.628814 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:01.628760    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-87f9g" podStartSLOduration=1.320197046 podStartE2EDuration="3.628740284s" totalImagesPullingTime="2.308543238s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:15:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:16:01.627721467 +0000 UTC m=+521.758191493" watchObservedRunningTime="2026-07-22 16:16:01.628740284 +0000 UTC m=+521.759210288"
Jul 22 16:16:04.162629 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:04.162570    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:16:04.808114 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:04.808081    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:16:06.401116 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:06.401077    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:16:06.497765 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:06.497735    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:16:10.952606 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:10.952573    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:16:11.054822 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:11.054788    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:16:12.612161 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:12.612129    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-87f9g"
Jul 22 16:16:20.450145 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:20.448772    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:16:38.143201 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:38.143170    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:16:38.737693 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:38.737660    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:16:42.912118 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:42.912084    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:16:43.111735 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:43.111697    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:16:43.211925 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:43.211891    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:16:43.824871 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:43.824814    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:16:43.893423 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:43.893388    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:16:43.998758 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:43.998725    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-6jgt8"]
Jul 22 16:16:44.000984 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:44.000968    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-6jgt8"
Jul 22 16:16:44.003709 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:44.003689    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-gn4s2\""
Jul 22 16:16:44.013916 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:44.013040    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-6jgt8"]
Jul 22 16:16:44.064925 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:44.064882    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jw6gk\" (UniqueName: \"kubernetes.io/projected/5f27f76b-9fcd-457a-abeb-30fdb72c40ca-kube-api-access-jw6gk\") pod \"authorino-75d556bd77-6jgt8\" (UID: \"5f27f76b-9fcd-457a-abeb-30fdb72c40ca\") " pod="kuadrant-system/authorino-75d556bd77-6jgt8"
Jul 22 16:16:44.165992 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:44.165897    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jw6gk\" (UniqueName: \"kubernetes.io/projected/5f27f76b-9fcd-457a-abeb-30fdb72c40ca-kube-api-access-jw6gk\") pod \"authorino-75d556bd77-6jgt8\" (UID: \"5f27f76b-9fcd-457a-abeb-30fdb72c40ca\") " pod="kuadrant-system/authorino-75d556bd77-6jgt8"
Jul 22 16:16:44.174793 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:44.174760    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw6gk\" (UniqueName: \"kubernetes.io/projected/5f27f76b-9fcd-457a-abeb-30fdb72c40ca-kube-api-access-jw6gk\") pod \"authorino-75d556bd77-6jgt8\" (UID: \"5f27f76b-9fcd-457a-abeb-30fdb72c40ca\") " pod="kuadrant-system/authorino-75d556bd77-6jgt8"
Jul 22 16:16:44.310029 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:44.309998    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-6jgt8"
Jul 22 16:16:44.436978 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:44.436934    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-6jgt8"]
Jul 22 16:16:44.442135 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:16:44.442102    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5f27f76b_9fcd_457a_abeb_30fdb72c40ca.slice/crio-ddabce3d5bfdefc1cc210d22c72fd23ed1c5170d0bb73cae525152a06a21842a WatchSource:0}: Error finding container ddabce3d5bfdefc1cc210d22c72fd23ed1c5170d0bb73cae525152a06a21842a: Status 404 returned error can't find the container with id ddabce3d5bfdefc1cc210d22c72fd23ed1c5170d0bb73cae525152a06a21842a
Jul 22 16:16:44.775393 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:44.775309    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-6jgt8" event={"ID":"5f27f76b-9fcd-457a-abeb-30fdb72c40ca","Type":"ContainerStarted","Data":"ddabce3d5bfdefc1cc210d22c72fd23ed1c5170d0bb73cae525152a06a21842a"}
Jul 22 16:16:47.706190 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:47.706119    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:16:47.805729 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:47.805697    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:16:48.796402 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:48.796367    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-6jgt8" event={"ID":"5f27f76b-9fcd-457a-abeb-30fdb72c40ca","Type":"ContainerStarted","Data":"8d1793931e7eef0fb9eadcbb673e9b7c9cf1656c23dd89a464904a1d7364bce7"}
Jul 22 16:16:48.814309 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:48.814262    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-6jgt8" podStartSLOduration=2.412857772 podStartE2EDuration="5.814245746s" totalImagesPullingTime="3.401387974s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:16:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:16:48.813420777 +0000 UTC m=+568.943890779" watchObservedRunningTime="2026-07-22 16:16:48.814245746 +0000 UTC m=+568.944715748"
Jul 22 16:16:59.104349 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:59.104311    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:16:59.200418 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:59.200385    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:16:59.688476 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:16:59.688441    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:17:12.394117 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:12.394085    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:17:12.521459 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:12.521427    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:17:12.618848 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:12.618817    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-m4z7p"]
Jul 22 16:17:12.621034 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:12.621013    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-m4z7p"
Jul 22 16:17:12.631685 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:12.631657    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-m4z7p"]
Jul 22 16:17:12.812314 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:12.812272    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vcrw\" (UniqueName: \"kubernetes.io/projected/29a9ac30-3e3a-4689-9643-45794a4161db-kube-api-access-7vcrw\") pod \"authorino-5d4b67c446-m4z7p\" (UID: \"29a9ac30-3e3a-4689-9643-45794a4161db\") " pod="kuadrant-system/authorino-5d4b67c446-m4z7p"
Jul 22 16:17:12.879412 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:12.879379    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-m4z7p"]
Jul 22 16:17:12.879632 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:17:12.879614    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-7vcrw], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-m4z7p" podUID="29a9ac30-3e3a-4689-9643-45794a4161db"
Jul 22 16:17:12.894364 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:12.894330    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-m4z7p"
Jul 22 16:17:12.899928 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:12.899908    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-m4z7p"
Jul 22 16:17:12.913560 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:12.913536    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vcrw\" (UniqueName: \"kubernetes.io/projected/29a9ac30-3e3a-4689-9643-45794a4161db-kube-api-access-7vcrw\") pod \"authorino-5d4b67c446-m4z7p\" (UID: \"29a9ac30-3e3a-4689-9643-45794a4161db\") " pod="kuadrant-system/authorino-5d4b67c446-m4z7p"
Jul 22 16:17:12.914820 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:12.914800    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-fd7d769f6-mzppm"]
Jul 22 16:17:12.916728 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:12.916715    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-fd7d769f6-mzppm"
Jul 22 16:17:12.927689 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:12.927668    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-fd7d769f6-mzppm"]
Jul 22 16:17:12.929291 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:12.929267    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vcrw\" (UniqueName: \"kubernetes.io/projected/29a9ac30-3e3a-4689-9643-45794a4161db-kube-api-access-7vcrw\") pod \"authorino-5d4b67c446-m4z7p\" (UID: \"29a9ac30-3e3a-4689-9643-45794a4161db\") " pod="kuadrant-system/authorino-5d4b67c446-m4z7p"
Jul 22 16:17:13.112008 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:13.111924    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-fd7d769f6-mzppm"]
Jul 22 16:17:13.112179 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:17:13.112160    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-x92rc], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-fd7d769f6-mzppm" podUID="1edc9d08-743f-4a3d-a548-9a3d8f903737"
Jul 22 16:17:13.115508 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:13.115486    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vcrw\" (UniqueName: \"kubernetes.io/projected/29a9ac30-3e3a-4689-9643-45794a4161db-kube-api-access-7vcrw\") pod \"29a9ac30-3e3a-4689-9643-45794a4161db\" (UID: \"29a9ac30-3e3a-4689-9643-45794a4161db\") "
Jul 22 16:17:13.115652 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:13.115638    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x92rc\" (UniqueName: \"kubernetes.io/projected/1edc9d08-743f-4a3d-a548-9a3d8f903737-kube-api-access-x92rc\") pod \"authorino-fd7d769f6-mzppm\" (UID: \"1edc9d08-743f-4a3d-a548-9a3d8f903737\") " pod="kuadrant-system/authorino-fd7d769f6-mzppm"
Jul 22 16:17:13.117557 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:13.117537    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29a9ac30-3e3a-4689-9643-45794a4161db-kube-api-access-7vcrw" (OuterVolumeSpecName: "kube-api-access-7vcrw") pod "29a9ac30-3e3a-4689-9643-45794a4161db" (UID: "29a9ac30-3e3a-4689-9643-45794a4161db"). InnerVolumeSpecName "kube-api-access-7vcrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:17:13.153602 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:13.153566    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58c9ffbbfb-bkkvc"]
Jul 22 16:17:13.155668 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:13.155651    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58c9ffbbfb-bkkvc"
Jul 22 16:17:13.158629 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:13.158607    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 22 16:17:13.165970 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:13.165946    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58c9ffbbfb-bkkvc"]
Jul 22 16:17:13.216316 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:13.216282    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x92rc\" (UniqueName: \"kubernetes.io/projected/1edc9d08-743f-4a3d-a548-9a3d8f903737-kube-api-access-x92rc\") pod \"authorino-fd7d769f6-mzppm\" (UID: \"1edc9d08-743f-4a3d-a548-9a3d8f903737\") " pod="kuadrant-system/authorino-fd7d769f6-mzppm"
Jul 22 16:17:13.216491 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:13.216358    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vcrw\" (UniqueName: \"kubernetes.io/projected/29a9ac30-3e3a-4689-9643-45794a4161db-kube-api-access-7vcrw\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:17:13.228195 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:13.228170    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x92rc\" (UniqueName: \"kubernetes.io/projected/1edc9d08-743f-4a3d-a548-9a3d8f903737-kube-api-access-x92rc\") pod \"authorino-fd7d769f6-mzppm\" (UID: \"1edc9d08-743f-4a3d-a548-9a3d8f903737\") " pod="kuadrant-system/authorino-fd7d769f6-mzppm"
Jul 22 16:17:13.317504 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:13.317468    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqqpt\" (UniqueName: \"kubernetes.io/projected/e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d-kube-api-access-gqqpt\") pod \"authorino-58c9ffbbfb-bkkvc\" (UID: \"e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d\") " pod="kuadrant-system/authorino-58c9ffbbfb-bkkvc"
Jul 22 16:17:13.317504 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:13.317517    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d-tls-cert\") pod \"authorino-58c9ffbbfb-bkkvc\" (UID: \"e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d\") " pod="kuadrant-system/authorino-58c9ffbbfb-bkkvc"
Jul 22 16:17:13.418029 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:13.417940    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqqpt\" (UniqueName: \"kubernetes.io/projected/e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d-kube-api-access-gqqpt\") pod \"authorino-58c9ffbbfb-bkkvc\" (UID: \"e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d\") " pod="kuadrant-system/authorino-58c9ffbbfb-bkkvc"
Jul 22 16:17:13.418029 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:13.417984    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d-tls-cert\") pod \"authorino-58c9ffbbfb-bkkvc\" (UID: \"e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d\") " pod="kuadrant-system/authorino-58c9ffbbfb-bkkvc"
Jul 22 16:17:13.420455 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:13.420436    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d-tls-cert\") pod \"authorino-58c9ffbbfb-bkkvc\" (UID: \"e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d\") " pod="kuadrant-system/authorino-58c9ffbbfb-bkkvc"
Jul 22 16:17:13.427881 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:13.427842    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqqpt\" (UniqueName: \"kubernetes.io/projected/e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d-kube-api-access-gqqpt\") pod \"authorino-58c9ffbbfb-bkkvc\" (UID: \"e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d\") " pod="kuadrant-system/authorino-58c9ffbbfb-bkkvc"
Jul 22 16:17:13.464658 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:13.464625    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58c9ffbbfb-bkkvc"
Jul 22 16:17:13.598441 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:13.598410    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58c9ffbbfb-bkkvc"]
Jul 22 16:17:13.600780 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:17:13.600753    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode5627ffc_d2a3_494d_a0b2_4b5fa6ae6f0d.slice/crio-5497b8e17430a1d1f3ee0d38374ff0e0d4d16e7443fffda6dce5aeeae451bfb8 WatchSource:0}: Error finding container 5497b8e17430a1d1f3ee0d38374ff0e0d4d16e7443fffda6dce5aeeae451bfb8: Status 404 returned error can't find the container with id 5497b8e17430a1d1f3ee0d38374ff0e0d4d16e7443fffda6dce5aeeae451bfb8
Jul 22 16:17:13.899638 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:13.899605    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58c9ffbbfb-bkkvc" event={"ID":"e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d","Type":"ContainerStarted","Data":"5497b8e17430a1d1f3ee0d38374ff0e0d4d16e7443fffda6dce5aeeae451bfb8"}
Jul 22 16:17:13.899638 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:13.899632    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-fd7d769f6-mzppm"
Jul 22 16:17:13.899927 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:13.899648    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-m4z7p"
Jul 22 16:17:13.907062 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:13.907041    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-fd7d769f6-mzppm"
Jul 22 16:17:13.921898 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:13.921871    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x92rc\" (UniqueName: \"kubernetes.io/projected/1edc9d08-743f-4a3d-a548-9a3d8f903737-kube-api-access-x92rc\") pod \"1edc9d08-743f-4a3d-a548-9a3d8f903737\" (UID: \"1edc9d08-743f-4a3d-a548-9a3d8f903737\") "
Jul 22 16:17:13.933145 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:13.933116    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1edc9d08-743f-4a3d-a548-9a3d8f903737-kube-api-access-x92rc" (OuterVolumeSpecName: "kube-api-access-x92rc") pod "1edc9d08-743f-4a3d-a548-9a3d8f903737" (UID: "1edc9d08-743f-4a3d-a548-9a3d8f903737"). InnerVolumeSpecName "kube-api-access-x92rc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:17:13.943626 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:13.943601    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-m4z7p"]
Jul 22 16:17:13.947006 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:13.946976    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-m4z7p"]
Jul 22 16:17:14.022391 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:14.022366    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x92rc\" (UniqueName: \"kubernetes.io/projected/1edc9d08-743f-4a3d-a548-9a3d8f903737-kube-api-access-x92rc\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:17:14.448242 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:14.448208    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29a9ac30-3e3a-4689-9643-45794a4161db" path="/var/lib/kubelet/pods/29a9ac30-3e3a-4689-9643-45794a4161db/volumes"
Jul 22 16:17:14.905229 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:14.905144    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-fd7d769f6-mzppm"
Jul 22 16:17:14.905229 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:14.905175    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58c9ffbbfb-bkkvc" event={"ID":"e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d","Type":"ContainerStarted","Data":"18576f951ce4632d38facd4b2d6777bd3392f85567e8789256dc9259ab3000e5"}
Jul 22 16:17:14.969082 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:14.969028    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58c9ffbbfb-bkkvc" podStartSLOduration=1.614847439 podStartE2EDuration="1.969011619s" totalImagesPullingTime="354.16418ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:17:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:17:14.944833252 +0000 UTC m=+595.075303254" watchObservedRunningTime="2026-07-22 16:17:14.969011619 +0000 UTC m=+595.099481621"
Jul 22 16:17:14.980844 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:14.980812    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-fd7d769f6-mzppm"]
Jul 22 16:17:14.988197 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:14.988150    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-fd7d769f6-mzppm"]
Jul 22 16:17:14.989361 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:14.989341    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-6jgt8"]
Jul 22 16:17:14.989538 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:14.989514    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-6jgt8" podUID="5f27f76b-9fcd-457a-abeb-30fdb72c40ca" containerName="authorino" containerID="cri-o://8d1793931e7eef0fb9eadcbb673e9b7c9cf1656c23dd89a464904a1d7364bce7" gracePeriod=30
Jul 22 16:17:15.233429 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:15.233407    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-6jgt8"
Jul 22 16:17:15.332566 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:15.332534    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jw6gk\" (UniqueName: \"kubernetes.io/projected/5f27f76b-9fcd-457a-abeb-30fdb72c40ca-kube-api-access-jw6gk\") pod \"5f27f76b-9fcd-457a-abeb-30fdb72c40ca\" (UID: \"5f27f76b-9fcd-457a-abeb-30fdb72c40ca\") "
Jul 22 16:17:15.334705 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:15.334678    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f27f76b-9fcd-457a-abeb-30fdb72c40ca-kube-api-access-jw6gk" (OuterVolumeSpecName: "kube-api-access-jw6gk") pod "5f27f76b-9fcd-457a-abeb-30fdb72c40ca" (UID: "5f27f76b-9fcd-457a-abeb-30fdb72c40ca"). InnerVolumeSpecName "kube-api-access-jw6gk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:17:15.433910 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:15.433875    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jw6gk\" (UniqueName: \"kubernetes.io/projected/5f27f76b-9fcd-457a-abeb-30fdb72c40ca-kube-api-access-jw6gk\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:17:15.910629 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:15.910595    2563 generic.go:356] "Generic (PLEG): container finished" podID="5f27f76b-9fcd-457a-abeb-30fdb72c40ca" containerID="8d1793931e7eef0fb9eadcbb673e9b7c9cf1656c23dd89a464904a1d7364bce7" exitCode=0
Jul 22 16:17:15.910629 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:15.910633    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-6jgt8"
Jul 22 16:17:15.911177 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:15.910682    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-6jgt8" event={"ID":"5f27f76b-9fcd-457a-abeb-30fdb72c40ca","Type":"ContainerDied","Data":"8d1793931e7eef0fb9eadcbb673e9b7c9cf1656c23dd89a464904a1d7364bce7"}
Jul 22 16:17:15.911177 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:15.910717    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-6jgt8" event={"ID":"5f27f76b-9fcd-457a-abeb-30fdb72c40ca","Type":"ContainerDied","Data":"ddabce3d5bfdefc1cc210d22c72fd23ed1c5170d0bb73cae525152a06a21842a"}
Jul 22 16:17:15.911177 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:15.910733    2563 scope.go:117] "RemoveContainer" containerID="8d1793931e7eef0fb9eadcbb673e9b7c9cf1656c23dd89a464904a1d7364bce7"
Jul 22 16:17:15.919145 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:15.919130    2563 scope.go:117] "RemoveContainer" containerID="8d1793931e7eef0fb9eadcbb673e9b7c9cf1656c23dd89a464904a1d7364bce7"
Jul 22 16:17:15.919383 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:17:15.919363    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d1793931e7eef0fb9eadcbb673e9b7c9cf1656c23dd89a464904a1d7364bce7\": container with ID starting with 8d1793931e7eef0fb9eadcbb673e9b7c9cf1656c23dd89a464904a1d7364bce7 not found: ID does not exist" containerID="8d1793931e7eef0fb9eadcbb673e9b7c9cf1656c23dd89a464904a1d7364bce7"
Jul 22 16:17:15.919427 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:15.919391    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d1793931e7eef0fb9eadcbb673e9b7c9cf1656c23dd89a464904a1d7364bce7"} err="failed to get container status \"8d1793931e7eef0fb9eadcbb673e9b7c9cf1656c23dd89a464904a1d7364bce7\": rpc error: code = NotFound desc = could not find container \"8d1793931e7eef0fb9eadcbb673e9b7c9cf1656c23dd89a464904a1d7364bce7\": container with ID starting with 8d1793931e7eef0fb9eadcbb673e9b7c9cf1656c23dd89a464904a1d7364bce7 not found: ID does not exist"
Jul 22 16:17:15.936739 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:15.936706    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-6jgt8"]
Jul 22 16:17:15.940887 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:15.940868    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-6jgt8"]
Jul 22 16:17:16.447574 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:16.447541    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1edc9d08-743f-4a3d-a548-9a3d8f903737" path="/var/lib/kubelet/pods/1edc9d08-743f-4a3d-a548-9a3d8f903737/volumes"
Jul 22 16:17:16.447790 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:16.447777    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f27f76b-9fcd-457a-abeb-30fdb72c40ca" path="/var/lib/kubelet/pods/5f27f76b-9fcd-457a-abeb-30fdb72c40ca/volumes"
Jul 22 16:17:20.026507 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:20.026477    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:17:20.126891 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:20.126835    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:17:20.200896 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:20.200847    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:17:23.253632 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:23.253594    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:17:23.349220 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:23.349191    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:17:25.858352 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:25.858319    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:17:27.483395 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:27.483347    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:17:54.957966 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:54.957879    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:17:55.075264 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:55.075220    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:17:55.515869 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:55.515815    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:17:55.732690 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:55.732652    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:17:56.146515 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:56.146482    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:17:56.259070 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:17:56.259033    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:18:06.372911 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:06.372879    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:18:06.467865 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:06.467821    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:18:22.711551 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:22.711516    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:18:30.778327 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:30.778290    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:18:56.553958 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:56.553922    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-9d4rx"]
Jul 22 16:18:56.554397 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:56.554218    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f27f76b-9fcd-457a-abeb-30fdb72c40ca" containerName="authorino"
Jul 22 16:18:56.554397 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:56.554229    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f27f76b-9fcd-457a-abeb-30fdb72c40ca" containerName="authorino"
Jul 22 16:18:56.554397 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:56.554313    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f27f76b-9fcd-457a-abeb-30fdb72c40ca" containerName="authorino"
Jul 22 16:18:56.556149 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:56.556134    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9d4rx"
Jul 22 16:18:56.559290 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:56.559270    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-t84pc\""
Jul 22 16:18:56.560688 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:56.560674    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 22 16:18:56.560783 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:56.560748    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 22 16:18:56.560783 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:56.560764    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 22 16:18:56.571659 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:56.571632    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-9d4rx"]
Jul 22 16:18:56.644750 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:56.644716    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:18:56.658805 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:56.658783    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d76b9940-884e-43c6-b562-7829ace2b94c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9d4rx\" (UID: \"d76b9940-884e-43c6-b562-7829ace2b94c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9d4rx"
Jul 22 16:18:56.658952 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:56.658814    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d76b9940-884e-43c6-b562-7829ace2b94c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9d4rx\" (UID: \"d76b9940-884e-43c6-b562-7829ace2b94c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9d4rx"
Jul 22 16:18:56.658952 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:56.658840    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d76b9940-884e-43c6-b562-7829ace2b94c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9d4rx\" (UID: \"d76b9940-884e-43c6-b562-7829ace2b94c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9d4rx"
Jul 22 16:18:56.658952 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:56.658917    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d76b9940-884e-43c6-b562-7829ace2b94c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9d4rx\" (UID: \"d76b9940-884e-43c6-b562-7829ace2b94c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9d4rx"
Jul 22 16:18:56.659052 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:56.658969    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d76b9940-884e-43c6-b562-7829ace2b94c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9d4rx\" (UID: \"d76b9940-884e-43c6-b562-7829ace2b94c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9d4rx"
Jul 22 16:18:56.659052 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:56.659022    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbjds\" (UniqueName: \"kubernetes.io/projected/d76b9940-884e-43c6-b562-7829ace2b94c-kube-api-access-cbjds\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9d4rx\" (UID: \"d76b9940-884e-43c6-b562-7829ace2b94c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9d4rx"
Jul 22 16:18:56.760240 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:56.760195    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d76b9940-884e-43c6-b562-7829ace2b94c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9d4rx\" (UID: \"d76b9940-884e-43c6-b562-7829ace2b94c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9d4rx"
Jul 22 16:18:56.760240 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:56.760241    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d76b9940-884e-43c6-b562-7829ace2b94c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9d4rx\" (UID: \"d76b9940-884e-43c6-b562-7829ace2b94c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9d4rx"
Jul 22 16:18:56.760437 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:56.760366    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d76b9940-884e-43c6-b562-7829ace2b94c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9d4rx\" (UID: \"d76b9940-884e-43c6-b562-7829ace2b94c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9d4rx"
Jul 22 16:18:56.760437 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:56.760419    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d76b9940-884e-43c6-b562-7829ace2b94c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9d4rx\" (UID: \"d76b9940-884e-43c6-b562-7829ace2b94c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9d4rx"
Jul 22 16:18:56.760549 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:56.760487    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d76b9940-884e-43c6-b562-7829ace2b94c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9d4rx\" (UID: \"d76b9940-884e-43c6-b562-7829ace2b94c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9d4rx"
Jul 22 16:18:56.760603 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:56.760546    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbjds\" (UniqueName: \"kubernetes.io/projected/d76b9940-884e-43c6-b562-7829ace2b94c-kube-api-access-cbjds\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9d4rx\" (UID: \"d76b9940-884e-43c6-b562-7829ace2b94c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9d4rx"
Jul 22 16:18:56.760677 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:56.760651    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d76b9940-884e-43c6-b562-7829ace2b94c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9d4rx\" (UID: \"d76b9940-884e-43c6-b562-7829ace2b94c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9d4rx"
Jul 22 16:18:56.760877 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:56.760830    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d76b9940-884e-43c6-b562-7829ace2b94c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9d4rx\" (UID: \"d76b9940-884e-43c6-b562-7829ace2b94c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9d4rx"
Jul 22 16:18:56.761000 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:56.760916    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d76b9940-884e-43c6-b562-7829ace2b94c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9d4rx\" (UID: \"d76b9940-884e-43c6-b562-7829ace2b94c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9d4rx"
Jul 22 16:18:56.762551 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:56.762516    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d76b9940-884e-43c6-b562-7829ace2b94c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9d4rx\" (UID: \"d76b9940-884e-43c6-b562-7829ace2b94c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9d4rx"
Jul 22 16:18:56.762837 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:56.762820    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d76b9940-884e-43c6-b562-7829ace2b94c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9d4rx\" (UID: \"d76b9940-884e-43c6-b562-7829ace2b94c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9d4rx"
Jul 22 16:18:56.768891 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:56.768864    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbjds\" (UniqueName: \"kubernetes.io/projected/d76b9940-884e-43c6-b562-7829ace2b94c-kube-api-access-cbjds\") pod \"e2e-trlp-test-simulated-kserve-598466548d-9d4rx\" (UID: \"d76b9940-884e-43c6-b562-7829ace2b94c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9d4rx"
Jul 22 16:18:56.866691 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:56.866592    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9d4rx"
Jul 22 16:18:57.008559 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:57.008345    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-9d4rx"]
Jul 22 16:18:57.010900 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:18:57.010869    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd76b9940_884e_43c6_b562_7829ace2b94c.slice/crio-703929286eeb41c24ac118fd234578ad2052f8ea63638c90fae55ccb8f7e82f8 WatchSource:0}: Error finding container 703929286eeb41c24ac118fd234578ad2052f8ea63638c90fae55ccb8f7e82f8: Status 404 returned error can't find the container with id 703929286eeb41c24ac118fd234578ad2052f8ea63638c90fae55ccb8f7e82f8
Jul 22 16:18:57.012290 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:57.012266    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 16:18:57.329916 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:57.329844    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9d4rx" event={"ID":"d76b9940-884e-43c6-b562-7829ace2b94c","Type":"ContainerStarted","Data":"703929286eeb41c24ac118fd234578ad2052f8ea63638c90fae55ccb8f7e82f8"}
Jul 22 16:18:58.337512 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:58.337472    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9d4rx" event={"ID":"d76b9940-884e-43c6-b562-7829ace2b94c","Type":"ContainerStarted","Data":"8db7786e2521e12dd67b9ca61825ad430907178e5b6fc26409fb28f0f262ff17"}
Jul 22 16:18:59.804145 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:59.804110    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:18:59.905058 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:18:59.905024    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:19:01.033235 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:01.033172    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9d4rx" podStartSLOduration=3.821319851 podStartE2EDuration="5.033148989s" totalImagesPullingTime="1.211829138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:18:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:18:58.357396509 +0000 UTC m=+698.487866544" watchObservedRunningTime="2026-07-22 16:19:01.033148989 +0000 UTC m=+701.163618993"
Jul 22 16:19:01.035274 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:01.035247    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5"]
Jul 22 16:19:01.038016 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:01.037998    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5"
Jul 22 16:19:01.041799 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:01.041774    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 22 16:19:01.051014 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:01.050989    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5"]
Jul 22 16:19:01.082600 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:01.082565    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:19:01.099038 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:01.099008    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66hvd\" (UniqueName: \"kubernetes.io/projected/f690408c-96a3-4268-9596-4a0b12d05ac1-kube-api-access-66hvd\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5\" (UID: \"f690408c-96a3-4268-9596-4a0b12d05ac1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5"
Jul 22 16:19:01.099132 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:01.099045    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f690408c-96a3-4268-9596-4a0b12d05ac1-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5\" (UID: \"f690408c-96a3-4268-9596-4a0b12d05ac1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5"
Jul 22 16:19:01.099132 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:01.099074    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f690408c-96a3-4268-9596-4a0b12d05ac1-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5\" (UID: \"f690408c-96a3-4268-9596-4a0b12d05ac1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5"
Jul 22 16:19:01.099132 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:01.099107    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f690408c-96a3-4268-9596-4a0b12d05ac1-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5\" (UID: \"f690408c-96a3-4268-9596-4a0b12d05ac1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5"
Jul 22 16:19:01.099247 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:01.099147    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f690408c-96a3-4268-9596-4a0b12d05ac1-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5\" (UID: \"f690408c-96a3-4268-9596-4a0b12d05ac1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5"
Jul 22 16:19:01.099247 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:01.099224    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f690408c-96a3-4268-9596-4a0b12d05ac1-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5\" (UID: \"f690408c-96a3-4268-9596-4a0b12d05ac1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5"
Jul 22 16:19:01.200167 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:01.200116    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f690408c-96a3-4268-9596-4a0b12d05ac1-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5\" (UID: \"f690408c-96a3-4268-9596-4a0b12d05ac1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5"
Jul 22 16:19:01.200167 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:01.200180    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f690408c-96a3-4268-9596-4a0b12d05ac1-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5\" (UID: \"f690408c-96a3-4268-9596-4a0b12d05ac1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5"
Jul 22 16:19:01.200435 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:01.200247    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f690408c-96a3-4268-9596-4a0b12d05ac1-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5\" (UID: \"f690408c-96a3-4268-9596-4a0b12d05ac1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5"
Jul 22 16:19:01.200435 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:01.200298    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f690408c-96a3-4268-9596-4a0b12d05ac1-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5\" (UID: \"f690408c-96a3-4268-9596-4a0b12d05ac1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5"
Jul 22 16:19:01.200435 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:01.200350    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66hvd\" (UniqueName: \"kubernetes.io/projected/f690408c-96a3-4268-9596-4a0b12d05ac1-kube-api-access-66hvd\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5\" (UID: \"f690408c-96a3-4268-9596-4a0b12d05ac1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5"
Jul 22 16:19:01.200435 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:01.200376    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f690408c-96a3-4268-9596-4a0b12d05ac1-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5\" (UID: \"f690408c-96a3-4268-9596-4a0b12d05ac1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5"
Jul 22 16:19:01.200690 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:01.200659    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f690408c-96a3-4268-9596-4a0b12d05ac1-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5\" (UID: \"f690408c-96a3-4268-9596-4a0b12d05ac1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5"
Jul 22 16:19:01.200806 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:01.200691    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f690408c-96a3-4268-9596-4a0b12d05ac1-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5\" (UID: \"f690408c-96a3-4268-9596-4a0b12d05ac1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5"
Jul 22 16:19:01.200806 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:01.200724    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f690408c-96a3-4268-9596-4a0b12d05ac1-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5\" (UID: \"f690408c-96a3-4268-9596-4a0b12d05ac1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5"
Jul 22 16:19:01.202413 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:01.202394    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f690408c-96a3-4268-9596-4a0b12d05ac1-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5\" (UID: \"f690408c-96a3-4268-9596-4a0b12d05ac1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5"
Jul 22 16:19:01.202757 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:01.202741    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f690408c-96a3-4268-9596-4a0b12d05ac1-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5\" (UID: \"f690408c-96a3-4268-9596-4a0b12d05ac1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5"
Jul 22 16:19:01.208698 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:01.208676    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66hvd\" (UniqueName: \"kubernetes.io/projected/f690408c-96a3-4268-9596-4a0b12d05ac1-kube-api-access-66hvd\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5\" (UID: \"f690408c-96a3-4268-9596-4a0b12d05ac1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5"
Jul 22 16:19:01.347737 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:01.347656    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5"
Jul 22 16:19:01.481409 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:01.481387    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5"]
Jul 22 16:19:01.484106 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:19:01.484081    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf690408c_96a3_4268_9596_4a0b12d05ac1.slice/crio-5d465e8dee909c28ead653fd54e99a2a7ae5fc89770a5b43c429ef10b4f417f8 WatchSource:0}: Error finding container 5d465e8dee909c28ead653fd54e99a2a7ae5fc89770a5b43c429ef10b4f417f8: Status 404 returned error can't find the container with id 5d465e8dee909c28ead653fd54e99a2a7ae5fc89770a5b43c429ef10b4f417f8
Jul 22 16:19:02.357357 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:02.357262    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5" event={"ID":"f690408c-96a3-4268-9596-4a0b12d05ac1","Type":"ContainerStarted","Data":"a6398c1d264be8824ac91661f8e8f9717c4ebe9062e8dab50c2bcdaaee7c65d6"}
Jul 22 16:19:02.357357 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:02.357312    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5" event={"ID":"f690408c-96a3-4268-9596-4a0b12d05ac1","Type":"ContainerStarted","Data":"5d465e8dee909c28ead653fd54e99a2a7ae5fc89770a5b43c429ef10b4f417f8"}
Jul 22 16:19:02.387211 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:02.387155    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5" podStartSLOduration=0.890442586 podStartE2EDuration="1.387139003s" totalImagesPullingTime="496.696417ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:19:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:19:02.386770707 +0000 UTC m=+702.517240732" watchObservedRunningTime="2026-07-22 16:19:02.387139003 +0000 UTC m=+702.517609072"
Jul 22 16:19:06.867109 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:06.867072    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9d4rx"
Jul 22 16:19:06.867561 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:06.867384    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9d4rx"
Jul 22 16:19:06.879823 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:06.879797    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9d4rx"
Jul 22 16:19:07.394194 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:07.394170    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-9d4rx"
Jul 22 16:19:10.566623 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:10.566590    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:19:11.348053 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:11.348008    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5"
Jul 22 16:19:11.348226 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:11.348082    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5"
Jul 22 16:19:11.360680 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:11.360652    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5"
Jul 22 16:19:11.409319 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:11.409242    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5"
Jul 22 16:19:16.952799 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:19:16.952765    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:20:16.442269 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:20:16.442228    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:20:16.606086 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:20:16.606054    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:20:17.272156 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:20:17.272118    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58c9ffbbfb-bkkvc"]
Jul 22 16:20:17.272323 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:20:17.272306    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58c9ffbbfb-bkkvc" podUID="e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d" containerName="authorino" containerID="cri-o://18576f951ce4632d38facd4b2d6777bd3392f85567e8789256dc9259ab3000e5" gracePeriod=30
Jul 22 16:20:17.507358 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:20:17.507333    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58c9ffbbfb-bkkvc"
Jul 22 16:20:17.561328 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:20:17.561237    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gqqpt\" (UniqueName: \"kubernetes.io/projected/e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d-kube-api-access-gqqpt\") pod \"e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d\" (UID: \"e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d\") "
Jul 22 16:20:17.561328 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:20:17.561282    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d-tls-cert\") pod \"e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d\" (UID: \"e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d\") "
Jul 22 16:20:17.563438 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:20:17.563402    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d-kube-api-access-gqqpt" (OuterVolumeSpecName: "kube-api-access-gqqpt") pod "e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d" (UID: "e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d"). InnerVolumeSpecName "kube-api-access-gqqpt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:20:17.570745 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:20:17.570712    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d" (UID: "e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:20:17.662955 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:20:17.662912    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gqqpt\" (UniqueName: \"kubernetes.io/projected/e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d-kube-api-access-gqqpt\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:20:17.662955 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:20:17.662942    2563 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d-tls-cert\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:20:17.671260 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:20:17.671225    2563 generic.go:356] "Generic (PLEG): container finished" podID="e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d" containerID="18576f951ce4632d38facd4b2d6777bd3392f85567e8789256dc9259ab3000e5" exitCode=0
Jul 22 16:20:17.671424 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:20:17.671373    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58c9ffbbfb-bkkvc" event={"ID":"e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d","Type":"ContainerDied","Data":"18576f951ce4632d38facd4b2d6777bd3392f85567e8789256dc9259ab3000e5"}
Jul 22 16:20:17.671424 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:20:17.671380    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58c9ffbbfb-bkkvc"
Jul 22 16:20:17.671543 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:20:17.671433    2563 scope.go:117] "RemoveContainer" containerID="18576f951ce4632d38facd4b2d6777bd3392f85567e8789256dc9259ab3000e5"
Jul 22 16:20:17.671543 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:20:17.671416    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58c9ffbbfb-bkkvc" event={"ID":"e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d","Type":"ContainerDied","Data":"5497b8e17430a1d1f3ee0d38374ff0e0d4d16e7443fffda6dce5aeeae451bfb8"}
Jul 22 16:20:17.680554 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:20:17.680530    2563 scope.go:117] "RemoveContainer" containerID="18576f951ce4632d38facd4b2d6777bd3392f85567e8789256dc9259ab3000e5"
Jul 22 16:20:17.680892 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:20:17.680844    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18576f951ce4632d38facd4b2d6777bd3392f85567e8789256dc9259ab3000e5\": container with ID starting with 18576f951ce4632d38facd4b2d6777bd3392f85567e8789256dc9259ab3000e5 not found: ID does not exist" containerID="18576f951ce4632d38facd4b2d6777bd3392f85567e8789256dc9259ab3000e5"
Jul 22 16:20:17.680985 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:20:17.680905    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18576f951ce4632d38facd4b2d6777bd3392f85567e8789256dc9259ab3000e5"} err="failed to get container status \"18576f951ce4632d38facd4b2d6777bd3392f85567e8789256dc9259ab3000e5\": rpc error: code = NotFound desc = could not find container \"18576f951ce4632d38facd4b2d6777bd3392f85567e8789256dc9259ab3000e5\": container with ID starting with 18576f951ce4632d38facd4b2d6777bd3392f85567e8789256dc9259ab3000e5 not found: ID does not exist"
Jul 22 16:20:17.771803 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:20:17.771770    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58c9ffbbfb-bkkvc"]
Jul 22 16:20:17.789242 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:20:17.789214    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58c9ffbbfb-bkkvc"]
Jul 22 16:20:17.793936 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:20:17.793912    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:20:17.935292 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:20:17.935256    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:20:18.447827 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:20:18.447790    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d" path="/var/lib/kubelet/pods/e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d/volumes"
Jul 22 16:20:23.129865 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:20:23.129821    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:20:33.742994 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:20:33.742964    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:22:05.603394 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:05.603363    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6848f74968-rczvt"]
Jul 22 16:22:05.603815 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:05.603695    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d" containerName="authorino"
Jul 22 16:22:05.603815 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:05.603708    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d" containerName="authorino"
Jul 22 16:22:05.603815 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:05.603761    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5627ffc-d2a3-494d-a0b2-4b5fa6ae6f0d" containerName="authorino"
Jul 22 16:22:05.606587 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:05.606571    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6848f74968-rczvt"
Jul 22 16:22:05.611041 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:05.611018    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 22 16:22:05.611170 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:05.611024    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6xqzg\""
Jul 22 16:22:05.611170 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:05.611056    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 22 16:22:05.620024 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:05.619994    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6848f74968-rczvt"]
Jul 22 16:22:05.667142 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:05.667101    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f5073fb6-bb74-4280-a151-e0341048bd1d-webhook-cert\") pod \"maas-controller-6848f74968-rczvt\" (UID: \"f5073fb6-bb74-4280-a151-e0341048bd1d\") " pod="opendatahub/maas-controller-6848f74968-rczvt"
Jul 22 16:22:05.667297 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:05.667172    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzwr2\" (UniqueName: \"kubernetes.io/projected/f5073fb6-bb74-4280-a151-e0341048bd1d-kube-api-access-gzwr2\") pod \"maas-controller-6848f74968-rczvt\" (UID: \"f5073fb6-bb74-4280-a151-e0341048bd1d\") " pod="opendatahub/maas-controller-6848f74968-rczvt"
Jul 22 16:22:05.768684 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:05.768641    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzwr2\" (UniqueName: \"kubernetes.io/projected/f5073fb6-bb74-4280-a151-e0341048bd1d-kube-api-access-gzwr2\") pod \"maas-controller-6848f74968-rczvt\" (UID: \"f5073fb6-bb74-4280-a151-e0341048bd1d\") " pod="opendatahub/maas-controller-6848f74968-rczvt"
Jul 22 16:22:05.768894 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:05.768746    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f5073fb6-bb74-4280-a151-e0341048bd1d-webhook-cert\") pod \"maas-controller-6848f74968-rczvt\" (UID: \"f5073fb6-bb74-4280-a151-e0341048bd1d\") " pod="opendatahub/maas-controller-6848f74968-rczvt"
Jul 22 16:22:05.771184 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:05.771160    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f5073fb6-bb74-4280-a151-e0341048bd1d-webhook-cert\") pod \"maas-controller-6848f74968-rczvt\" (UID: \"f5073fb6-bb74-4280-a151-e0341048bd1d\") " pod="opendatahub/maas-controller-6848f74968-rczvt"
Jul 22 16:22:05.778870 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:05.778823    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzwr2\" (UniqueName: \"kubernetes.io/projected/f5073fb6-bb74-4280-a151-e0341048bd1d-kube-api-access-gzwr2\") pod \"maas-controller-6848f74968-rczvt\" (UID: \"f5073fb6-bb74-4280-a151-e0341048bd1d\") " pod="opendatahub/maas-controller-6848f74968-rczvt"
Jul 22 16:22:05.937392 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:05.937360    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6848f74968-rczvt"
Jul 22 16:22:06.076153 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:06.076059    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6848f74968-rczvt"]
Jul 22 16:22:06.078801 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:22:06.078772    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5073fb6_bb74_4280_a151_e0341048bd1d.slice/crio-37e7b89128b9da822ade47151682186b08b3b2c8be658b338179cac0d9159f11 WatchSource:0}: Error finding container 37e7b89128b9da822ade47151682186b08b3b2c8be658b338179cac0d9159f11: Status 404 returned error can't find the container with id 37e7b89128b9da822ade47151682186b08b3b2c8be658b338179cac0d9159f11
Jul 22 16:22:06.108847 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:06.108808    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6848f74968-rczvt" event={"ID":"f5073fb6-bb74-4280-a151-e0341048bd1d","Type":"ContainerStarted","Data":"37e7b89128b9da822ade47151682186b08b3b2c8be658b338179cac0d9159f11"}
Jul 22 16:22:09.124894 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:09.124831    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6848f74968-rczvt" event={"ID":"f5073fb6-bb74-4280-a151-e0341048bd1d","Type":"ContainerStarted","Data":"3f392d8a9ba8e276394707fbda2efcb4de818bffecba7e914ff7675ab05ea830"}
Jul 22 16:22:09.125323 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:09.125002    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6848f74968-rczvt"
Jul 22 16:22:14.927959 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:14.927874    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:22:15.015807 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:15.015769    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:22:20.132147 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:20.132113    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6848f74968-rczvt"
Jul 22 16:22:20.154156 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:20.154096    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6848f74968-rczvt" podStartSLOduration=12.804839491 podStartE2EDuration="15.154079143s" totalImagesPullingTime="2.349239652s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:22:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:22:09.158268176 +0000 UTC m=+889.288738179" watchObservedRunningTime="2026-07-22 16:22:20.154079143 +0000 UTC m=+900.284549186"
Jul 22 16:22:24.687041 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:24.687003    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:22:24.792124 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:24.792091    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6848f74968-rczvt"]
Jul 22 16:22:24.792370 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:24.792346    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6848f74968-rczvt" podUID="f5073fb6-bb74-4280-a151-e0341048bd1d" containerName="manager" containerID="cri-o://3f392d8a9ba8e276394707fbda2efcb4de818bffecba7e914ff7675ab05ea830" gracePeriod=30
Jul 22 16:22:25.036178 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:25.036156    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6848f74968-rczvt"
Jul 22 16:22:25.139192 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:25.139161    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gzwr2\" (UniqueName: \"kubernetes.io/projected/f5073fb6-bb74-4280-a151-e0341048bd1d-kube-api-access-gzwr2\") pod \"f5073fb6-bb74-4280-a151-e0341048bd1d\" (UID: \"f5073fb6-bb74-4280-a151-e0341048bd1d\") "
Jul 22 16:22:25.139361 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:25.139237    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f5073fb6-bb74-4280-a151-e0341048bd1d-webhook-cert\") pod \"f5073fb6-bb74-4280-a151-e0341048bd1d\" (UID: \"f5073fb6-bb74-4280-a151-e0341048bd1d\") "
Jul 22 16:22:25.141385 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:25.141352    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5073fb6-bb74-4280-a151-e0341048bd1d-kube-api-access-gzwr2" (OuterVolumeSpecName: "kube-api-access-gzwr2") pod "f5073fb6-bb74-4280-a151-e0341048bd1d" (UID: "f5073fb6-bb74-4280-a151-e0341048bd1d"). InnerVolumeSpecName "kube-api-access-gzwr2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:22:25.141385 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:25.141365    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5073fb6-bb74-4280-a151-e0341048bd1d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f5073fb6-bb74-4280-a151-e0341048bd1d" (UID: "f5073fb6-bb74-4280-a151-e0341048bd1d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:22:25.193983 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:25.193947    2563 generic.go:356] "Generic (PLEG): container finished" podID="f5073fb6-bb74-4280-a151-e0341048bd1d" containerID="3f392d8a9ba8e276394707fbda2efcb4de818bffecba7e914ff7675ab05ea830" exitCode=0
Jul 22 16:22:25.194135 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:25.193998    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6848f74968-rczvt"
Jul 22 16:22:25.194135 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:25.194033    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6848f74968-rczvt" event={"ID":"f5073fb6-bb74-4280-a151-e0341048bd1d","Type":"ContainerDied","Data":"3f392d8a9ba8e276394707fbda2efcb4de818bffecba7e914ff7675ab05ea830"}
Jul 22 16:22:25.194135 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:25.194088    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6848f74968-rczvt" event={"ID":"f5073fb6-bb74-4280-a151-e0341048bd1d","Type":"ContainerDied","Data":"37e7b89128b9da822ade47151682186b08b3b2c8be658b338179cac0d9159f11"}
Jul 22 16:22:25.194135 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:25.194112    2563 scope.go:117] "RemoveContainer" containerID="3f392d8a9ba8e276394707fbda2efcb4de818bffecba7e914ff7675ab05ea830"
Jul 22 16:22:25.202622 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:25.202606    2563 scope.go:117] "RemoveContainer" containerID="3f392d8a9ba8e276394707fbda2efcb4de818bffecba7e914ff7675ab05ea830"
Jul 22 16:22:25.202934 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:22:25.202913    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f392d8a9ba8e276394707fbda2efcb4de818bffecba7e914ff7675ab05ea830\": container with ID starting with 3f392d8a9ba8e276394707fbda2efcb4de818bffecba7e914ff7675ab05ea830 not found: ID does not exist" containerID="3f392d8a9ba8e276394707fbda2efcb4de818bffecba7e914ff7675ab05ea830"
Jul 22 16:22:25.202982 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:25.202944    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f392d8a9ba8e276394707fbda2efcb4de818bffecba7e914ff7675ab05ea830"} err="failed to get container status \"3f392d8a9ba8e276394707fbda2efcb4de818bffecba7e914ff7675ab05ea830\": rpc error: code = NotFound desc = could not find container \"3f392d8a9ba8e276394707fbda2efcb4de818bffecba7e914ff7675ab05ea830\": container with ID starting with 3f392d8a9ba8e276394707fbda2efcb4de818bffecba7e914ff7675ab05ea830 not found: ID does not exist"
Jul 22 16:22:25.233139 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:25.233072    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6848f74968-rczvt"]
Jul 22 16:22:25.239901 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:25.239882    2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f5073fb6-bb74-4280-a151-e0341048bd1d-webhook-cert\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:22:25.239995 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:25.239906    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzwr2\" (UniqueName: \"kubernetes.io/projected/f5073fb6-bb74-4280-a151-e0341048bd1d-kube-api-access-gzwr2\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:22:25.248806 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:25.248773    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6848f74968-rczvt"]
Jul 22 16:22:25.942101 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:25.942069    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:22:26.062976 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:26.062944    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6848f74968-64kv4"]
Jul 22 16:22:26.063261 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:26.063249    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5073fb6-bb74-4280-a151-e0341048bd1d" containerName="manager"
Jul 22 16:22:26.063313 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:26.063263    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5073fb6-bb74-4280-a151-e0341048bd1d" containerName="manager"
Jul 22 16:22:26.063347 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:26.063312    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5073fb6-bb74-4280-a151-e0341048bd1d" containerName="manager"
Jul 22 16:22:26.066065 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:26.066050    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6848f74968-64kv4"
Jul 22 16:22:26.069714 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:26.069692    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 22 16:22:26.069839 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:26.069755    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 22 16:22:26.071140 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:26.071105    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6xqzg\""
Jul 22 16:22:26.077651 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:26.077629    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6848f74968-64kv4"]
Jul 22 16:22:26.146049 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:26.146011    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9d14f63a-49aa-44a4-b1a2-eece5bb92883-webhook-cert\") pod \"maas-controller-6848f74968-64kv4\" (UID: \"9d14f63a-49aa-44a4-b1a2-eece5bb92883\") " pod="opendatahub/maas-controller-6848f74968-64kv4"
Jul 22 16:22:26.146049 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:26.146047    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp9pm\" (UniqueName: \"kubernetes.io/projected/9d14f63a-49aa-44a4-b1a2-eece5bb92883-kube-api-access-gp9pm\") pod \"maas-controller-6848f74968-64kv4\" (UID: \"9d14f63a-49aa-44a4-b1a2-eece5bb92883\") " pod="opendatahub/maas-controller-6848f74968-64kv4"
Jul 22 16:22:26.247385 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:26.247290    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9d14f63a-49aa-44a4-b1a2-eece5bb92883-webhook-cert\") pod \"maas-controller-6848f74968-64kv4\" (UID: \"9d14f63a-49aa-44a4-b1a2-eece5bb92883\") " pod="opendatahub/maas-controller-6848f74968-64kv4"
Jul 22 16:22:26.247385 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:26.247336    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp9pm\" (UniqueName: \"kubernetes.io/projected/9d14f63a-49aa-44a4-b1a2-eece5bb92883-kube-api-access-gp9pm\") pod \"maas-controller-6848f74968-64kv4\" (UID: \"9d14f63a-49aa-44a4-b1a2-eece5bb92883\") " pod="opendatahub/maas-controller-6848f74968-64kv4"
Jul 22 16:22:26.249887 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:26.249842    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9d14f63a-49aa-44a4-b1a2-eece5bb92883-webhook-cert\") pod \"maas-controller-6848f74968-64kv4\" (UID: \"9d14f63a-49aa-44a4-b1a2-eece5bb92883\") " pod="opendatahub/maas-controller-6848f74968-64kv4"
Jul 22 16:22:26.257688 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:26.257661    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp9pm\" (UniqueName: \"kubernetes.io/projected/9d14f63a-49aa-44a4-b1a2-eece5bb92883-kube-api-access-gp9pm\") pod \"maas-controller-6848f74968-64kv4\" (UID: \"9d14f63a-49aa-44a4-b1a2-eece5bb92883\") " pod="opendatahub/maas-controller-6848f74968-64kv4"
Jul 22 16:22:26.376211 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:26.376181    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6848f74968-64kv4"
Jul 22 16:22:26.447787 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:26.447759    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5073fb6-bb74-4280-a151-e0341048bd1d" path="/var/lib/kubelet/pods/f5073fb6-bb74-4280-a151-e0341048bd1d/volumes"
Jul 22 16:22:26.505783 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:26.505706    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6848f74968-64kv4"]
Jul 22 16:22:26.508933 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:22:26.508903    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d14f63a_49aa_44a4_b1a2_eece5bb92883.slice/crio-77967099cb32686fc703941bbf285d87f4164bf94c5c6f83c60e5875a01ba4a7 WatchSource:0}: Error finding container 77967099cb32686fc703941bbf285d87f4164bf94c5c6f83c60e5875a01ba4a7: Status 404 returned error can't find the container with id 77967099cb32686fc703941bbf285d87f4164bf94c5c6f83c60e5875a01ba4a7
Jul 22 16:22:27.204840 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:27.204747    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6848f74968-64kv4" event={"ID":"9d14f63a-49aa-44a4-b1a2-eece5bb92883","Type":"ContainerStarted","Data":"1358d6497ae55a4989d59073a54840a77dc1fe5f70b1213bf58638a5096edac9"}
Jul 22 16:22:27.204840 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:27.204805    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6848f74968-64kv4" event={"ID":"9d14f63a-49aa-44a4-b1a2-eece5bb92883","Type":"ContainerStarted","Data":"77967099cb32686fc703941bbf285d87f4164bf94c5c6f83c60e5875a01ba4a7"}
Jul 22 16:22:27.205254 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:27.204886    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6848f74968-64kv4"
Jul 22 16:22:27.225383 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:27.225334    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6848f74968-64kv4" podStartSLOduration=0.799548357 podStartE2EDuration="1.225318149s" totalImagesPullingTime="425.769792ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:22:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:22:27.225142206 +0000 UTC m=+907.355612208" watchObservedRunningTime="2026-07-22 16:22:27.225318149 +0000 UTC m=+907.355788150"
Jul 22 16:22:38.211772 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:22:38.211743    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6848f74968-64kv4"
Jul 22 16:23:11.679950 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:23:11.679914    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:23:13.256269 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:23:13.256231    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:23:17.766653 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:23:17.766612    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:23:17.887272 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:23:17.887239    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:23:17.987971 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:23:17.987933    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:23:18.192643 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:23:18.192607    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:23:23.853406 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:23:23.853367    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:23:23.955119 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:23:23.955086    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:24:09.905193 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:24:09.905098    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:26:39.463046 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:26:39.463009    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:26:39.545327 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:26:39.545293    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:29:15.127011 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:29:15.126974    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:30:00.146556 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:30:00.146477    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745630-v5cfc"]
Jul 22 16:30:00.149293 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:30:00.149272    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745630-v5cfc"
Jul 22 16:30:00.152323 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:30:00.152304    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-phfdj\""
Jul 22 16:30:00.157925 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:30:00.157897    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745630-v5cfc"]
Jul 22 16:30:00.159784 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:30:00.159768    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745630-v5cfc"
Jul 22 16:30:00.289765 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:30:00.289669    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745630-v5cfc"]
Jul 22 16:30:00.294086 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:30:00.294055    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9179d684_6a21_4aaa_9f9f_d8a3f55360c2.slice/crio-29e55163f6bac20678046bd1c04d721c9a080bead8d76c855230b696ceb33cb4 WatchSource:0}: Error finding container 29e55163f6bac20678046bd1c04d721c9a080bead8d76c855230b696ceb33cb4: Status 404 returned error can't find the container with id 29e55163f6bac20678046bd1c04d721c9a080bead8d76c855230b696ceb33cb4
Jul 22 16:30:00.296336 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:30:00.296319    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 16:30:01.111754 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:30:01.111716    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745630-v5cfc" event={"ID":"9179d684-6a21-4aaa-9f9f-d8a3f55360c2","Type":"ContainerStarted","Data":"29e55163f6bac20678046bd1c04d721c9a080bead8d76c855230b696ceb33cb4"}
Jul 22 16:30:09.147873 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:30:09.147822    2563 generic.go:356] "Generic (PLEG): container finished" podID="9179d684-6a21-4aaa-9f9f-d8a3f55360c2" containerID="84ff2a758695eddf9ee23a4aae47572afd61fd54dd184ae9846ceb4d056d9c72" exitCode=0
Jul 22 16:30:09.148289 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:30:09.147891    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745630-v5cfc" event={"ID":"9179d684-6a21-4aaa-9f9f-d8a3f55360c2","Type":"ContainerDied","Data":"84ff2a758695eddf9ee23a4aae47572afd61fd54dd184ae9846ceb4d056d9c72"}
Jul 22 16:30:09.393091 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:30:09.393000    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:30:09.451360 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:30:09.451331    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:30:10.277623 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:30:10.277597    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745630-v5cfc"
Jul 22 16:30:11.157979 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:30:11.157944    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745630-v5cfc" event={"ID":"9179d684-6a21-4aaa-9f9f-d8a3f55360c2","Type":"ContainerDied","Data":"29e55163f6bac20678046bd1c04d721c9a080bead8d76c855230b696ceb33cb4"}
Jul 22 16:30:11.157979 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:30:11.157969    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745630-v5cfc"
Jul 22 16:30:11.158206 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:30:11.157990    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="29e55163f6bac20678046bd1c04d721c9a080bead8d76c855230b696ceb33cb4"
Jul 22 16:31:08.160377 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:08.160343    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn"]
Jul 22 16:31:08.160846 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:08.160676    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9179d684-6a21-4aaa-9f9f-d8a3f55360c2" containerName="cleanup"
Jul 22 16:31:08.160846 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:08.160688    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9179d684-6a21-4aaa-9f9f-d8a3f55360c2" containerName="cleanup"
Jul 22 16:31:08.160846 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:08.160757    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9179d684-6a21-4aaa-9f9f-d8a3f55360c2" containerName="cleanup"
Jul 22 16:31:08.163986 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:08.163972    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn"
Jul 22 16:31:08.167355 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:08.167336    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-c04d8c33-kserve-self-signed-certs\""
Jul 22 16:31:08.189792 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:08.189765    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn"]
Jul 22 16:31:08.224665 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:08.224631    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdjs9\" (UniqueName: \"kubernetes.io/projected/a0de6dc5-b2dd-4293-bf86-28716cc0770a-kube-api-access-qdjs9\") pod \"e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn\" (UID: \"a0de6dc5-b2dd-4293-bf86-28716cc0770a\") " pod="llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn"
Jul 22 16:31:08.224832 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:08.224689    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0de6dc5-b2dd-4293-bf86-28716cc0770a-home\") pod \"e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn\" (UID: \"a0de6dc5-b2dd-4293-bf86-28716cc0770a\") " pod="llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn"
Jul 22 16:31:08.224832 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:08.224722    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0de6dc5-b2dd-4293-bf86-28716cc0770a-model-cache\") pod \"e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn\" (UID: \"a0de6dc5-b2dd-4293-bf86-28716cc0770a\") " pod="llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn"
Jul 22 16:31:08.224832 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:08.224751    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0de6dc5-b2dd-4293-bf86-28716cc0770a-tls-certs\") pod \"e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn\" (UID: \"a0de6dc5-b2dd-4293-bf86-28716cc0770a\") " pod="llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn"
Jul 22 16:31:08.224832 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:08.224774    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0de6dc5-b2dd-4293-bf86-28716cc0770a-dshm\") pod \"e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn\" (UID: \"a0de6dc5-b2dd-4293-bf86-28716cc0770a\") " pod="llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn"
Jul 22 16:31:08.224832 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:08.224796    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0de6dc5-b2dd-4293-bf86-28716cc0770a-tmp-dir\") pod \"e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn\" (UID: \"a0de6dc5-b2dd-4293-bf86-28716cc0770a\") " pod="llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn"
Jul 22 16:31:08.278062 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:08.278031    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:31:08.325190 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:08.325158    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0de6dc5-b2dd-4293-bf86-28716cc0770a-tmp-dir\") pod \"e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn\" (UID: \"a0de6dc5-b2dd-4293-bf86-28716cc0770a\") " pod="llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn"
Jul 22 16:31:08.325355 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:08.325209    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdjs9\" (UniqueName: \"kubernetes.io/projected/a0de6dc5-b2dd-4293-bf86-28716cc0770a-kube-api-access-qdjs9\") pod \"e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn\" (UID: \"a0de6dc5-b2dd-4293-bf86-28716cc0770a\") " pod="llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn"
Jul 22 16:31:08.325355 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:08.325238    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0de6dc5-b2dd-4293-bf86-28716cc0770a-home\") pod \"e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn\" (UID: \"a0de6dc5-b2dd-4293-bf86-28716cc0770a\") " pod="llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn"
Jul 22 16:31:08.325355 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:08.325268    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0de6dc5-b2dd-4293-bf86-28716cc0770a-model-cache\") pod \"e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn\" (UID: \"a0de6dc5-b2dd-4293-bf86-28716cc0770a\") " pod="llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn"
Jul 22 16:31:08.325355 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:08.325296    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0de6dc5-b2dd-4293-bf86-28716cc0770a-tls-certs\") pod \"e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn\" (UID: \"a0de6dc5-b2dd-4293-bf86-28716cc0770a\") " pod="llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn"
Jul 22 16:31:08.325355 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:08.325317    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0de6dc5-b2dd-4293-bf86-28716cc0770a-dshm\") pod \"e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn\" (UID: \"a0de6dc5-b2dd-4293-bf86-28716cc0770a\") " pod="llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn"
Jul 22 16:31:08.325614 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:08.325561    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0de6dc5-b2dd-4293-bf86-28716cc0770a-tmp-dir\") pod \"e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn\" (UID: \"a0de6dc5-b2dd-4293-bf86-28716cc0770a\") " pod="llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn"
Jul 22 16:31:08.325829 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:08.325677    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0de6dc5-b2dd-4293-bf86-28716cc0770a-model-cache\") pod \"e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn\" (UID: \"a0de6dc5-b2dd-4293-bf86-28716cc0770a\") " pod="llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn"
Jul 22 16:31:08.325829 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:08.325726    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0de6dc5-b2dd-4293-bf86-28716cc0770a-home\") pod \"e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn\" (UID: \"a0de6dc5-b2dd-4293-bf86-28716cc0770a\") " pod="llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn"
Jul 22 16:31:08.327490 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:08.327472    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0de6dc5-b2dd-4293-bf86-28716cc0770a-dshm\") pod \"e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn\" (UID: \"a0de6dc5-b2dd-4293-bf86-28716cc0770a\") " pod="llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn"
Jul 22 16:31:08.327746 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:08.327726    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0de6dc5-b2dd-4293-bf86-28716cc0770a-tls-certs\") pod \"e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn\" (UID: \"a0de6dc5-b2dd-4293-bf86-28716cc0770a\") " pod="llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn"
Jul 22 16:31:08.335958 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:08.335934    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdjs9\" (UniqueName: \"kubernetes.io/projected/a0de6dc5-b2dd-4293-bf86-28716cc0770a-kube-api-access-qdjs9\") pod \"e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn\" (UID: \"a0de6dc5-b2dd-4293-bf86-28716cc0770a\") " pod="llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn"
Jul 22 16:31:08.472512 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:08.472481    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn"
Jul 22 16:31:08.609656 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:08.609617    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn"]
Jul 22 16:31:08.613778 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:31:08.613749    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda0de6dc5_b2dd_4293_bf86_28716cc0770a.slice/crio-43653dbe7d021bace6ab0d25b4402d9866460f5c3179f7cc3a1fbd14ff03a211 WatchSource:0}: Error finding container 43653dbe7d021bace6ab0d25b4402d9866460f5c3179f7cc3a1fbd14ff03a211: Status 404 returned error can't find the container with id 43653dbe7d021bace6ab0d25b4402d9866460f5c3179f7cc3a1fbd14ff03a211
Jul 22 16:31:09.404240 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:09.404206    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn" event={"ID":"a0de6dc5-b2dd-4293-bf86-28716cc0770a","Type":"ContainerStarted","Data":"7b43f48d63d84af81463e8c0c68886c5fd8f7b30f0c07bd164b7c8c54b109904"}
Jul 22 16:31:09.404240 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:09.404246    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn" event={"ID":"a0de6dc5-b2dd-4293-bf86-28716cc0770a","Type":"ContainerStarted","Data":"43653dbe7d021bace6ab0d25b4402d9866460f5c3179f7cc3a1fbd14ff03a211"}
Jul 22 16:31:09.426022 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:09.425972    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn" podStartSLOduration=1.425957102 podStartE2EDuration="1.425957102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:31:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:31:09.424745493 +0000 UTC m=+1429.555215495" watchObservedRunningTime="2026-07-22 16:31:09.425957102 +0000 UTC m=+1429.556427163"
Jul 22 16:31:17.044116 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:17.044040    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk"]
Jul 22 16:31:17.047644 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:17.047623    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk"
Jul 22 16:31:17.051560 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:17.051539    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-c04d8c33-kserve-self-signed-certs\""
Jul 22 16:31:17.063610 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:17.063583    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk"]
Jul 22 16:31:17.074657 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:17.074632    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:31:17.209678 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:17.209629    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-tmp-dir\") pod \"e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk\" (UID: \"0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f\") " pod="llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk"
Jul 22 16:31:17.209678 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:17.209673    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-tls-certs\") pod \"e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk\" (UID: \"0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f\") " pod="llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk"
Jul 22 16:31:17.210004 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:17.209710    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjpzq\" (UniqueName: \"kubernetes.io/projected/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-kube-api-access-wjpzq\") pod \"e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk\" (UID: \"0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f\") " pod="llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk"
Jul 22 16:31:17.210004 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:17.209742    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-home\") pod \"e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk\" (UID: \"0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f\") " pod="llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk"
Jul 22 16:31:17.210004 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:17.209779    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-model-cache\") pod \"e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk\" (UID: \"0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f\") " pod="llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk"
Jul 22 16:31:17.210004 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:17.209870    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-dshm\") pod \"e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk\" (UID: \"0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f\") " pod="llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk"
Jul 22 16:31:17.311277 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:17.311191    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-dshm\") pod \"e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk\" (UID: \"0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f\") " pod="llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk"
Jul 22 16:31:17.311277 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:17.311248    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-tmp-dir\") pod \"e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk\" (UID: \"0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f\") " pod="llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk"
Jul 22 16:31:17.311472 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:17.311288    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-tls-certs\") pod \"e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk\" (UID: \"0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f\") " pod="llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk"
Jul 22 16:31:17.311472 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:17.311318    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjpzq\" (UniqueName: \"kubernetes.io/projected/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-kube-api-access-wjpzq\") pod \"e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk\" (UID: \"0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f\") " pod="llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk"
Jul 22 16:31:17.311472 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:17.311351    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-home\") pod \"e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk\" (UID: \"0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f\") " pod="llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk"
Jul 22 16:31:17.311472 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:17.311375    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-model-cache\") pod \"e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk\" (UID: \"0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f\") " pod="llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk"
Jul 22 16:31:17.311727 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:17.311700    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-tmp-dir\") pod \"e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk\" (UID: \"0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f\") " pod="llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk"
Jul 22 16:31:17.311788 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:17.311733    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-home\") pod \"e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk\" (UID: \"0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f\") " pod="llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk"
Jul 22 16:31:17.311788 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:17.311779    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-model-cache\") pod \"e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk\" (UID: \"0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f\") " pod="llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk"
Jul 22 16:31:17.313516 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:17.313492    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-dshm\") pod \"e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk\" (UID: \"0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f\") " pod="llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk"
Jul 22 16:31:17.313655 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:17.313639    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-tls-certs\") pod \"e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk\" (UID: \"0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f\") " pod="llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk"
Jul 22 16:31:17.326097 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:17.326076    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjpzq\" (UniqueName: \"kubernetes.io/projected/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-kube-api-access-wjpzq\") pod \"e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk\" (UID: \"0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f\") " pod="llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk"
Jul 22 16:31:17.356865 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:17.356831    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk"
Jul 22 16:31:17.488813 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:17.488608    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk"]
Jul 22 16:31:17.491137 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:31:17.491107    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0eb3ac74_f73f_4f1f_a0fc_63b9b4c2490f.slice/crio-9ce3748519e0829328ce1bb87578f5cfe99028ff88fc7eb8d2af0c57020d82ba WatchSource:0}: Error finding container 9ce3748519e0829328ce1bb87578f5cfe99028ff88fc7eb8d2af0c57020d82ba: Status 404 returned error can't find the container with id 9ce3748519e0829328ce1bb87578f5cfe99028ff88fc7eb8d2af0c57020d82ba
Jul 22 16:31:18.447766 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:18.447731    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk" event={"ID":"0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f","Type":"ContainerStarted","Data":"f885de1e939f231a50a7066977d276f4209662ec9b501abcef7798eaa7d90d8e"}
Jul 22 16:31:18.448166 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:18.447773    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk" event={"ID":"0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f","Type":"ContainerStarted","Data":"9ce3748519e0829328ce1bb87578f5cfe99028ff88fc7eb8d2af0c57020d82ba"}
Jul 22 16:31:18.466684 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:18.466632    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk" podStartSLOduration=1.466614958 podStartE2EDuration="1.466614958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:31:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:31:18.465334242 +0000 UTC m=+1438.595804255" watchObservedRunningTime="2026-07-22 16:31:18.466614958 +0000 UTC m=+1438.597084959"
Jul 22 16:31:18.472611 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:18.472581    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn"
Jul 22 16:31:18.472742 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:18.472630    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn"
Jul 22 16:31:18.485650 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:18.485624    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn"
Jul 22 16:31:19.461399 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:19.461371    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn"
Jul 22 16:31:27.357185 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:27.357138    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk"
Jul 22 16:31:27.357185 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:27.357196    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk"
Jul 22 16:31:27.369655 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:27.369630    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk"
Jul 22 16:31:27.496464 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:27.496437    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk"
Jul 22 16:31:47.860729 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:31:47.860679    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:32:05.897493 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:05.897448    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn"]
Jul 22 16:32:05.928517 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:05.928481    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:32:07.512931 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:07.512894    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk"]
Jul 22 16:32:07.540272 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:07.540235    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:32:20.918493 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:20.918448    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn" podUID="a0de6dc5-b2dd-4293-bf86-28716cc0770a" containerName="main" containerID="cri-o://7b43f48d63d84af81463e8c0c68886c5fd8f7b30f0c07bd164b7c8c54b109904" gracePeriod=45
Jul 22 16:32:21.157576 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:21.157544    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn"
Jul 22 16:32:21.260620 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:21.260583    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0de6dc5-b2dd-4293-bf86-28716cc0770a-home\") pod \"a0de6dc5-b2dd-4293-bf86-28716cc0770a\" (UID: \"a0de6dc5-b2dd-4293-bf86-28716cc0770a\") "
Jul 22 16:32:21.260620 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:21.260624    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0de6dc5-b2dd-4293-bf86-28716cc0770a-tls-certs\") pod \"a0de6dc5-b2dd-4293-bf86-28716cc0770a\" (UID: \"a0de6dc5-b2dd-4293-bf86-28716cc0770a\") "
Jul 22 16:32:21.260914 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:21.260659    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0de6dc5-b2dd-4293-bf86-28716cc0770a-dshm\") pod \"a0de6dc5-b2dd-4293-bf86-28716cc0770a\" (UID: \"a0de6dc5-b2dd-4293-bf86-28716cc0770a\") "
Jul 22 16:32:21.260914 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:21.260705    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0de6dc5-b2dd-4293-bf86-28716cc0770a-tmp-dir\") pod \"a0de6dc5-b2dd-4293-bf86-28716cc0770a\" (UID: \"a0de6dc5-b2dd-4293-bf86-28716cc0770a\") "
Jul 22 16:32:21.260914 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:21.260757    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qdjs9\" (UniqueName: \"kubernetes.io/projected/a0de6dc5-b2dd-4293-bf86-28716cc0770a-kube-api-access-qdjs9\") pod \"a0de6dc5-b2dd-4293-bf86-28716cc0770a\" (UID: \"a0de6dc5-b2dd-4293-bf86-28716cc0770a\") "
Jul 22 16:32:21.260914 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:21.260803    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0de6dc5-b2dd-4293-bf86-28716cc0770a-model-cache\") pod \"a0de6dc5-b2dd-4293-bf86-28716cc0770a\" (UID: \"a0de6dc5-b2dd-4293-bf86-28716cc0770a\") "
Jul 22 16:32:21.261084 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:21.260912    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0de6dc5-b2dd-4293-bf86-28716cc0770a-home" (OuterVolumeSpecName: "home") pod "a0de6dc5-b2dd-4293-bf86-28716cc0770a" (UID: "a0de6dc5-b2dd-4293-bf86-28716cc0770a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:32:21.261084 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:21.261053    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0de6dc5-b2dd-4293-bf86-28716cc0770a-model-cache" (OuterVolumeSpecName: "model-cache") pod "a0de6dc5-b2dd-4293-bf86-28716cc0770a" (UID: "a0de6dc5-b2dd-4293-bf86-28716cc0770a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:32:21.261084 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:21.261063    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0de6dc5-b2dd-4293-bf86-28716cc0770a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a0de6dc5-b2dd-4293-bf86-28716cc0770a" (UID: "a0de6dc5-b2dd-4293-bf86-28716cc0770a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:32:21.261193 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:21.261150    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0de6dc5-b2dd-4293-bf86-28716cc0770a-home\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:32:21.261227 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:21.261196    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0de6dc5-b2dd-4293-bf86-28716cc0770a-tmp-dir\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:32:21.261227 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:21.261212    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0de6dc5-b2dd-4293-bf86-28716cc0770a-model-cache\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:32:21.263149 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:21.263114    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0de6dc5-b2dd-4293-bf86-28716cc0770a-kube-api-access-qdjs9" (OuterVolumeSpecName: "kube-api-access-qdjs9") pod "a0de6dc5-b2dd-4293-bf86-28716cc0770a" (UID: "a0de6dc5-b2dd-4293-bf86-28716cc0770a"). InnerVolumeSpecName "kube-api-access-qdjs9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:32:21.263149 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:21.263130    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0de6dc5-b2dd-4293-bf86-28716cc0770a-dshm" (OuterVolumeSpecName: "dshm") pod "a0de6dc5-b2dd-4293-bf86-28716cc0770a" (UID: "a0de6dc5-b2dd-4293-bf86-28716cc0770a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:32:21.263294 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:21.263179    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0de6dc5-b2dd-4293-bf86-28716cc0770a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a0de6dc5-b2dd-4293-bf86-28716cc0770a" (UID: "a0de6dc5-b2dd-4293-bf86-28716cc0770a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:32:21.361928 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:21.361895    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0de6dc5-b2dd-4293-bf86-28716cc0770a-tls-certs\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:32:21.361928 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:21.361920    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0de6dc5-b2dd-4293-bf86-28716cc0770a-dshm\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:32:21.361928 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:21.361932    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdjs9\" (UniqueName: \"kubernetes.io/projected/a0de6dc5-b2dd-4293-bf86-28716cc0770a-kube-api-access-qdjs9\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:32:21.717758 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:21.717717    2563 generic.go:356] "Generic (PLEG): container finished" podID="a0de6dc5-b2dd-4293-bf86-28716cc0770a" containerID="7b43f48d63d84af81463e8c0c68886c5fd8f7b30f0c07bd164b7c8c54b109904" exitCode=0
Jul 22 16:32:21.717758 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:21.717758    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn"
Jul 22 16:32:21.717978 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:21.717803    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn" event={"ID":"a0de6dc5-b2dd-4293-bf86-28716cc0770a","Type":"ContainerDied","Data":"7b43f48d63d84af81463e8c0c68886c5fd8f7b30f0c07bd164b7c8c54b109904"}
Jul 22 16:32:21.717978 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:21.717847    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn" event={"ID":"a0de6dc5-b2dd-4293-bf86-28716cc0770a","Type":"ContainerDied","Data":"43653dbe7d021bace6ab0d25b4402d9866460f5c3179f7cc3a1fbd14ff03a211"}
Jul 22 16:32:21.717978 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:21.717888    2563 scope.go:117] "RemoveContainer" containerID="7b43f48d63d84af81463e8c0c68886c5fd8f7b30f0c07bd164b7c8c54b109904"
Jul 22 16:32:21.727050 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:21.727028    2563 scope.go:117] "RemoveContainer" containerID="7b43f48d63d84af81463e8c0c68886c5fd8f7b30f0c07bd164b7c8c54b109904"
Jul 22 16:32:21.727317 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:32:21.727300    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b43f48d63d84af81463e8c0c68886c5fd8f7b30f0c07bd164b7c8c54b109904\": container with ID starting with 7b43f48d63d84af81463e8c0c68886c5fd8f7b30f0c07bd164b7c8c54b109904 not found: ID does not exist" containerID="7b43f48d63d84af81463e8c0c68886c5fd8f7b30f0c07bd164b7c8c54b109904"
Jul 22 16:32:21.727368 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:21.727328    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b43f48d63d84af81463e8c0c68886c5fd8f7b30f0c07bd164b7c8c54b109904"} err="failed to get container status \"7b43f48d63d84af81463e8c0c68886c5fd8f7b30f0c07bd164b7c8c54b109904\": rpc error: code = NotFound desc = could not find container \"7b43f48d63d84af81463e8c0c68886c5fd8f7b30f0c07bd164b7c8c54b109904\": container with ID starting with 7b43f48d63d84af81463e8c0c68886c5fd8f7b30f0c07bd164b7c8c54b109904 not found: ID does not exist"
Jul 22 16:32:21.755071 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:21.755036    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn"]
Jul 22 16:32:21.764698 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:21.764670    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-c04d8c33-kserve-6658c75d6c-mxfwn"]
Jul 22 16:32:22.448980 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:22.448948    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0de6dc5-b2dd-4293-bf86-28716cc0770a" path="/var/lib/kubelet/pods/a0de6dc5-b2dd-4293-bf86-28716cc0770a/volumes"
Jul 22 16:32:22.532798 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:22.532752    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk" podUID="0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f" containerName="main" containerID="cri-o://f885de1e939f231a50a7066977d276f4209662ec9b501abcef7798eaa7d90d8e" gracePeriod=45
Jul 22 16:32:22.725073 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:22.725043    2563 generic.go:356] "Generic (PLEG): container finished" podID="0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f" containerID="f885de1e939f231a50a7066977d276f4209662ec9b501abcef7798eaa7d90d8e" exitCode=0
Jul 22 16:32:22.725225 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:22.725112    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk" event={"ID":"0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f","Type":"ContainerDied","Data":"f885de1e939f231a50a7066977d276f4209662ec9b501abcef7798eaa7d90d8e"}
Jul 22 16:32:22.779019 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:22.778995    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk"
Jul 22 16:32:22.874599 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:22.874566    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-dshm\") pod \"0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f\" (UID: \"0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f\") "
Jul 22 16:32:22.874599 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:22.874608    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wjpzq\" (UniqueName: \"kubernetes.io/projected/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-kube-api-access-wjpzq\") pod \"0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f\" (UID: \"0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f\") "
Jul 22 16:32:22.874846 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:22.874652    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-tls-certs\") pod \"0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f\" (UID: \"0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f\") "
Jul 22 16:32:22.874846 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:22.874672    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-model-cache\") pod \"0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f\" (UID: \"0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f\") "
Jul 22 16:32:22.874846 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:22.874690    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-tmp-dir\") pod \"0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f\" (UID: \"0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f\") "
Jul 22 16:32:22.874846 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:22.874711    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-home\") pod \"0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f\" (UID: \"0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f\") "
Jul 22 16:32:22.875080 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:22.875035    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f" (UID: "0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:32:22.875125 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:22.875071    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-model-cache" (OuterVolumeSpecName: "model-cache") pod "0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f" (UID: "0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:32:22.875163 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:22.875112    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-home" (OuterVolumeSpecName: "home") pod "0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f" (UID: "0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:32:22.876883 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:22.876825    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-dshm" (OuterVolumeSpecName: "dshm") pod "0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f" (UID: "0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:32:22.876883 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:22.876874    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-kube-api-access-wjpzq" (OuterVolumeSpecName: "kube-api-access-wjpzq") pod "0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f" (UID: "0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f"). InnerVolumeSpecName "kube-api-access-wjpzq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:32:22.877055 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:22.876963    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f" (UID: "0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:32:22.975977 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:22.975905    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-tls-certs\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:32:22.975977 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:22.975928    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-model-cache\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:32:22.975977 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:22.975936    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-tmp-dir\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:32:22.975977 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:22.975945    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-home\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:32:22.975977 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:22.975953    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-dshm\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:32:22.975977 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:22.975961    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjpzq\" (UniqueName: \"kubernetes.io/projected/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f-kube-api-access-wjpzq\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:32:23.736261 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:23.736222    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk" event={"ID":"0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f","Type":"ContainerDied","Data":"9ce3748519e0829328ce1bb87578f5cfe99028ff88fc7eb8d2af0c57020d82ba"}
Jul 22 16:32:23.736730 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:23.736270    2563 scope.go:117] "RemoveContainer" containerID="f885de1e939f231a50a7066977d276f4209662ec9b501abcef7798eaa7d90d8e"
Jul 22 16:32:23.736730 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:23.736236    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk"
Jul 22 16:32:23.764442 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:23.764413    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk"]
Jul 22 16:32:23.770348 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:23.770324    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-c04d8c33-kserve-5f8fd897f9-h8xqk"]
Jul 22 16:32:24.447770 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:32:24.447736    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f" path="/var/lib/kubelet/pods/0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f/volumes"
Jul 22 16:34:47.292825 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:34:47.292785    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:34:48.998587 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:34:48.998554    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:35:15.509585 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:15.509543    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:35:41.154207 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:41.154151    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:35:44.199174 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:44.199134    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:35:44.478625 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:44.478486    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:35:44.652272 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:44.652238    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:35:50.572844 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:50.572777    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:35:50.637325 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:50.637285    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:35:50.806612 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:50.806581    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:35:51.134285 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:51.134260    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:35:51.434356 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:51.434322    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:35:53.923369 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:53.923341    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:35:54.008123 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:54.008090    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz"]
Jul 22 16:35:54.008484 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:54.008467    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f" containerName="main"
Jul 22 16:35:54.008604 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:54.008486    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f" containerName="main"
Jul 22 16:35:54.008604 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:54.008503    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0de6dc5-b2dd-4293-bf86-28716cc0770a" containerName="main"
Jul 22 16:35:54.008604 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:54.008512    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0de6dc5-b2dd-4293-bf86-28716cc0770a" containerName="main"
Jul 22 16:35:54.008604 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:54.008589    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0eb3ac74-f73f-4f1f-a0fc-63b9b4c2490f" containerName="main"
Jul 22 16:35:54.008604 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:54.008606    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0de6dc5-b2dd-4293-bf86-28716cc0770a" containerName="main"
Jul 22 16:35:54.011603 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:54.011581    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz"
Jul 22 16:35:54.015338 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:54.015258    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-d0a29ed9-serving-cert\""
Jul 22 16:35:54.015338 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:54.015280    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 22 16:35:54.015338 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:54.015327    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 16:35:54.016515 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:54.016498    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vghv2\""
Jul 22 16:35:54.019612 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:54.019593    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:35:54.024409 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:54.024384    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz"]
Jul 22 16:35:54.120456 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:54.120418    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbp4q\" (UniqueName: \"kubernetes.io/projected/bd5de662-37e6-4582-b018-507128c9d682-kube-api-access-kbp4q\") pod \"maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz\" (UID: \"bd5de662-37e6-4582-b018-507128c9d682\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz"
Jul 22 16:35:54.120660 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:54.120575    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd5de662-37e6-4582-b018-507128c9d682-openshift-service-ca\") pod \"maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz\" (UID: \"bd5de662-37e6-4582-b018-507128c9d682\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz"
Jul 22 16:35:54.120660 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:54.120620    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd5de662-37e6-4582-b018-507128c9d682-maas-api-tls\") pod \"maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz\" (UID: \"bd5de662-37e6-4582-b018-507128c9d682\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz"
Jul 22 16:35:54.165087 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:54.165052    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:35:54.221107 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:54.221073    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd5de662-37e6-4582-b018-507128c9d682-openshift-service-ca\") pod \"maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz\" (UID: \"bd5de662-37e6-4582-b018-507128c9d682\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz"
Jul 22 16:35:54.221268 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:54.221116    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd5de662-37e6-4582-b018-507128c9d682-maas-api-tls\") pod \"maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz\" (UID: \"bd5de662-37e6-4582-b018-507128c9d682\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz"
Jul 22 16:35:54.221268 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:54.221168    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbp4q\" (UniqueName: \"kubernetes.io/projected/bd5de662-37e6-4582-b018-507128c9d682-kube-api-access-kbp4q\") pod \"maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz\" (UID: \"bd5de662-37e6-4582-b018-507128c9d682\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz"
Jul 22 16:35:54.221340 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:35:54.221316    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-d0a29ed9-serving-cert: secret "maas-api-e2e-ait-d0a29ed9-serving-cert" not found
Jul 22 16:35:54.221419 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:35:54.221407    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd5de662-37e6-4582-b018-507128c9d682-maas-api-tls podName:bd5de662-37e6-4582-b018-507128c9d682 nodeName:}" failed. No retries permitted until 2026-07-22 16:35:54.721383871 +0000 UTC m=+1714.851853854 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bd5de662-37e6-4582-b018-507128c9d682-maas-api-tls") pod "maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz" (UID: "bd5de662-37e6-4582-b018-507128c9d682") : secret "maas-api-e2e-ait-d0a29ed9-serving-cert" not found
Jul 22 16:35:54.221778 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:54.221761    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd5de662-37e6-4582-b018-507128c9d682-openshift-service-ca\") pod \"maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz\" (UID: \"bd5de662-37e6-4582-b018-507128c9d682\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz"
Jul 22 16:35:54.245583 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:54.245559    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbp4q\" (UniqueName: \"kubernetes.io/projected/bd5de662-37e6-4582-b018-507128c9d682-kube-api-access-kbp4q\") pod \"maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz\" (UID: \"bd5de662-37e6-4582-b018-507128c9d682\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz"
Jul 22 16:35:54.373363 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:54.373325    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:35:54.725976 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:54.725944    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd5de662-37e6-4582-b018-507128c9d682-maas-api-tls\") pod \"maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz\" (UID: \"bd5de662-37e6-4582-b018-507128c9d682\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz"
Jul 22 16:35:54.728499 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:54.728476    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd5de662-37e6-4582-b018-507128c9d682-maas-api-tls\") pod \"maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz\" (UID: \"bd5de662-37e6-4582-b018-507128c9d682\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz"
Jul 22 16:35:54.752310 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:54.752275    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:35:54.854361 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:54.854324    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:35:54.922027 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:54.921995    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz"
Jul 22 16:35:55.061176 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:55.060982    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz"]
Jul 22 16:35:55.067660 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:55.067634    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 16:35:55.631520 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:55.631483    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz" event={"ID":"bd5de662-37e6-4582-b018-507128c9d682","Type":"ContainerStarted","Data":"39ae9eeccb2d0b9de47d24a9994d22803d0a9e902506c7df0dab1130c6ddaeee"}
Jul 22 16:35:57.642710 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:57.642676    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz" event={"ID":"bd5de662-37e6-4582-b018-507128c9d682","Type":"ContainerStarted","Data":"0eeb409ab2834750c5a187afde075d4e1a03e546938652fa6b49173da57bd675"}
Jul 22 16:35:57.643127 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:57.642791    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz"
Jul 22 16:35:57.663424 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:35:57.663372    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz" podStartSLOduration=2.928913342 podStartE2EDuration="4.663355366s" totalImagesPullingTime="1.734442024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:35:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:35:57.661351873 +0000 UTC m=+1717.791821928" watchObservedRunningTime="2026-07-22 16:35:57.663355366 +0000 UTC m=+1717.793825368"
Jul 22 16:36:03.653267 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:03.653238    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz"
Jul 22 16:36:06.305482 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:06.305448    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:36:19.554523 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:19.552148    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d0a29ed9-zdjtn"]
Jul 22 16:36:19.558863 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:19.558831    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d0a29ed9-zdjtn"
Jul 22 16:36:19.562392 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:19.562369    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-phfdj\""
Jul 22 16:36:19.565741 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:19.565716    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d0a29ed9-zdjtn"]
Jul 22 16:36:19.645927 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:19.645895    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71de64bd-db99-4543-8d9b-bfb339775e93-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d0a29ed9-zdjtn\" (UID: \"71de64bd-db99-4543-8d9b-bfb339775e93\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d0a29ed9-zdjtn"
Jul 22 16:36:19.746800 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:19.746767    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71de64bd-db99-4543-8d9b-bfb339775e93-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d0a29ed9-zdjtn\" (UID: \"71de64bd-db99-4543-8d9b-bfb339775e93\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d0a29ed9-zdjtn"
Jul 22 16:36:19.747456 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:19.747435    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71de64bd-db99-4543-8d9b-bfb339775e93-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d0a29ed9-zdjtn\" (UID: \"71de64bd-db99-4543-8d9b-bfb339775e93\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d0a29ed9-zdjtn"
Jul 22 16:36:19.870775 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:19.870691    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d0a29ed9-zdjtn"
Jul 22 16:36:20.004549 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:20.004523    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d0a29ed9-zdjtn"]
Jul 22 16:36:20.006876 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:36:20.006824    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71de64bd_db99_4543_8d9b_bfb339775e93.slice/crio-58b387959d755bcc026076f2667664f3897b5c95a607cb6d044f57f6191e4c54 WatchSource:0}: Error finding container 58b387959d755bcc026076f2667664f3897b5c95a607cb6d044f57f6191e4c54: Status 404 returned error can't find the container with id 58b387959d755bcc026076f2667664f3897b5c95a607cb6d044f57f6191e4c54
Jul 22 16:36:20.742694 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:20.742662    2563 generic.go:356] "Generic (PLEG): container finished" podID="71de64bd-db99-4543-8d9b-bfb339775e93" containerID="02f8140c59db6de07c7fd32fdb13d791013869521e5c35ac5cff0095c03ee332" exitCode=0
Jul 22 16:36:20.743193 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:20.742782    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d0a29ed9-zdjtn" event={"ID":"71de64bd-db99-4543-8d9b-bfb339775e93","Type":"ContainerDied","Data":"02f8140c59db6de07c7fd32fdb13d791013869521e5c35ac5cff0095c03ee332"}
Jul 22 16:36:20.743193 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:20.742842    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d0a29ed9-zdjtn" event={"ID":"71de64bd-db99-4543-8d9b-bfb339775e93","Type":"ContainerStarted","Data":"58b387959d755bcc026076f2667664f3897b5c95a607cb6d044f57f6191e4c54"}
Jul 22 16:36:21.877194 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:21.877168    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d0a29ed9-zdjtn"
Jul 22 16:36:21.968484 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:21.968445    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71de64bd-db99-4543-8d9b-bfb339775e93-maas-api-service-ca\") pod \"71de64bd-db99-4543-8d9b-bfb339775e93\" (UID: \"71de64bd-db99-4543-8d9b-bfb339775e93\") "
Jul 22 16:36:21.968803 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:21.968780    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71de64bd-db99-4543-8d9b-bfb339775e93-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "71de64bd-db99-4543-8d9b-bfb339775e93" (UID: "71de64bd-db99-4543-8d9b-bfb339775e93"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 16:36:22.070148 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:22.070061    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71de64bd-db99-4543-8d9b-bfb339775e93-maas-api-service-ca\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:36:22.753092 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:22.753005    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d0a29ed9-zdjtn" event={"ID":"71de64bd-db99-4543-8d9b-bfb339775e93","Type":"ContainerDied","Data":"58b387959d755bcc026076f2667664f3897b5c95a607cb6d044f57f6191e4c54"}
Jul 22 16:36:22.753092 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:22.753031    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d0a29ed9-zdjtn"
Jul 22 16:36:22.753092 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:22.753059    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="58b387959d755bcc026076f2667664f3897b5c95a607cb6d044f57f6191e4c54"
Jul 22 16:36:30.130578 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:30.130542    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz"]
Jul 22 16:36:30.131070 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:30.130741    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz" podUID="bd5de662-37e6-4582-b018-507128c9d682" containerName="maas-api" containerID="cri-o://0eeb409ab2834750c5a187afde075d4e1a03e546938652fa6b49173da57bd675" gracePeriod=30
Jul 22 16:36:30.170026 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:30.169991    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:36:30.375192 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:30.375163    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz"
Jul 22 16:36:30.438815 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:30.438785    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbp4q\" (UniqueName: \"kubernetes.io/projected/bd5de662-37e6-4582-b018-507128c9d682-kube-api-access-kbp4q\") pod \"bd5de662-37e6-4582-b018-507128c9d682\" (UID: \"bd5de662-37e6-4582-b018-507128c9d682\") "
Jul 22 16:36:30.439061 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:30.438828    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd5de662-37e6-4582-b018-507128c9d682-openshift-service-ca\") pod \"bd5de662-37e6-4582-b018-507128c9d682\" (UID: \"bd5de662-37e6-4582-b018-507128c9d682\") "
Jul 22 16:36:30.439061 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:30.438927    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd5de662-37e6-4582-b018-507128c9d682-maas-api-tls\") pod \"bd5de662-37e6-4582-b018-507128c9d682\" (UID: \"bd5de662-37e6-4582-b018-507128c9d682\") "
Jul 22 16:36:30.439256 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:30.439234    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd5de662-37e6-4582-b018-507128c9d682-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bd5de662-37e6-4582-b018-507128c9d682" (UID: "bd5de662-37e6-4582-b018-507128c9d682"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 16:36:30.441200 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:30.441169    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd5de662-37e6-4582-b018-507128c9d682-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bd5de662-37e6-4582-b018-507128c9d682" (UID: "bd5de662-37e6-4582-b018-507128c9d682"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:36:30.441325 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:30.441174    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd5de662-37e6-4582-b018-507128c9d682-kube-api-access-kbp4q" (OuterVolumeSpecName: "kube-api-access-kbp4q") pod "bd5de662-37e6-4582-b018-507128c9d682" (UID: "bd5de662-37e6-4582-b018-507128c9d682"). InnerVolumeSpecName "kube-api-access-kbp4q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:36:30.539657 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:30.539620    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd5de662-37e6-4582-b018-507128c9d682-maas-api-tls\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:36:30.539657 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:30.539653    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbp4q\" (UniqueName: \"kubernetes.io/projected/bd5de662-37e6-4582-b018-507128c9d682-kube-api-access-kbp4q\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:36:30.539657 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:30.539663    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd5de662-37e6-4582-b018-507128c9d682-openshift-service-ca\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:36:30.788484 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:30.788391    2563 generic.go:356] "Generic (PLEG): container finished" podID="bd5de662-37e6-4582-b018-507128c9d682" containerID="0eeb409ab2834750c5a187afde075d4e1a03e546938652fa6b49173da57bd675" exitCode=0
Jul 22 16:36:30.788484 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:30.788441    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz"
Jul 22 16:36:30.788484 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:30.788474    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz" event={"ID":"bd5de662-37e6-4582-b018-507128c9d682","Type":"ContainerDied","Data":"0eeb409ab2834750c5a187afde075d4e1a03e546938652fa6b49173da57bd675"}
Jul 22 16:36:30.788767 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:30.788525    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz" event={"ID":"bd5de662-37e6-4582-b018-507128c9d682","Type":"ContainerDied","Data":"39ae9eeccb2d0b9de47d24a9994d22803d0a9e902506c7df0dab1130c6ddaeee"}
Jul 22 16:36:30.788767 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:30.788548    2563 scope.go:117] "RemoveContainer" containerID="0eeb409ab2834750c5a187afde075d4e1a03e546938652fa6b49173da57bd675"
Jul 22 16:36:30.797064 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:30.797048    2563 scope.go:117] "RemoveContainer" containerID="0eeb409ab2834750c5a187afde075d4e1a03e546938652fa6b49173da57bd675"
Jul 22 16:36:30.797359 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:36:30.797335    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0eeb409ab2834750c5a187afde075d4e1a03e546938652fa6b49173da57bd675\": container with ID starting with 0eeb409ab2834750c5a187afde075d4e1a03e546938652fa6b49173da57bd675 not found: ID does not exist" containerID="0eeb409ab2834750c5a187afde075d4e1a03e546938652fa6b49173da57bd675"
Jul 22 16:36:30.797451 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:30.797366    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0eeb409ab2834750c5a187afde075d4e1a03e546938652fa6b49173da57bd675"} err="failed to get container status \"0eeb409ab2834750c5a187afde075d4e1a03e546938652fa6b49173da57bd675\": rpc error: code = NotFound desc = could not find container \"0eeb409ab2834750c5a187afde075d4e1a03e546938652fa6b49173da57bd675\": container with ID starting with 0eeb409ab2834750c5a187afde075d4e1a03e546938652fa6b49173da57bd675 not found: ID does not exist"
Jul 22 16:36:30.812661 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:30.809841    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz"]
Jul 22 16:36:30.815255 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:30.815229    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d0a29ed9-57df6c68db-jlwdz"]
Jul 22 16:36:32.448110 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:32.448071    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd5de662-37e6-4582-b018-507128c9d682" path="/var/lib/kubelet/pods/bd5de662-37e6-4582-b018-507128c9d682/volumes"
Jul 22 16:36:34.631740 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:34.631668    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d0a29ed9-zdjtn"]
Jul 22 16:36:34.633384 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:34.633363    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d0a29ed9-zdjtn"]
Jul 22 16:36:34.637990 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:34.637962    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:36:34.791999 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:34.791965    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:36:35.036156 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:35.036121    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:36:35.282069 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:35.282029    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:36:35.372228 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:35.372142    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:36:35.514281 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:35.514246    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:36:35.811894 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:35.811861    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:36:36.297698 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:36.297670    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:36:36.451928 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:36.451891    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71de64bd-db99-4543-8d9b-bfb339775e93" path="/var/lib/kubelet/pods/71de64bd-db99-4543-8d9b-bfb339775e93/volumes"
Jul 22 16:36:38.283451 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:38.283412    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:36:38.340191 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:38.340141    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs"]
Jul 22 16:36:38.340669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:38.340653    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd5de662-37e6-4582-b018-507128c9d682" containerName="maas-api"
Jul 22 16:36:38.340718 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:38.340673    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd5de662-37e6-4582-b018-507128c9d682" containerName="maas-api"
Jul 22 16:36:38.340718 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:38.340707    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71de64bd-db99-4543-8d9b-bfb339775e93" containerName="revoke-keys"
Jul 22 16:36:38.340718 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:38.340716    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="71de64bd-db99-4543-8d9b-bfb339775e93" containerName="revoke-keys"
Jul 22 16:36:38.340811 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:38.340786    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd5de662-37e6-4582-b018-507128c9d682" containerName="maas-api"
Jul 22 16:36:38.340811 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:38.340802    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="71de64bd-db99-4543-8d9b-bfb339775e93" containerName="revoke-keys"
Jul 22 16:36:38.345297 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:38.345275    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs"
Jul 22 16:36:38.351603 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:38.351579    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-e47f5833-serving-cert\""
Jul 22 16:36:38.351730 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:38.351641    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 22 16:36:38.352828 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:38.352809    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 16:36:38.352828 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:38.352816    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vghv2\""
Jul 22 16:36:38.358088 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:38.358058    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs"]
Jul 22 16:36:38.385950 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:38.385916    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:36:38.402169 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:38.402141    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84pxj\" (UniqueName: \"kubernetes.io/projected/11dd2d3c-d155-4cee-80ea-1a0f6a935cae-kube-api-access-84pxj\") pod \"maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs\" (UID: \"11dd2d3c-d155-4cee-80ea-1a0f6a935cae\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs"
Jul 22 16:36:38.402345 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:38.402181    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11dd2d3c-d155-4cee-80ea-1a0f6a935cae-maas-api-tls\") pod \"maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs\" (UID: \"11dd2d3c-d155-4cee-80ea-1a0f6a935cae\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs"
Jul 22 16:36:38.402345 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:38.402212    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11dd2d3c-d155-4cee-80ea-1a0f6a935cae-openshift-service-ca\") pod \"maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs\" (UID: \"11dd2d3c-d155-4cee-80ea-1a0f6a935cae\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs"
Jul 22 16:36:38.503147 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:38.503108    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84pxj\" (UniqueName: \"kubernetes.io/projected/11dd2d3c-d155-4cee-80ea-1a0f6a935cae-kube-api-access-84pxj\") pod \"maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs\" (UID: \"11dd2d3c-d155-4cee-80ea-1a0f6a935cae\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs"
Jul 22 16:36:38.503147 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:38.503152    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11dd2d3c-d155-4cee-80ea-1a0f6a935cae-maas-api-tls\") pod \"maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs\" (UID: \"11dd2d3c-d155-4cee-80ea-1a0f6a935cae\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs"
Jul 22 16:36:38.503407 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:38.503184    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11dd2d3c-d155-4cee-80ea-1a0f6a935cae-openshift-service-ca\") pod \"maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs\" (UID: \"11dd2d3c-d155-4cee-80ea-1a0f6a935cae\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs"
Jul 22 16:36:38.503407 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:36:38.503352    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-e47f5833-serving-cert: secret "maas-api-e2e-migrate-e47f5833-serving-cert" not found
Jul 22 16:36:38.503530 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:36:38.503423    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11dd2d3c-d155-4cee-80ea-1a0f6a935cae-maas-api-tls podName:11dd2d3c-d155-4cee-80ea-1a0f6a935cae nodeName:}" failed. No retries permitted until 2026-07-22 16:36:39.003402274 +0000 UTC m=+1759.133872259 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/11dd2d3c-d155-4cee-80ea-1a0f6a935cae-maas-api-tls") pod "maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs" (UID: "11dd2d3c-d155-4cee-80ea-1a0f6a935cae") : secret "maas-api-e2e-migrate-e47f5833-serving-cert" not found
Jul 22 16:36:38.503906 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:38.503886    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11dd2d3c-d155-4cee-80ea-1a0f6a935cae-openshift-service-ca\") pod \"maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs\" (UID: \"11dd2d3c-d155-4cee-80ea-1a0f6a935cae\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs"
Jul 22 16:36:38.513602 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:38.513572    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84pxj\" (UniqueName: \"kubernetes.io/projected/11dd2d3c-d155-4cee-80ea-1a0f6a935cae-kube-api-access-84pxj\") pod \"maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs\" (UID: \"11dd2d3c-d155-4cee-80ea-1a0f6a935cae\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs"
Jul 22 16:36:38.873998 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:38.873964    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:36:39.008658 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:39.008624    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11dd2d3c-d155-4cee-80ea-1a0f6a935cae-maas-api-tls\") pod \"maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs\" (UID: \"11dd2d3c-d155-4cee-80ea-1a0f6a935cae\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs"
Jul 22 16:36:39.011053 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:39.011031    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11dd2d3c-d155-4cee-80ea-1a0f6a935cae-maas-api-tls\") pod \"maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs\" (UID: \"11dd2d3c-d155-4cee-80ea-1a0f6a935cae\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs"
Jul 22 16:36:39.069884 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:39.069841    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:36:39.171695 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:39.171614    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:36:39.256031 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:39.255991    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs"
Jul 22 16:36:39.392215 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:39.392192    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs"]
Jul 22 16:36:39.394750 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:36:39.394722    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11dd2d3c_d155_4cee_80ea_1a0f6a935cae.slice/crio-5368b6ef8ac386771ef0c5e942ebf4069e5d02e560ace06a604a04246d20acdb WatchSource:0}: Error finding container 5368b6ef8ac386771ef0c5e942ebf4069e5d02e560ace06a604a04246d20acdb: Status 404 returned error can't find the container with id 5368b6ef8ac386771ef0c5e942ebf4069e5d02e560ace06a604a04246d20acdb
Jul 22 16:36:39.826120 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:39.826090    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs" event={"ID":"11dd2d3c-d155-4cee-80ea-1a0f6a935cae","Type":"ContainerStarted","Data":"5368b6ef8ac386771ef0c5e942ebf4069e5d02e560ace06a604a04246d20acdb"}
Jul 22 16:36:40.831618 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:40.831580    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs" event={"ID":"11dd2d3c-d155-4cee-80ea-1a0f6a935cae","Type":"ContainerStarted","Data":"92dfa76a71e269892a6bf5ba0e15d4c69c321abd1e2149dd81d24bf387676761"}
Jul 22 16:36:40.832173 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:40.831704    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs"
Jul 22 16:36:40.851069 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:40.851012    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs" podStartSLOduration=2.468454073 podStartE2EDuration="2.850996619s" totalImagesPullingTime="382.542546ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:36:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:36:40.849894748 +0000 UTC m=+1760.980364752" watchObservedRunningTime="2026-07-22 16:36:40.850996619 +0000 UTC m=+1760.981466620"
Jul 22 16:36:46.841369 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:36:46.841342    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs"
Jul 22 16:37:03.284752 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:03.284715    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e47f5833-csz62"]
Jul 22 16:37:03.288050 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:03.288033    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e47f5833-csz62"
Jul 22 16:37:03.291395 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:03.291378    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-phfdj\""
Jul 22 16:37:03.301356 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:03.301330    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e47f5833-csz62"]
Jul 22 16:37:03.309785 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:03.309755    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5883335b-4a0e-4a00-aaa9-91eef5b1c530-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-e47f5833-csz62\" (UID: \"5883335b-4a0e-4a00-aaa9-91eef5b1c530\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e47f5833-csz62"
Jul 22 16:37:03.410302 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:03.410266    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5883335b-4a0e-4a00-aaa9-91eef5b1c530-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-e47f5833-csz62\" (UID: \"5883335b-4a0e-4a00-aaa9-91eef5b1c530\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e47f5833-csz62"
Jul 22 16:37:03.410974 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:03.410952    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5883335b-4a0e-4a00-aaa9-91eef5b1c530-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-e47f5833-csz62\" (UID: \"5883335b-4a0e-4a00-aaa9-91eef5b1c530\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e47f5833-csz62"
Jul 22 16:37:03.597675 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:03.597581    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e47f5833-csz62"
Jul 22 16:37:03.733743 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:03.733707    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e47f5833-csz62"]
Jul 22 16:37:03.737152 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:37:03.737125    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5883335b_4a0e_4a00_aaa9_91eef5b1c530.slice/crio-3175ae87d961d4bdaeb3b0a826e4484381b3694829678eeb85b4b0c17a014039 WatchSource:0}: Error finding container 3175ae87d961d4bdaeb3b0a826e4484381b3694829678eeb85b4b0c17a014039: Status 404 returned error can't find the container with id 3175ae87d961d4bdaeb3b0a826e4484381b3694829678eeb85b4b0c17a014039
Jul 22 16:37:03.951565 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:03.951533    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e47f5833-csz62" event={"ID":"5883335b-4a0e-4a00-aaa9-91eef5b1c530","Type":"ContainerStarted","Data":"3175ae87d961d4bdaeb3b0a826e4484381b3694829678eeb85b4b0c17a014039"}
Jul 22 16:37:04.957606 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:04.957573    2563 generic.go:356] "Generic (PLEG): container finished" podID="5883335b-4a0e-4a00-aaa9-91eef5b1c530" containerID="8069d9e535ac969dec6ac8b7a0653e360034b1f4a3dd32a63a919296d58c0035" exitCode=0
Jul 22 16:37:04.958030 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:04.957642    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e47f5833-csz62" event={"ID":"5883335b-4a0e-4a00-aaa9-91eef5b1c530","Type":"ContainerDied","Data":"8069d9e535ac969dec6ac8b7a0653e360034b1f4a3dd32a63a919296d58c0035"}
Jul 22 16:37:06.093870 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:06.093827    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e47f5833-csz62"
Jul 22 16:37:06.133615 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:06.133581    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5883335b-4a0e-4a00-aaa9-91eef5b1c530-maas-api-service-ca\") pod \"5883335b-4a0e-4a00-aaa9-91eef5b1c530\" (UID: \"5883335b-4a0e-4a00-aaa9-91eef5b1c530\") "
Jul 22 16:37:06.133980 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:06.133954    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5883335b-4a0e-4a00-aaa9-91eef5b1c530-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5883335b-4a0e-4a00-aaa9-91eef5b1c530" (UID: "5883335b-4a0e-4a00-aaa9-91eef5b1c530"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 16:37:06.235002 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:06.234921    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5883335b-4a0e-4a00-aaa9-91eef5b1c530-maas-api-service-ca\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:37:06.967513 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:06.967486    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e47f5833-csz62"
Jul 22 16:37:06.967719 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:06.967528    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e47f5833-csz62" event={"ID":"5883335b-4a0e-4a00-aaa9-91eef5b1c530","Type":"ContainerDied","Data":"3175ae87d961d4bdaeb3b0a826e4484381b3694829678eeb85b4b0c17a014039"}
Jul 22 16:37:06.967719 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:06.967578    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3175ae87d961d4bdaeb3b0a826e4484381b3694829678eeb85b4b0c17a014039"
Jul 22 16:37:13.814239 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:13.814156    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs"]
Jul 22 16:37:13.814838 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:13.814395    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs" podUID="11dd2d3c-d155-4cee-80ea-1a0f6a935cae" containerName="maas-api" containerID="cri-o://92dfa76a71e269892a6bf5ba0e15d4c69c321abd1e2149dd81d24bf387676761" gracePeriod=30
Jul 22 16:37:13.847433 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:13.847409    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:37:13.999839 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:13.999806    2563 generic.go:356] "Generic (PLEG): container finished" podID="11dd2d3c-d155-4cee-80ea-1a0f6a935cae" containerID="92dfa76a71e269892a6bf5ba0e15d4c69c321abd1e2149dd81d24bf387676761" exitCode=0
Jul 22 16:37:14.000022 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:13.999878    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs" event={"ID":"11dd2d3c-d155-4cee-80ea-1a0f6a935cae","Type":"ContainerDied","Data":"92dfa76a71e269892a6bf5ba0e15d4c69c321abd1e2149dd81d24bf387676761"}
Jul 22 16:37:14.065803 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:14.065740    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs"
Jul 22 16:37:14.099059 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:14.099027    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84pxj\" (UniqueName: \"kubernetes.io/projected/11dd2d3c-d155-4cee-80ea-1a0f6a935cae-kube-api-access-84pxj\") pod \"11dd2d3c-d155-4cee-80ea-1a0f6a935cae\" (UID: \"11dd2d3c-d155-4cee-80ea-1a0f6a935cae\") "
Jul 22 16:37:14.099226 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:14.099065    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11dd2d3c-d155-4cee-80ea-1a0f6a935cae-openshift-service-ca\") pod \"11dd2d3c-d155-4cee-80ea-1a0f6a935cae\" (UID: \"11dd2d3c-d155-4cee-80ea-1a0f6a935cae\") "
Jul 22 16:37:14.099226 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:14.099103    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11dd2d3c-d155-4cee-80ea-1a0f6a935cae-maas-api-tls\") pod \"11dd2d3c-d155-4cee-80ea-1a0f6a935cae\" (UID: \"11dd2d3c-d155-4cee-80ea-1a0f6a935cae\") "
Jul 22 16:37:14.099478 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:14.099449    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11dd2d3c-d155-4cee-80ea-1a0f6a935cae-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "11dd2d3c-d155-4cee-80ea-1a0f6a935cae" (UID: "11dd2d3c-d155-4cee-80ea-1a0f6a935cae"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 16:37:14.101187 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:14.101156    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11dd2d3c-d155-4cee-80ea-1a0f6a935cae-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "11dd2d3c-d155-4cee-80ea-1a0f6a935cae" (UID: "11dd2d3c-d155-4cee-80ea-1a0f6a935cae"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:37:14.101316 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:14.101294    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11dd2d3c-d155-4cee-80ea-1a0f6a935cae-kube-api-access-84pxj" (OuterVolumeSpecName: "kube-api-access-84pxj") pod "11dd2d3c-d155-4cee-80ea-1a0f6a935cae" (UID: "11dd2d3c-d155-4cee-80ea-1a0f6a935cae"). InnerVolumeSpecName "kube-api-access-84pxj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:37:14.200061 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:14.200020    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84pxj\" (UniqueName: \"kubernetes.io/projected/11dd2d3c-d155-4cee-80ea-1a0f6a935cae-kube-api-access-84pxj\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:37:14.200061 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:14.200054    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11dd2d3c-d155-4cee-80ea-1a0f6a935cae-openshift-service-ca\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:37:14.200061 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:14.200065    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11dd2d3c-d155-4cee-80ea-1a0f6a935cae-maas-api-tls\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:37:15.005207 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:15.005178    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs"
Jul 22 16:37:15.005207 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:15.005197    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs" event={"ID":"11dd2d3c-d155-4cee-80ea-1a0f6a935cae","Type":"ContainerDied","Data":"5368b6ef8ac386771ef0c5e942ebf4069e5d02e560ace06a604a04246d20acdb"}
Jul 22 16:37:15.005688 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:15.005262    2563 scope.go:117] "RemoveContainer" containerID="92dfa76a71e269892a6bf5ba0e15d4c69c321abd1e2149dd81d24bf387676761"
Jul 22 16:37:15.023781 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:15.023753    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs"]
Jul 22 16:37:15.027132 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:15.027109    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e47f5833-66d4dd8fc6-fgcfs"]
Jul 22 16:37:16.447742 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:16.447706    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11dd2d3c-d155-4cee-80ea-1a0f6a935cae" path="/var/lib/kubelet/pods/11dd2d3c-d155-4cee-80ea-1a0f6a935cae/volumes"
Jul 22 16:37:18.356392 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:18.356356    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:37:18.360119 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:18.360090    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e47f5833-csz62"]
Jul 22 16:37:18.363206 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:18.363179    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e47f5833-csz62"]
Jul 22 16:37:18.448105 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:18.448073    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5883335b-4a0e-4a00-aaa9-91eef5b1c530" path="/var/lib/kubelet/pods/5883335b-4a0e-4a00-aaa9-91eef5b1c530/volumes"
Jul 22 16:37:18.526675 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:18.526625    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:37:18.791140 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:18.791109    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:37:19.074836 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:19.074757    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:37:19.450488 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:19.450458    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:37:19.572149 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:19.572116    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:37:21.540732 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:21.540692    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:37:21.659684 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:21.659651    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:37:21.668286 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:21.668257    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8"]
Jul 22 16:37:21.668584 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:21.668571    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11dd2d3c-d155-4cee-80ea-1a0f6a935cae" containerName="maas-api"
Jul 22 16:37:21.668644 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:21.668585    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="11dd2d3c-d155-4cee-80ea-1a0f6a935cae" containerName="maas-api"
Jul 22 16:37:21.668644 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:21.668602    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5883335b-4a0e-4a00-aaa9-91eef5b1c530" containerName="revoke-keys"
Jul 22 16:37:21.668644 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:21.668607    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5883335b-4a0e-4a00-aaa9-91eef5b1c530" containerName="revoke-keys"
Jul 22 16:37:21.668801 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:21.668653    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="11dd2d3c-d155-4cee-80ea-1a0f6a935cae" containerName="maas-api"
Jul 22 16:37:21.668801 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:21.668664    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5883335b-4a0e-4a00-aaa9-91eef5b1c530" containerName="revoke-keys"
Jul 22 16:37:21.671448 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:21.671433    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8"
Jul 22 16:37:21.676554 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:21.676529    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 16:37:21.676671 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:21.676604    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vghv2\""
Jul 22 16:37:21.678975 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:21.678955    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-4c7e4f77-serving-cert\""
Jul 22 16:37:21.679091 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:21.678957    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 22 16:37:21.685750 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:21.685725    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8"]
Jul 22 16:37:21.764021 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:21.763975    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ae5b763-7641-44d9-bab3-ba620a92c5e6-openshift-service-ca\") pod \"maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8\" (UID: \"5ae5b763-7641-44d9-bab3-ba620a92c5e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8"
Jul 22 16:37:21.764207 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:21.764108    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ae5b763-7641-44d9-bab3-ba620a92c5e6-maas-api-tls\") pod \"maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8\" (UID: \"5ae5b763-7641-44d9-bab3-ba620a92c5e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8"
Jul 22 16:37:21.764207 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:21.764136    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrkph\" (UniqueName: \"kubernetes.io/projected/5ae5b763-7641-44d9-bab3-ba620a92c5e6-kube-api-access-nrkph\") pod \"maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8\" (UID: \"5ae5b763-7641-44d9-bab3-ba620a92c5e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8"
Jul 22 16:37:21.769679 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:21.769607    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:37:21.865278 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:21.865173    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ae5b763-7641-44d9-bab3-ba620a92c5e6-openshift-service-ca\") pod \"maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8\" (UID: \"5ae5b763-7641-44d9-bab3-ba620a92c5e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8"
Jul 22 16:37:21.865278 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:21.865261    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ae5b763-7641-44d9-bab3-ba620a92c5e6-maas-api-tls\") pod \"maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8\" (UID: \"5ae5b763-7641-44d9-bab3-ba620a92c5e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8"
Jul 22 16:37:21.865516 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:21.865292    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrkph\" (UniqueName: \"kubernetes.io/projected/5ae5b763-7641-44d9-bab3-ba620a92c5e6-kube-api-access-nrkph\") pod \"maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8\" (UID: \"5ae5b763-7641-44d9-bab3-ba620a92c5e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8"
Jul 22 16:37:21.865516 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:37:21.865429    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-4c7e4f77-serving-cert: secret "maas-api-e2e-ait-4c7e4f77-serving-cert" not found
Jul 22 16:37:21.865615 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:37:21.865520    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ae5b763-7641-44d9-bab3-ba620a92c5e6-maas-api-tls podName:5ae5b763-7641-44d9-bab3-ba620a92c5e6 nodeName:}" failed. No retries permitted until 2026-07-22 16:37:22.365496109 +0000 UTC m=+1802.495966096 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5ae5b763-7641-44d9-bab3-ba620a92c5e6-maas-api-tls") pod "maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8" (UID: "5ae5b763-7641-44d9-bab3-ba620a92c5e6") : secret "maas-api-e2e-ait-4c7e4f77-serving-cert" not found
Jul 22 16:37:21.865834 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:21.865816    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ae5b763-7641-44d9-bab3-ba620a92c5e6-openshift-service-ca\") pod \"maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8\" (UID: \"5ae5b763-7641-44d9-bab3-ba620a92c5e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8"
Jul 22 16:37:21.876909 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:21.876876    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrkph\" (UniqueName: \"kubernetes.io/projected/5ae5b763-7641-44d9-bab3-ba620a92c5e6-kube-api-access-nrkph\") pod \"maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8\" (UID: \"5ae5b763-7641-44d9-bab3-ba620a92c5e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8"
Jul 22 16:37:22.369783 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:22.369748    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ae5b763-7641-44d9-bab3-ba620a92c5e6-maas-api-tls\") pod \"maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8\" (UID: \"5ae5b763-7641-44d9-bab3-ba620a92c5e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8"
Jul 22 16:37:22.372247 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:22.372217    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ae5b763-7641-44d9-bab3-ba620a92c5e6-maas-api-tls\") pod \"maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8\" (UID: \"5ae5b763-7641-44d9-bab3-ba620a92c5e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8"
Jul 22 16:37:22.582945 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:22.582907    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8"
Jul 22 16:37:22.704213 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:22.704180    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:37:22.735883 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:22.735832    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8"]
Jul 22 16:37:22.737760 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:37:22.737732    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ae5b763_7641_44d9_bab3_ba620a92c5e6.slice/crio-b658ed363b1ee66e78ab44de26b7f8c34abb0fcdcf20ab5a9349ddf853ac76ef WatchSource:0}: Error finding container b658ed363b1ee66e78ab44de26b7f8c34abb0fcdcf20ab5a9349ddf853ac76ef: Status 404 returned error can't find the container with id b658ed363b1ee66e78ab44de26b7f8c34abb0fcdcf20ab5a9349ddf853ac76ef
Jul 22 16:37:22.844192 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:22.844153    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:37:23.043426 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:23.043388    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8" event={"ID":"5ae5b763-7641-44d9-bab3-ba620a92c5e6","Type":"ContainerStarted","Data":"b658ed363b1ee66e78ab44de26b7f8c34abb0fcdcf20ab5a9349ddf853ac76ef"}
Jul 22 16:37:24.049504 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:24.049450    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8" event={"ID":"5ae5b763-7641-44d9-bab3-ba620a92c5e6","Type":"ContainerStarted","Data":"bdcfd0a6639c55616d406fb5d7db139a8f49ec4d743f9c5a5df6b100e169cd18"}
Jul 22 16:37:24.049937 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:24.049563    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8"
Jul 22 16:37:24.071694 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:24.071642    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8" podStartSLOduration=2.675720392 podStartE2EDuration="3.071627561s" totalImagesPullingTime="395.907169ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:37:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:37:24.070032086 +0000 UTC m=+1804.200502087" watchObservedRunningTime="2026-07-22 16:37:24.071627561 +0000 UTC m=+1804.202097562"
Jul 22 16:37:30.060341 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:30.060315    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8"
Jul 22 16:37:47.337362 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:47.337325    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4c7e4f77-456f6"]
Jul 22 16:37:47.341361 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:47.341342    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4c7e4f77-456f6"
Jul 22 16:37:47.344360 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:47.344343    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-phfdj\""
Jul 22 16:37:47.349380 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:47.349350    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4c7e4f77-456f6"]
Jul 22 16:37:47.489532 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:47.489489    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e5ff946-59a8-4dad-bb9a-45cfb912c8a6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4c7e4f77-456f6\" (UID: \"2e5ff946-59a8-4dad-bb9a-45cfb912c8a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4c7e4f77-456f6"
Jul 22 16:37:47.591006 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:47.590905    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e5ff946-59a8-4dad-bb9a-45cfb912c8a6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4c7e4f77-456f6\" (UID: \"2e5ff946-59a8-4dad-bb9a-45cfb912c8a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4c7e4f77-456f6"
Jul 22 16:37:47.591679 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:47.591655    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e5ff946-59a8-4dad-bb9a-45cfb912c8a6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4c7e4f77-456f6\" (UID: \"2e5ff946-59a8-4dad-bb9a-45cfb912c8a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4c7e4f77-456f6"
Jul 22 16:37:47.652978 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:47.652939    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4c7e4f77-456f6"
Jul 22 16:37:47.800294 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:47.800269    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4c7e4f77-456f6"]
Jul 22 16:37:47.802210 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:37:47.802184    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e5ff946_59a8_4dad_bb9a_45cfb912c8a6.slice/crio-12db3c57f46f50f01064d3ec1675c7756bb112de62732fe37248b37eb8d1b999 WatchSource:0}: Error finding container 12db3c57f46f50f01064d3ec1675c7756bb112de62732fe37248b37eb8d1b999: Status 404 returned error can't find the container with id 12db3c57f46f50f01064d3ec1675c7756bb112de62732fe37248b37eb8d1b999
Jul 22 16:37:48.160678 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:48.160637    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4c7e4f77-456f6" event={"ID":"2e5ff946-59a8-4dad-bb9a-45cfb912c8a6","Type":"ContainerStarted","Data":"7380c2973103bcda77b6ecddb7bee000a3d58065580d71f3442f43431393c2e1"}
Jul 22 16:37:48.160874 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:48.160720    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4c7e4f77-456f6" event={"ID":"2e5ff946-59a8-4dad-bb9a-45cfb912c8a6","Type":"ContainerStarted","Data":"12db3c57f46f50f01064d3ec1675c7756bb112de62732fe37248b37eb8d1b999"}
Jul 22 16:37:49.106604 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:49.106573    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:37:49.166080 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:49.166046    2563 generic.go:356] "Generic (PLEG): container finished" podID="2e5ff946-59a8-4dad-bb9a-45cfb912c8a6" containerID="7380c2973103bcda77b6ecddb7bee000a3d58065580d71f3442f43431393c2e1" exitCode=0
Jul 22 16:37:49.166266 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:49.166121    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4c7e4f77-456f6" event={"ID":"2e5ff946-59a8-4dad-bb9a-45cfb912c8a6","Type":"ContainerDied","Data":"7380c2973103bcda77b6ecddb7bee000a3d58065580d71f3442f43431393c2e1"}
Jul 22 16:37:49.189674 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:49.189641    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:37:50.298746 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:50.298716    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4c7e4f77-456f6"
Jul 22 16:37:50.417303 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:50.417269    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e5ff946-59a8-4dad-bb9a-45cfb912c8a6-maas-api-service-ca\") pod \"2e5ff946-59a8-4dad-bb9a-45cfb912c8a6\" (UID: \"2e5ff946-59a8-4dad-bb9a-45cfb912c8a6\") "
Jul 22 16:37:50.417609 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:50.417585    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e5ff946-59a8-4dad-bb9a-45cfb912c8a6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2e5ff946-59a8-4dad-bb9a-45cfb912c8a6" (UID: "2e5ff946-59a8-4dad-bb9a-45cfb912c8a6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 16:37:50.518733 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:50.518699    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e5ff946-59a8-4dad-bb9a-45cfb912c8a6-maas-api-service-ca\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:37:51.176091 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:51.176056    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4c7e4f77-456f6" event={"ID":"2e5ff946-59a8-4dad-bb9a-45cfb912c8a6","Type":"ContainerDied","Data":"12db3c57f46f50f01064d3ec1675c7756bb112de62732fe37248b37eb8d1b999"}
Jul 22 16:37:51.176091 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:51.176086    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4c7e4f77-456f6"
Jul 22 16:37:51.176299 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:51.176106    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="12db3c57f46f50f01064d3ec1675c7756bb112de62732fe37248b37eb8d1b999"
Jul 22 16:37:57.870589 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:57.870557    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:37:57.976223 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:57.976191    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:37:58.106307 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:58.106272    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8"]
Jul 22 16:37:58.106509 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:58.106483    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8" podUID="5ae5b763-7641-44d9-bab3-ba620a92c5e6" containerName="maas-api" containerID="cri-o://bdcfd0a6639c55616d406fb5d7db139a8f49ec4d743f9c5a5df6b100e169cd18" gracePeriod=30
Jul 22 16:37:58.135248 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:58.135181    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:37:58.356278 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:58.356253    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8"
Jul 22 16:37:58.382054 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:58.382025    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ae5b763-7641-44d9-bab3-ba620a92c5e6-maas-api-tls\") pod \"5ae5b763-7641-44d9-bab3-ba620a92c5e6\" (UID: \"5ae5b763-7641-44d9-bab3-ba620a92c5e6\") "
Jul 22 16:37:58.382216 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:58.382090    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ae5b763-7641-44d9-bab3-ba620a92c5e6-openshift-service-ca\") pod \"5ae5b763-7641-44d9-bab3-ba620a92c5e6\" (UID: \"5ae5b763-7641-44d9-bab3-ba620a92c5e6\") "
Jul 22 16:37:58.382216 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:58.382114    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrkph\" (UniqueName: \"kubernetes.io/projected/5ae5b763-7641-44d9-bab3-ba620a92c5e6-kube-api-access-nrkph\") pod \"5ae5b763-7641-44d9-bab3-ba620a92c5e6\" (UID: \"5ae5b763-7641-44d9-bab3-ba620a92c5e6\") "
Jul 22 16:37:58.382475 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:58.382448    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ae5b763-7641-44d9-bab3-ba620a92c5e6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5ae5b763-7641-44d9-bab3-ba620a92c5e6" (UID: "5ae5b763-7641-44d9-bab3-ba620a92c5e6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 16:37:58.384083 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:58.384054    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ae5b763-7641-44d9-bab3-ba620a92c5e6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5ae5b763-7641-44d9-bab3-ba620a92c5e6" (UID: "5ae5b763-7641-44d9-bab3-ba620a92c5e6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:37:58.384203 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:58.384184    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ae5b763-7641-44d9-bab3-ba620a92c5e6-kube-api-access-nrkph" (OuterVolumeSpecName: "kube-api-access-nrkph") pod "5ae5b763-7641-44d9-bab3-ba620a92c5e6" (UID: "5ae5b763-7641-44d9-bab3-ba620a92c5e6"). InnerVolumeSpecName "kube-api-access-nrkph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:37:58.483347 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:58.483317    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ae5b763-7641-44d9-bab3-ba620a92c5e6-maas-api-tls\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:37:58.483347 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:58.483341    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ae5b763-7641-44d9-bab3-ba620a92c5e6-openshift-service-ca\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:37:58.483347 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:58.483351    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrkph\" (UniqueName: \"kubernetes.io/projected/5ae5b763-7641-44d9-bab3-ba620a92c5e6-kube-api-access-nrkph\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:37:59.214224 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:59.214187    2563 generic.go:356] "Generic (PLEG): container finished" podID="5ae5b763-7641-44d9-bab3-ba620a92c5e6" containerID="bdcfd0a6639c55616d406fb5d7db139a8f49ec4d743f9c5a5df6b100e169cd18" exitCode=0
Jul 22 16:37:59.214734 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:59.214253    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8"
Jul 22 16:37:59.214734 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:59.214278    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8" event={"ID":"5ae5b763-7641-44d9-bab3-ba620a92c5e6","Type":"ContainerDied","Data":"bdcfd0a6639c55616d406fb5d7db139a8f49ec4d743f9c5a5df6b100e169cd18"}
Jul 22 16:37:59.214734 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:59.214342    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8" event={"ID":"5ae5b763-7641-44d9-bab3-ba620a92c5e6","Type":"ContainerDied","Data":"b658ed363b1ee66e78ab44de26b7f8c34abb0fcdcf20ab5a9349ddf853ac76ef"}
Jul 22 16:37:59.214734 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:59.214376    2563 scope.go:117] "RemoveContainer" containerID="bdcfd0a6639c55616d406fb5d7db139a8f49ec4d743f9c5a5df6b100e169cd18"
Jul 22 16:37:59.222651 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:59.222631    2563 scope.go:117] "RemoveContainer" containerID="bdcfd0a6639c55616d406fb5d7db139a8f49ec4d743f9c5a5df6b100e169cd18"
Jul 22 16:37:59.222950 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:37:59.222926    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdcfd0a6639c55616d406fb5d7db139a8f49ec4d743f9c5a5df6b100e169cd18\": container with ID starting with bdcfd0a6639c55616d406fb5d7db139a8f49ec4d743f9c5a5df6b100e169cd18 not found: ID does not exist" containerID="bdcfd0a6639c55616d406fb5d7db139a8f49ec4d743f9c5a5df6b100e169cd18"
Jul 22 16:37:59.223030 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:59.222962    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdcfd0a6639c55616d406fb5d7db139a8f49ec4d743f9c5a5df6b100e169cd18"} err="failed to get container status \"bdcfd0a6639c55616d406fb5d7db139a8f49ec4d743f9c5a5df6b100e169cd18\": rpc error: code = NotFound desc = could not find container \"bdcfd0a6639c55616d406fb5d7db139a8f49ec4d743f9c5a5df6b100e169cd18\": container with ID starting with bdcfd0a6639c55616d406fb5d7db139a8f49ec4d743f9c5a5df6b100e169cd18 not found: ID does not exist"
Jul 22 16:37:59.232879 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:59.232842    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8"]
Jul 22 16:37:59.239601 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:37:59.239575    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4c7e4f77-68d657b985-fknd8"]
Jul 22 16:38:00.448077 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:00.448041    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ae5b763-7641-44d9-bab3-ba620a92c5e6" path="/var/lib/kubelet/pods/5ae5b763-7641-44d9-bab3-ba620a92c5e6/volumes"
Jul 22 16:38:02.419806 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:02.419772    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4c7e4f77-456f6"]
Jul 22 16:38:02.422480 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:02.422443    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4c7e4f77-456f6"]
Jul 22 16:38:02.422920 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:02.422891    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:38:02.448034 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:02.447991    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e5ff946-59a8-4dad-bb9a-45cfb912c8a6" path="/var/lib/kubelet/pods/2e5ff946-59a8-4dad-bb9a-45cfb912c8a6/volumes"
Jul 22 16:38:03.672112 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:03.672070    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:38:03.881779 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:03.881746    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:38:04.120697 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.120659    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:38:04.422867 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.422771    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"]
Jul 22 16:38:04.423157 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.423144    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e5ff946-59a8-4dad-bb9a-45cfb912c8a6" containerName="revoke-keys"
Jul 22 16:38:04.423201 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.423159    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e5ff946-59a8-4dad-bb9a-45cfb912c8a6" containerName="revoke-keys"
Jul 22 16:38:04.423201 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.423185    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ae5b763-7641-44d9-bab3-ba620a92c5e6" containerName="maas-api"
Jul 22 16:38:04.423201 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.423194    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ae5b763-7641-44d9-bab3-ba620a92c5e6" containerName="maas-api"
Jul 22 16:38:04.423297 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.423246    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ae5b763-7641-44d9-bab3-ba620a92c5e6" containerName="maas-api"
Jul 22 16:38:04.423297 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.423254    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e5ff946-59a8-4dad-bb9a-45cfb912c8a6" containerName="revoke-keys"
Jul 22 16:38:04.427717 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.427700    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:04.430779 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.430756    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 22 16:38:04.431234 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.431217    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-7d60fbcd-openshift-default-dockercfg-6kvs5\""
Jul 22 16:38:04.437785 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.437764    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"]
Jul 22 16:38:04.445606 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.445582    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:38:04.534062 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.534026    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de8da499-68de-4a31-a34a-167297a1a982-istio-podinfo\") pod \"e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") " pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:04.534256 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.534100    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de8da499-68de-4a31-a34a-167297a1a982-istio-data\") pod \"e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") " pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:04.534256 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.534150    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vg8lx\" (UniqueName: \"kubernetes.io/projected/de8da499-68de-4a31-a34a-167297a1a982-kube-api-access-vg8lx\") pod \"e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") " pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:04.534256 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.534205    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de8da499-68de-4a31-a34a-167297a1a982-workload-socket\") pod \"e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") " pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:04.534256 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.534230    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de8da499-68de-4a31-a34a-167297a1a982-istio-envoy\") pod \"e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") " pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:04.534256 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.534252    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de8da499-68de-4a31-a34a-167297a1a982-workload-certs\") pod \"e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") " pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:04.534492 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.534308    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de8da499-68de-4a31-a34a-167297a1a982-istiod-ca-cert\") pod \"e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") " pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:04.534492 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.534343    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de8da499-68de-4a31-a34a-167297a1a982-credential-socket\") pod \"e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") " pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:04.534492 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.534374    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de8da499-68de-4a31-a34a-167297a1a982-istio-token\") pod \"e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") " pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:04.635363 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.635319    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de8da499-68de-4a31-a34a-167297a1a982-istio-envoy\") pod \"e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") " pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:04.635363 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.635367    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de8da499-68de-4a31-a34a-167297a1a982-workload-certs\") pod \"e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") " pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:04.635630 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.635392    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de8da499-68de-4a31-a34a-167297a1a982-istiod-ca-cert\") pod \"e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") " pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:04.635630 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.635416    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de8da499-68de-4a31-a34a-167297a1a982-credential-socket\") pod \"e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") " pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:04.635630 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.635434    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de8da499-68de-4a31-a34a-167297a1a982-istio-token\") pod \"e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") " pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:04.635630 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.635458    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de8da499-68de-4a31-a34a-167297a1a982-istio-podinfo\") pod \"e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") " pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:04.635630 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.635497    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de8da499-68de-4a31-a34a-167297a1a982-istio-data\") pod \"e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") " pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:04.635630 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.635529    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vg8lx\" (UniqueName: \"kubernetes.io/projected/de8da499-68de-4a31-a34a-167297a1a982-kube-api-access-vg8lx\") pod \"e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") " pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:04.635630 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.635571    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de8da499-68de-4a31-a34a-167297a1a982-workload-socket\") pod \"e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") " pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:04.636039 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.635877    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de8da499-68de-4a31-a34a-167297a1a982-credential-socket\") pod \"e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") " pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:04.636039 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.635847    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de8da499-68de-4a31-a34a-167297a1a982-workload-certs\") pod \"e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") " pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:04.636435 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.636410    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de8da499-68de-4a31-a34a-167297a1a982-istiod-ca-cert\") pod \"e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") " pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:04.636597 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.636577    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de8da499-68de-4a31-a34a-167297a1a982-workload-socket\") pod \"e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") " pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:04.636671 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.636641    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de8da499-68de-4a31-a34a-167297a1a982-istio-data\") pod \"e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") " pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:04.637797 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.637778    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de8da499-68de-4a31-a34a-167297a1a982-istio-envoy\") pod \"e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") " pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:04.638057 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.638038    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de8da499-68de-4a31-a34a-167297a1a982-istio-podinfo\") pod \"e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") " pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:04.645495 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.645469    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de8da499-68de-4a31-a34a-167297a1a982-istio-token\") pod \"e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") " pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:04.645564 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.645505    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vg8lx\" (UniqueName: \"kubernetes.io/projected/de8da499-68de-4a31-a34a-167297a1a982-kube-api-access-vg8lx\") pod \"e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") " pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:04.738868 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.738832    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:04.786151 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.786117    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:38:04.875301 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:04.875275    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"]
Jul 22 16:38:04.878158 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:38:04.878130    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde8da499_68de_4a31_a34a_167297a1a982.slice/crio-ff9cdeedc4f2b16dad1d1aec9b2994243354f139585255db76e3b1c869462866 WatchSource:0}: Error finding container ff9cdeedc4f2b16dad1d1aec9b2994243354f139585255db76e3b1c869462866: Status 404 returned error can't find the container with id ff9cdeedc4f2b16dad1d1aec9b2994243354f139585255db76e3b1c869462866
Jul 22 16:38:05.243058 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:05.243021    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz" event={"ID":"de8da499-68de-4a31-a34a-167297a1a982","Type":"ContainerStarted","Data":"ff9cdeedc4f2b16dad1d1aec9b2994243354f139585255db76e3b1c869462866"}
Jul 22 16:38:08.313045 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:08.313009    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:38:08.313319 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:08.313103    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:38:08.313319 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:08.313135    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:38:09.265204 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:09.265168    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz" event={"ID":"de8da499-68de-4a31-a34a-167297a1a982","Type":"ContainerStarted","Data":"cc2572d54be8dedf2097a635d716204d61f8e0ef7ad98cf34d41dccc9e980d31"}
Jul 22 16:38:09.288367 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:09.288237    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz" podStartSLOduration=1.85618674 podStartE2EDuration="5.288218548s" totalImagesPullingTime="3.432031808s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:38:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:38:09.287187575 +0000 UTC m=+1849.417657588" watchObservedRunningTime="2026-07-22 16:38:09.288218548 +0000 UTC m=+1849.418688551"
Jul 22 16:38:09.739733 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:09.739699    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:09.745483 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:09.745452    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:10.270630 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:10.270594    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:10.272742 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:10.271473    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:10.636935 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:10.636809    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:38:10.715095 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:10.715061    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb"]
Jul 22 16:38:10.718687 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:10.718668    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb"
Jul 22 16:38:10.723060 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:10.723034    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 16:38:10.723200 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:10.723068    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vghv2\""
Jul 22 16:38:10.723200 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:10.723105    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 22 16:38:10.723200 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:10.723048    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-7d60fbcd-serving-cert\""
Jul 22 16:38:10.731035 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:10.731009    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb"]
Jul 22 16:38:10.731928 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:10.731900    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:38:10.794103 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:10.794064    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9fe0162a-0801-423d-8fb4-b3d9774b31a1-maas-api-tls\") pod \"maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb\" (UID: \"9fe0162a-0801-423d-8fb4-b3d9774b31a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb"
Jul 22 16:38:10.794103 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:10.794120    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ztwp\" (UniqueName: \"kubernetes.io/projected/9fe0162a-0801-423d-8fb4-b3d9774b31a1-kube-api-access-9ztwp\") pod \"maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb\" (UID: \"9fe0162a-0801-423d-8fb4-b3d9774b31a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb"
Jul 22 16:38:10.794552 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:10.794195    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fe0162a-0801-423d-8fb4-b3d9774b31a1-openshift-service-ca\") pod \"maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb\" (UID: \"9fe0162a-0801-423d-8fb4-b3d9774b31a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb"
Jul 22 16:38:10.823254 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:10.823219    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:38:10.895729 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:10.895625    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ztwp\" (UniqueName: \"kubernetes.io/projected/9fe0162a-0801-423d-8fb4-b3d9774b31a1-kube-api-access-9ztwp\") pod \"maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb\" (UID: \"9fe0162a-0801-423d-8fb4-b3d9774b31a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb"
Jul 22 16:38:10.895729 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:10.895680    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fe0162a-0801-423d-8fb4-b3d9774b31a1-openshift-service-ca\") pod \"maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb\" (UID: \"9fe0162a-0801-423d-8fb4-b3d9774b31a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb"
Jul 22 16:38:10.896015 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:10.895743    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9fe0162a-0801-423d-8fb4-b3d9774b31a1-maas-api-tls\") pod \"maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb\" (UID: \"9fe0162a-0801-423d-8fb4-b3d9774b31a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb"
Jul 22 16:38:10.896490 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:10.896468    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fe0162a-0801-423d-8fb4-b3d9774b31a1-openshift-service-ca\") pod \"maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb\" (UID: \"9fe0162a-0801-423d-8fb4-b3d9774b31a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb"
Jul 22 16:38:10.898247 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:10.898222    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9fe0162a-0801-423d-8fb4-b3d9774b31a1-maas-api-tls\") pod \"maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb\" (UID: \"9fe0162a-0801-423d-8fb4-b3d9774b31a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb"
Jul 22 16:38:10.905591 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:10.905563    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ztwp\" (UniqueName: \"kubernetes.io/projected/9fe0162a-0801-423d-8fb4-b3d9774b31a1-kube-api-access-9ztwp\") pod \"maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb\" (UID: \"9fe0162a-0801-423d-8fb4-b3d9774b31a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb"
Jul 22 16:38:10.929326 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:10.929286    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:38:11.030570 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:11.030532    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb"
Jul 22 16:38:11.166378 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:11.166206    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb"]
Jul 22 16:38:11.169111 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:38:11.169083    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fe0162a_0801_423d_8fb4_b3d9774b31a1.slice/crio-20758acdb677e7aef9866c23019f84ae6897607b3b96b9b44ef427ca532613b4 WatchSource:0}: Error finding container 20758acdb677e7aef9866c23019f84ae6897607b3b96b9b44ef427ca532613b4: Status 404 returned error can't find the container with id 20758acdb677e7aef9866c23019f84ae6897607b3b96b9b44ef427ca532613b4
Jul 22 16:38:11.276313 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:11.276276    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb" event={"ID":"9fe0162a-0801-423d-8fb4-b3d9774b31a1","Type":"ContainerStarted","Data":"20758acdb677e7aef9866c23019f84ae6897607b3b96b9b44ef427ca532613b4"}
Jul 22 16:38:11.420444 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:11.420359    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:38:11.527949 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:11.527909    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:38:12.282636 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:12.282599    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb" event={"ID":"9fe0162a-0801-423d-8fb4-b3d9774b31a1","Type":"ContainerStarted","Data":"b27e5ae746decbe5790551e3ad0ff09b8dbbdd36da27e4c39c7592905a737242"}
Jul 22 16:38:12.283060 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:12.282667    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb"
Jul 22 16:38:12.302322 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:12.302276    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb" podStartSLOduration=1.93432209 podStartE2EDuration="2.302261368s" totalImagesPullingTime="367.939278ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:38:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:38:12.300683422 +0000 UTC m=+1852.431153423" watchObservedRunningTime="2026-07-22 16:38:12.302261368 +0000 UTC m=+1852.432731370"
Jul 22 16:38:18.293060 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:18.293028    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb"
Jul 22 16:38:24.358401 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:24.358299    2563 generic.go:356] "Generic (PLEG): container finished" podID="de8da499-68de-4a31-a34a-167297a1a982" containerID="cc2572d54be8dedf2097a635d716204d61f8e0ef7ad98cf34d41dccc9e980d31" exitCode=0
Jul 22 16:38:24.358401 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:24.358371    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz" event={"ID":"de8da499-68de-4a31-a34a-167297a1a982","Type":"ContainerDied","Data":"cc2572d54be8dedf2097a635d716204d61f8e0ef7ad98cf34d41dccc9e980d31"}
Jul 22 16:38:24.358827 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:24.358756    2563 scope.go:117] "RemoveContainer" containerID="cc2572d54be8dedf2097a635d716204d61f8e0ef7ad98cf34d41dccc9e980d31"
Jul 22 16:38:24.359680 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:24.359649    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:38:24.359732 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:24.359721    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:38:24.359785 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:24.359774    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:38:25.271153 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:25.271098    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:25.364710 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:25.364669    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz" event={"ID":"de8da499-68de-4a31-a34a-167297a1a982","Type":"ContainerStarted","Data":"45b4525d0cd53e28cedcb602a2dd5a8dbac517890417913312ecac5f86c917eb"}
Jul 22 16:38:25.739677 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:25.739637    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:25.740723 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:25.740696    2563 patch_prober.go:28] interesting pod/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Jul 22 16:38:25.740842 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:25.740757    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz" podUID="de8da499-68de-4a31-a34a-167297a1a982" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Jul 22 16:38:26.740291 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:26.740251    2563 patch_prober.go:28] interesting pod/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Jul 22 16:38:26.740687 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:26.740311    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz" podUID="de8da499-68de-4a31-a34a-167297a1a982" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Jul 22 16:38:27.740242 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:27.740202    2563 patch_prober.go:28] interesting pod/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Jul 22 16:38:27.740464 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:27.740268    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz" podUID="de8da499-68de-4a31-a34a-167297a1a982" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Jul 22 16:38:28.739359 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:28.739325    2563 patch_prober.go:28] interesting pod/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Jul 22 16:38:28.739540 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:28.739390    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz" podUID="de8da499-68de-4a31-a34a-167297a1a982" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Jul 22 16:38:28.999268 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:28.999182    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:38:29.055517 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:29.055479    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:38:29.743516 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:29.743485    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:29.743883 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:29.743847    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:29.744551 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:29.744529    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:35.530044 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:35.530007    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7d60fbcd-q2qc6"]
Jul 22 16:38:35.534610 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:35.534591    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7d60fbcd-q2qc6"
Jul 22 16:38:35.537534 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:35.537509    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-phfdj\""
Jul 22 16:38:35.540718 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:35.540687    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7d60fbcd-q2qc6"]
Jul 22 16:38:35.620304 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:35.620265    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6efba5b8-431e-4f6a-8a3e-e6c520ea8107-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-7d60fbcd-q2qc6\" (UID: \"6efba5b8-431e-4f6a-8a3e-e6c520ea8107\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7d60fbcd-q2qc6"
Jul 22 16:38:35.721108 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:35.721073    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6efba5b8-431e-4f6a-8a3e-e6c520ea8107-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-7d60fbcd-q2qc6\" (UID: \"6efba5b8-431e-4f6a-8a3e-e6c520ea8107\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7d60fbcd-q2qc6"
Jul 22 16:38:35.721689 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:35.721670    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6efba5b8-431e-4f6a-8a3e-e6c520ea8107-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-7d60fbcd-q2qc6\" (UID: \"6efba5b8-431e-4f6a-8a3e-e6c520ea8107\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7d60fbcd-q2qc6"
Jul 22 16:38:35.848821 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:35.848734    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7d60fbcd-q2qc6"
Jul 22 16:38:35.979503 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:35.979470    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7d60fbcd-q2qc6"]
Jul 22 16:38:35.982094 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:38:35.982067    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6efba5b8_431e_4f6a_8a3e_e6c520ea8107.slice/crio-4f8f85a8f8e242b19c83efcb35c57f020ba880005e9d6a703cf7bb2761f2af37 WatchSource:0}: Error finding container 4f8f85a8f8e242b19c83efcb35c57f020ba880005e9d6a703cf7bb2761f2af37: Status 404 returned error can't find the container with id 4f8f85a8f8e242b19c83efcb35c57f020ba880005e9d6a703cf7bb2761f2af37
Jul 22 16:38:36.414641 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:36.414603    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7d60fbcd-q2qc6" event={"ID":"6efba5b8-431e-4f6a-8a3e-e6c520ea8107","Type":"ContainerStarted","Data":"460e70b7b595d73a2912102eec215a232dc8881684079ae9422ea4aa0e6104b1"}
Jul 22 16:38:36.414875 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:36.414670    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7d60fbcd-q2qc6" event={"ID":"6efba5b8-431e-4f6a-8a3e-e6c520ea8107","Type":"ContainerStarted","Data":"4f8f85a8f8e242b19c83efcb35c57f020ba880005e9d6a703cf7bb2761f2af37"}
Jul 22 16:38:36.431918 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:36.431867    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7d60fbcd-q2qc6" podStartSLOduration=1.4318349160000001 podStartE2EDuration="1.431834916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:38:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:38:36.430755236 +0000 UTC m=+1876.561225270" watchObservedRunningTime="2026-07-22 16:38:36.431834916 +0000 UTC m=+1876.562304918"
Jul 22 16:38:37.420938 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:37.420905    2563 generic.go:356] "Generic (PLEG): container finished" podID="6efba5b8-431e-4f6a-8a3e-e6c520ea8107" containerID="460e70b7b595d73a2912102eec215a232dc8881684079ae9422ea4aa0e6104b1" exitCode=0
Jul 22 16:38:37.421335 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:37.420977    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7d60fbcd-q2qc6" event={"ID":"6efba5b8-431e-4f6a-8a3e-e6c520ea8107","Type":"ContainerDied","Data":"460e70b7b595d73a2912102eec215a232dc8881684079ae9422ea4aa0e6104b1"}
Jul 22 16:38:38.556530 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:38.556506    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7d60fbcd-q2qc6"
Jul 22 16:38:38.644676 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:38.644643    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6efba5b8-431e-4f6a-8a3e-e6c520ea8107-maas-api-service-ca\") pod \"6efba5b8-431e-4f6a-8a3e-e6c520ea8107\" (UID: \"6efba5b8-431e-4f6a-8a3e-e6c520ea8107\") "
Jul 22 16:38:38.645057 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:38.645028    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6efba5b8-431e-4f6a-8a3e-e6c520ea8107-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6efba5b8-431e-4f6a-8a3e-e6c520ea8107" (UID: "6efba5b8-431e-4f6a-8a3e-e6c520ea8107"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 16:38:38.746060 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:38.745951    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6efba5b8-431e-4f6a-8a3e-e6c520ea8107-maas-api-service-ca\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:38:39.431205 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:39.431162    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7d60fbcd-q2qc6" event={"ID":"6efba5b8-431e-4f6a-8a3e-e6c520ea8107","Type":"ContainerDied","Data":"4f8f85a8f8e242b19c83efcb35c57f020ba880005e9d6a703cf7bb2761f2af37"}
Jul 22 16:38:39.431205 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:39.431211    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4f8f85a8f8e242b19c83efcb35c57f020ba880005e9d6a703cf7bb2761f2af37"
Jul 22 16:38:39.431412 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:39.431172    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7d60fbcd-q2qc6"
Jul 22 16:38:45.942582 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:45.942485    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb"]
Jul 22 16:38:45.943067 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:45.942741    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb" podUID="9fe0162a-0801-423d-8fb4-b3d9774b31a1" containerName="maas-api" containerID="cri-o://b27e5ae746decbe5790551e3ad0ff09b8dbbdd36da27e4c39c7592905a737242" gracePeriod=30
Jul 22 16:38:45.976213 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:45.976183    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:38:46.187271 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:46.187248    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb"
Jul 22 16:38:46.211567 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:46.211480    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9fe0162a-0801-423d-8fb4-b3d9774b31a1-maas-api-tls\") pod \"9fe0162a-0801-423d-8fb4-b3d9774b31a1\" (UID: \"9fe0162a-0801-423d-8fb4-b3d9774b31a1\") "
Jul 22 16:38:46.211567 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:46.211516    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9ztwp\" (UniqueName: \"kubernetes.io/projected/9fe0162a-0801-423d-8fb4-b3d9774b31a1-kube-api-access-9ztwp\") pod \"9fe0162a-0801-423d-8fb4-b3d9774b31a1\" (UID: \"9fe0162a-0801-423d-8fb4-b3d9774b31a1\") "
Jul 22 16:38:46.211797 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:46.211578    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fe0162a-0801-423d-8fb4-b3d9774b31a1-openshift-service-ca\") pod \"9fe0162a-0801-423d-8fb4-b3d9774b31a1\" (UID: \"9fe0162a-0801-423d-8fb4-b3d9774b31a1\") "
Jul 22 16:38:46.212079 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:46.212044    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fe0162a-0801-423d-8fb4-b3d9774b31a1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9fe0162a-0801-423d-8fb4-b3d9774b31a1" (UID: "9fe0162a-0801-423d-8fb4-b3d9774b31a1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 16:38:46.214274 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:46.214240    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fe0162a-0801-423d-8fb4-b3d9774b31a1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9fe0162a-0801-423d-8fb4-b3d9774b31a1" (UID: "9fe0162a-0801-423d-8fb4-b3d9774b31a1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:38:46.214377 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:46.214253    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fe0162a-0801-423d-8fb4-b3d9774b31a1-kube-api-access-9ztwp" (OuterVolumeSpecName: "kube-api-access-9ztwp") pod "9fe0162a-0801-423d-8fb4-b3d9774b31a1" (UID: "9fe0162a-0801-423d-8fb4-b3d9774b31a1"). InnerVolumeSpecName "kube-api-access-9ztwp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:38:46.312897 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:46.312839    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9fe0162a-0801-423d-8fb4-b3d9774b31a1-maas-api-tls\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:38:46.312897 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:46.312892    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ztwp\" (UniqueName: \"kubernetes.io/projected/9fe0162a-0801-423d-8fb4-b3d9774b31a1-kube-api-access-9ztwp\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:38:46.312897 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:46.312902    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fe0162a-0801-423d-8fb4-b3d9774b31a1-openshift-service-ca\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:38:46.463402 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:46.463328    2563 generic.go:356] "Generic (PLEG): container finished" podID="9fe0162a-0801-423d-8fb4-b3d9774b31a1" containerID="b27e5ae746decbe5790551e3ad0ff09b8dbbdd36da27e4c39c7592905a737242" exitCode=0
Jul 22 16:38:46.463402 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:46.463372    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb"
Jul 22 16:38:46.463577 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:46.463413    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb" event={"ID":"9fe0162a-0801-423d-8fb4-b3d9774b31a1","Type":"ContainerDied","Data":"b27e5ae746decbe5790551e3ad0ff09b8dbbdd36da27e4c39c7592905a737242"}
Jul 22 16:38:46.463577 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:46.463462    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb" event={"ID":"9fe0162a-0801-423d-8fb4-b3d9774b31a1","Type":"ContainerDied","Data":"20758acdb677e7aef9866c23019f84ae6897607b3b96b9b44ef427ca532613b4"}
Jul 22 16:38:46.463577 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:46.463486    2563 scope.go:117] "RemoveContainer" containerID="b27e5ae746decbe5790551e3ad0ff09b8dbbdd36da27e4c39c7592905a737242"
Jul 22 16:38:46.471923 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:46.471833    2563 scope.go:117] "RemoveContainer" containerID="b27e5ae746decbe5790551e3ad0ff09b8dbbdd36da27e4c39c7592905a737242"
Jul 22 16:38:46.472130 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:38:46.472105    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b27e5ae746decbe5790551e3ad0ff09b8dbbdd36da27e4c39c7592905a737242\": container with ID starting with b27e5ae746decbe5790551e3ad0ff09b8dbbdd36da27e4c39c7592905a737242 not found: ID does not exist" containerID="b27e5ae746decbe5790551e3ad0ff09b8dbbdd36da27e4c39c7592905a737242"
Jul 22 16:38:46.472191 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:46.472131    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b27e5ae746decbe5790551e3ad0ff09b8dbbdd36da27e4c39c7592905a737242"} err="failed to get container status \"b27e5ae746decbe5790551e3ad0ff09b8dbbdd36da27e4c39c7592905a737242\": rpc error: code = NotFound desc = could not find container \"b27e5ae746decbe5790551e3ad0ff09b8dbbdd36da27e4c39c7592905a737242\": container with ID starting with b27e5ae746decbe5790551e3ad0ff09b8dbbdd36da27e4c39c7592905a737242 not found: ID does not exist"
Jul 22 16:38:46.481907 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:46.481880    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb"]
Jul 22 16:38:46.486123 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:46.486096    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7d60fbcd-7c5475bd59-qpprb"]
Jul 22 16:38:48.447960 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:48.447924    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fe0162a-0801-423d-8fb4-b3d9774b31a1" path="/var/lib/kubelet/pods/9fe0162a-0801-423d-8fb4-b3d9774b31a1/volumes"
Jul 22 16:38:50.602072 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:50.602036    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:38:50.602548 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:50.602104    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7d60fbcd-q2qc6"]
Jul 22 16:38:50.604829 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:50.604807    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7d60fbcd-q2qc6"]
Jul 22 16:38:50.741278 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:50.741247    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"]
Jul 22 16:38:50.741539 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:50.741492    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz" podUID="de8da499-68de-4a31-a34a-167297a1a982" containerName="istio-proxy" containerID="cri-o://45b4525d0cd53e28cedcb602a2dd5a8dbac517890417913312ecac5f86c917eb" gracePeriod=30
Jul 22 16:38:50.757018 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:50.756990    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:38:50.997905 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:50.997847    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:38:51.503619 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:51.503588    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:38:51.591257 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:51.591219    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:38:52.447428 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:52.447394    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6efba5b8-431e-4f6a-8a3e-e6c520ea8107" path="/var/lib/kubelet/pods/6efba5b8-431e-4f6a-8a3e-e6c520ea8107/volumes"
Jul 22 16:38:53.103616 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:53.103581    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:38:55.988890 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:55.988866    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:56.097603 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.097507    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de8da499-68de-4a31-a34a-167297a1a982-istio-data\") pod \"de8da499-68de-4a31-a34a-167297a1a982\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") "
Jul 22 16:38:56.097603 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.097547    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vg8lx\" (UniqueName: \"kubernetes.io/projected/de8da499-68de-4a31-a34a-167297a1a982-kube-api-access-vg8lx\") pod \"de8da499-68de-4a31-a34a-167297a1a982\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") "
Jul 22 16:38:56.097603 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.097564    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de8da499-68de-4a31-a34a-167297a1a982-istiod-ca-cert\") pod \"de8da499-68de-4a31-a34a-167297a1a982\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") "
Jul 22 16:38:56.097603 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.097594    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de8da499-68de-4a31-a34a-167297a1a982-istio-token\") pod \"de8da499-68de-4a31-a34a-167297a1a982\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") "
Jul 22 16:38:56.097959 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.097640    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de8da499-68de-4a31-a34a-167297a1a982-istio-podinfo\") pod \"de8da499-68de-4a31-a34a-167297a1a982\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") "
Jul 22 16:38:56.097959 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.097701    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de8da499-68de-4a31-a34a-167297a1a982-credential-socket\") pod \"de8da499-68de-4a31-a34a-167297a1a982\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") "
Jul 22 16:38:56.097959 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.097731    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de8da499-68de-4a31-a34a-167297a1a982-workload-certs\") pod \"de8da499-68de-4a31-a34a-167297a1a982\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") "
Jul 22 16:38:56.097959 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.097786    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de8da499-68de-4a31-a34a-167297a1a982-istio-envoy\") pod \"de8da499-68de-4a31-a34a-167297a1a982\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") "
Jul 22 16:38:56.097959 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.097907    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de8da499-68de-4a31-a34a-167297a1a982-workload-socket\") pod \"de8da499-68de-4a31-a34a-167297a1a982\" (UID: \"de8da499-68de-4a31-a34a-167297a1a982\") "
Jul 22 16:38:56.098239 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.098133    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de8da499-68de-4a31-a34a-167297a1a982-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "de8da499-68de-4a31-a34a-167297a1a982" (UID: "de8da499-68de-4a31-a34a-167297a1a982"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 16:38:56.098239 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.098150    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de8da499-68de-4a31-a34a-167297a1a982-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "de8da499-68de-4a31-a34a-167297a1a982" (UID: "de8da499-68de-4a31-a34a-167297a1a982"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:38:56.098353 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.098241    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de8da499-68de-4a31-a34a-167297a1a982-istiod-ca-cert\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:38:56.098353 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.098335    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de8da499-68de-4a31-a34a-167297a1a982-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "de8da499-68de-4a31-a34a-167297a1a982" (UID: "de8da499-68de-4a31-a34a-167297a1a982"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:38:56.098353 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.098342    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de8da499-68de-4a31-a34a-167297a1a982-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "de8da499-68de-4a31-a34a-167297a1a982" (UID: "de8da499-68de-4a31-a34a-167297a1a982"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:38:56.098564 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.098368    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de8da499-68de-4a31-a34a-167297a1a982-workload-certs\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:38:56.098564 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.098428    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de8da499-68de-4a31-a34a-167297a1a982-istio-data" (OuterVolumeSpecName: "istio-data") pod "de8da499-68de-4a31-a34a-167297a1a982" (UID: "de8da499-68de-4a31-a34a-167297a1a982"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:38:56.099973 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.099948    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/de8da499-68de-4a31-a34a-167297a1a982-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "de8da499-68de-4a31-a34a-167297a1a982" (UID: "de8da499-68de-4a31-a34a-167297a1a982"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 16:38:56.100095 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.100009    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de8da499-68de-4a31-a34a-167297a1a982-istio-token" (OuterVolumeSpecName: "istio-token") pod "de8da499-68de-4a31-a34a-167297a1a982" (UID: "de8da499-68de-4a31-a34a-167297a1a982"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:38:56.100095 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.100046    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de8da499-68de-4a31-a34a-167297a1a982-kube-api-access-vg8lx" (OuterVolumeSpecName: "kube-api-access-vg8lx") pod "de8da499-68de-4a31-a34a-167297a1a982" (UID: "de8da499-68de-4a31-a34a-167297a1a982"). InnerVolumeSpecName "kube-api-access-vg8lx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:38:56.100095 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.100068    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de8da499-68de-4a31-a34a-167297a1a982-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "de8da499-68de-4a31-a34a-167297a1a982" (UID: "de8da499-68de-4a31-a34a-167297a1a982"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:38:56.199134 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.199097    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de8da499-68de-4a31-a34a-167297a1a982-credential-socket\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:38:56.199134 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.199125    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de8da499-68de-4a31-a34a-167297a1a982-istio-envoy\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:38:56.199134 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.199134    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de8da499-68de-4a31-a34a-167297a1a982-workload-socket\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:38:56.199134 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.199145    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de8da499-68de-4a31-a34a-167297a1a982-istio-data\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:38:56.199406 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.199154    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vg8lx\" (UniqueName: \"kubernetes.io/projected/de8da499-68de-4a31-a34a-167297a1a982-kube-api-access-vg8lx\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:38:56.199406 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.199162    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de8da499-68de-4a31-a34a-167297a1a982-istio-token\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:38:56.199406 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.199171    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de8da499-68de-4a31-a34a-167297a1a982-istio-podinfo\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:38:56.513324 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.513286    2563 generic.go:356] "Generic (PLEG): container finished" podID="de8da499-68de-4a31-a34a-167297a1a982" containerID="45b4525d0cd53e28cedcb602a2dd5a8dbac517890417913312ecac5f86c917eb" exitCode=0
Jul 22 16:38:56.513518 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.513347    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"
Jul 22 16:38:56.513518 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.513368    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz" event={"ID":"de8da499-68de-4a31-a34a-167297a1a982","Type":"ContainerDied","Data":"45b4525d0cd53e28cedcb602a2dd5a8dbac517890417913312ecac5f86c917eb"}
Jul 22 16:38:56.513518 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.513425    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz" event={"ID":"de8da499-68de-4a31-a34a-167297a1a982","Type":"ContainerDied","Data":"ff9cdeedc4f2b16dad1d1aec9b2994243354f139585255db76e3b1c869462866"}
Jul 22 16:38:56.513518 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.513448    2563 scope.go:117] "RemoveContainer" containerID="45b4525d0cd53e28cedcb602a2dd5a8dbac517890417913312ecac5f86c917eb"
Jul 22 16:38:56.522008 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.521988    2563 scope.go:117] "RemoveContainer" containerID="cc2572d54be8dedf2097a635d716204d61f8e0ef7ad98cf34d41dccc9e980d31"
Jul 22 16:38:56.530387 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.530367    2563 scope.go:117] "RemoveContainer" containerID="45b4525d0cd53e28cedcb602a2dd5a8dbac517890417913312ecac5f86c917eb"
Jul 22 16:38:56.530707 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:38:56.530677    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45b4525d0cd53e28cedcb602a2dd5a8dbac517890417913312ecac5f86c917eb\": container with ID starting with 45b4525d0cd53e28cedcb602a2dd5a8dbac517890417913312ecac5f86c917eb not found: ID does not exist" containerID="45b4525d0cd53e28cedcb602a2dd5a8dbac517890417913312ecac5f86c917eb"
Jul 22 16:38:56.530774 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.530724    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45b4525d0cd53e28cedcb602a2dd5a8dbac517890417913312ecac5f86c917eb"} err="failed to get container status \"45b4525d0cd53e28cedcb602a2dd5a8dbac517890417913312ecac5f86c917eb\": rpc error: code = NotFound desc = could not find container \"45b4525d0cd53e28cedcb602a2dd5a8dbac517890417913312ecac5f86c917eb\": container with ID starting with 45b4525d0cd53e28cedcb602a2dd5a8dbac517890417913312ecac5f86c917eb not found: ID does not exist"
Jul 22 16:38:56.530774 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.530750    2563 scope.go:117] "RemoveContainer" containerID="cc2572d54be8dedf2097a635d716204d61f8e0ef7ad98cf34d41dccc9e980d31"
Jul 22 16:38:56.531342 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:38:56.531312    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc2572d54be8dedf2097a635d716204d61f8e0ef7ad98cf34d41dccc9e980d31\": container with ID starting with cc2572d54be8dedf2097a635d716204d61f8e0ef7ad98cf34d41dccc9e980d31 not found: ID does not exist" containerID="cc2572d54be8dedf2097a635d716204d61f8e0ef7ad98cf34d41dccc9e980d31"
Jul 22 16:38:56.531452 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.531351    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc2572d54be8dedf2097a635d716204d61f8e0ef7ad98cf34d41dccc9e980d31"} err="failed to get container status \"cc2572d54be8dedf2097a635d716204d61f8e0ef7ad98cf34d41dccc9e980d31\": rpc error: code = NotFound desc = could not find container \"cc2572d54be8dedf2097a635d716204d61f8e0ef7ad98cf34d41dccc9e980d31\": container with ID starting with cc2572d54be8dedf2097a635d716204d61f8e0ef7ad98cf34d41dccc9e980d31 not found: ID does not exist"
Jul 22 16:38:56.532953 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.532933    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"]
Jul 22 16:38:56.536592 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:56.536572    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-7d60fbcd-openshift-default-66b4b59f59-g4kkz"]
Jul 22 16:38:58.447538 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:58.447505    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de8da499-68de-4a31-a34a-167297a1a982" path="/var/lib/kubelet/pods/de8da499-68de-4a31-a34a-167297a1a982/volumes"
Jul 22 16:38:59.250507 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:59.250472    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:38:59.350488 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:59.350455    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:38:59.449976 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:59.449941    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:38:59.550215 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:59.550128    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:38:59.974355 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:38:59.974325    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:39:06.511138 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:39:06.511102    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:39:25.973126 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:39:25.973089    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:39:30.780749 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:39:30.780691    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:39:32.799725 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:39:32.799690    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:39:32.966412 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:39:32.966379    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:39:33.079559 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:39:33.079473    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:39:48.934691 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:39:48.934646    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:39:51.807130 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:39:51.807055    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:39:54.802202 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:39:54.802157    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:39:54.873740 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:39:54.873696    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:39:54.972665 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:39:54.972616    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:05.519107 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:05.519062    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:06.343818 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:06.343771    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:12.912959 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:12.912816    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:13.023133 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:13.023090    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:13.153668 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:13.153617    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:13.290667 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:13.290622    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:14.021988 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:14.021954    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:14.112389 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:14.112356    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:14.213073 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:14.213039    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:15.829158 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:15.829126    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:18.233934 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:18.233890    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:18.336175 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:18.336138    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:21.834645 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:21.834616    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:21.935316 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:21.935283    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:22.520017 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:22.519977    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:28.451787 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:28.451753    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:28.553769 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:28.553729    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:28.702811 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:28.702727    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:28.990678 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:28.990593    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"]
Jul 22 16:40:28.990934 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:28.990920    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6efba5b8-431e-4f6a-8a3e-e6c520ea8107" containerName="revoke-keys"
Jul 22 16:40:28.991005 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:28.990935    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6efba5b8-431e-4f6a-8a3e-e6c520ea8107" containerName="revoke-keys"
Jul 22 16:40:28.991005 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:28.990947    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fe0162a-0801-423d-8fb4-b3d9774b31a1" containerName="maas-api"
Jul 22 16:40:28.991005 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:28.990953    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fe0162a-0801-423d-8fb4-b3d9774b31a1" containerName="maas-api"
Jul 22 16:40:28.991005 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:28.990960    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de8da499-68de-4a31-a34a-167297a1a982" containerName="istio-proxy"
Jul 22 16:40:28.991005 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:28.990966    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="de8da499-68de-4a31-a34a-167297a1a982" containerName="istio-proxy"
Jul 22 16:40:28.991005 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:28.990977    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de8da499-68de-4a31-a34a-167297a1a982" containerName="istio-proxy"
Jul 22 16:40:28.991005 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:28.990982    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="de8da499-68de-4a31-a34a-167297a1a982" containerName="istio-proxy"
Jul 22 16:40:28.991359 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:28.991053    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6efba5b8-431e-4f6a-8a3e-e6c520ea8107" containerName="revoke-keys"
Jul 22 16:40:28.991359 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:28.991061    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="de8da499-68de-4a31-a34a-167297a1a982" containerName="istio-proxy"
Jul 22 16:40:28.991359 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:28.991069    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="de8da499-68de-4a31-a34a-167297a1a982" containerName="istio-proxy"
Jul 22 16:40:28.991359 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:28.991075    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fe0162a-0801-423d-8fb4-b3d9774b31a1" containerName="maas-api"
Jul 22 16:40:28.993990 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:28.993972    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:28.997152 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:28.997134    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-0bf04dbf-openshift-default-dockercfg-bj5s2\""
Jul 22 16:40:28.997240 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:28.997138    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 22 16:40:29.013300 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.013274    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:29.019263 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.019239    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"]
Jul 22 16:40:29.083459 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.083423    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-workload-certs\") pod \"e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") " pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:29.083644 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.083485    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-workload-socket\") pod \"e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") " pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:29.083644 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.083531    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-istiod-ca-cert\") pod \"e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") " pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:29.083644 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.083561    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-credential-socket\") pod \"e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") " pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:29.083644 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.083600    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjtmp\" (UniqueName: \"kubernetes.io/projected/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-kube-api-access-zjtmp\") pod \"e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") " pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:29.083800 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.083649    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-istio-podinfo\") pod \"e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") " pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:29.083800 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.083676    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-istio-data\") pod \"e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") " pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:29.083800 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.083705    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-istio-token\") pod \"e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") " pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:29.083800 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.083732    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-istio-envoy\") pod \"e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") " pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:29.184770 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.184734    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-istiod-ca-cert\") pod \"e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") " pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:29.184770 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.184770    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-credential-socket\") pod \"e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") " pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:29.185054 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.184791    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjtmp\" (UniqueName: \"kubernetes.io/projected/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-kube-api-access-zjtmp\") pod \"e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") " pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:29.185054 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.184811    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-istio-podinfo\") pod \"e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") " pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:29.185054 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.184838    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-istio-data\") pod \"e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") " pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:29.185054 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.184910    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-istio-token\") pod \"e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") " pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:29.185054 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.184951    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-istio-envoy\") pod \"e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") " pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:29.185054 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.185005    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-workload-certs\") pod \"e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") " pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:29.185054 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.185053    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-workload-socket\") pod \"e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") " pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:29.185468 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.185310    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-istio-data\") pod \"e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") " pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:29.185528 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.185466    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-workload-socket\") pod \"e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") " pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:29.185583 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.185543    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-credential-socket\") pod \"e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") " pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:29.185634 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.185579    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-workload-certs\") pod \"e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") " pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:29.185634 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.185601    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-istiod-ca-cert\") pod \"e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") " pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:29.187366 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.187334    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-istio-envoy\") pod \"e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") " pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:29.187646 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.187629    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-istio-podinfo\") pod \"e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") " pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:29.195628 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.195602    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjtmp\" (UniqueName: \"kubernetes.io/projected/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-kube-api-access-zjtmp\") pod \"e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") " pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:29.195752 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.195679    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-istio-token\") pod \"e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") " pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:29.283839 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.283743    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:29.303834 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.303805    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:29.439229 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.439199    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"]
Jul 22 16:40:29.441105 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:40:29.441078    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4950bf09_fb48_44e4_95c2_dcb6ee2e895a.slice/crio-c9125220b1cc3489230f7fe056559f5875fddabc277159b2ce3db74629d7aecd WatchSource:0}: Error finding container c9125220b1cc3489230f7fe056559f5875fddabc277159b2ce3db74629d7aecd: Status 404 returned error can't find the container with id c9125220b1cc3489230f7fe056559f5875fddabc277159b2ce3db74629d7aecd
Jul 22 16:40:29.443191 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.443159    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:40:29.443250 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.443240    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:40:29.443295 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.443268    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:40:29.939535 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.939504    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx" event={"ID":"4950bf09-fb48-44e4-95c2-dcb6ee2e895a","Type":"ContainerStarted","Data":"07d2090b31a1f6a5a9347ff6acd6c93dfe91a4911a29a37f00b91ae1d449d5b4"}
Jul 22 16:40:29.940033 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.939563    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx" event={"ID":"4950bf09-fb48-44e4-95c2-dcb6ee2e895a","Type":"ContainerStarted","Data":"c9125220b1cc3489230f7fe056559f5875fddabc277159b2ce3db74629d7aecd"}
Jul 22 16:40:29.962591 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:29.962544    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx" podStartSLOduration=1.96253038 podStartE2EDuration="1.96253038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:40:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:40:29.960935635 +0000 UTC m=+1990.091405656" watchObservedRunningTime="2026-07-22 16:40:29.96253038 +0000 UTC m=+1990.093000379"
Jul 22 16:40:30.304445 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:30.304355    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:30.309173 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:30.309150    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:30.944141 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:30.944110    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:30.945208 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:30.945189    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:31.929384 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:31.929349    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:32.006350 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:32.006314    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx"]
Jul 22 16:40:32.010198 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:32.010177    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx"
Jul 22 16:40:32.013378 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:32.013352    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-0bf04dbf-serving-cert\""
Jul 22 16:40:32.013774 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:32.013755    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 16:40:32.014773 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:32.014754    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vghv2\""
Jul 22 16:40:32.014903 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:32.014838    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 22 16:40:32.025005 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:32.024978    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx"]
Jul 22 16:40:32.042093 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:32.042060    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:32.106669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:32.106630    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qmd6\" (UniqueName: \"kubernetes.io/projected/9e87ae49-4b64-4fa4-833e-457d29a6368d-kube-api-access-8qmd6\") pod \"maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx\" (UID: \"9e87ae49-4b64-4fa4-833e-457d29a6368d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx"
Jul 22 16:40:32.106836 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:32.106696    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e87ae49-4b64-4fa4-833e-457d29a6368d-maas-api-tls\") pod \"maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx\" (UID: \"9e87ae49-4b64-4fa4-833e-457d29a6368d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx"
Jul 22 16:40:32.106836 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:32.106767    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e87ae49-4b64-4fa4-833e-457d29a6368d-openshift-service-ca\") pod \"maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx\" (UID: \"9e87ae49-4b64-4fa4-833e-457d29a6368d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx"
Jul 22 16:40:32.133069 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:32.133035    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:32.208260 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:32.208161    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qmd6\" (UniqueName: \"kubernetes.io/projected/9e87ae49-4b64-4fa4-833e-457d29a6368d-kube-api-access-8qmd6\") pod \"maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx\" (UID: \"9e87ae49-4b64-4fa4-833e-457d29a6368d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx"
Jul 22 16:40:32.208260 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:32.208215    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e87ae49-4b64-4fa4-833e-457d29a6368d-maas-api-tls\") pod \"maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx\" (UID: \"9e87ae49-4b64-4fa4-833e-457d29a6368d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx"
Jul 22 16:40:32.208490 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:32.208266    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e87ae49-4b64-4fa4-833e-457d29a6368d-openshift-service-ca\") pod \"maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx\" (UID: \"9e87ae49-4b64-4fa4-833e-457d29a6368d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx"
Jul 22 16:40:32.208490 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:40:32.208375    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-0bf04dbf-serving-cert: secret "maas-api-e2e-mt-0bf04dbf-serving-cert" not found
Jul 22 16:40:32.208490 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:40:32.208445    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e87ae49-4b64-4fa4-833e-457d29a6368d-maas-api-tls podName:9e87ae49-4b64-4fa4-833e-457d29a6368d nodeName:}" failed. No retries permitted until 2026-07-22 16:40:32.708426209 +0000 UTC m=+1992.838896203 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9e87ae49-4b64-4fa4-833e-457d29a6368d-maas-api-tls") pod "maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx" (UID: "9e87ae49-4b64-4fa4-833e-457d29a6368d") : secret "maas-api-e2e-mt-0bf04dbf-serving-cert" not found
Jul 22 16:40:32.208845 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:32.208827    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e87ae49-4b64-4fa4-833e-457d29a6368d-openshift-service-ca\") pod \"maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx\" (UID: \"9e87ae49-4b64-4fa4-833e-457d29a6368d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx"
Jul 22 16:40:32.217986 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:32.217963    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qmd6\" (UniqueName: \"kubernetes.io/projected/9e87ae49-4b64-4fa4-833e-457d29a6368d-kube-api-access-8qmd6\") pod \"maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx\" (UID: \"9e87ae49-4b64-4fa4-833e-457d29a6368d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx"
Jul 22 16:40:32.713822 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:32.713783    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e87ae49-4b64-4fa4-833e-457d29a6368d-maas-api-tls\") pod \"maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx\" (UID: \"9e87ae49-4b64-4fa4-833e-457d29a6368d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx"
Jul 22 16:40:32.714037 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:40:32.713926    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-0bf04dbf-serving-cert: secret "maas-api-e2e-mt-0bf04dbf-serving-cert" not found
Jul 22 16:40:32.714037 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:40:32.714002    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e87ae49-4b64-4fa4-833e-457d29a6368d-maas-api-tls podName:9e87ae49-4b64-4fa4-833e-457d29a6368d nodeName:}" failed. No retries permitted until 2026-07-22 16:40:33.713983278 +0000 UTC m=+1993.844453261 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9e87ae49-4b64-4fa4-833e-457d29a6368d-maas-api-tls") pod "maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx" (UID: "9e87ae49-4b64-4fa4-833e-457d29a6368d") : secret "maas-api-e2e-mt-0bf04dbf-serving-cert" not found
Jul 22 16:40:32.750198 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:32.750161    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:32.850720 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:32.850681    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:32.951285 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:32.951252    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:33.724519 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:33.724480    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e87ae49-4b64-4fa4-833e-457d29a6368d-maas-api-tls\") pod \"maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx\" (UID: \"9e87ae49-4b64-4fa4-833e-457d29a6368d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx"
Jul 22 16:40:33.726978 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:33.726945    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e87ae49-4b64-4fa4-833e-457d29a6368d-maas-api-tls\") pod \"maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx\" (UID: \"9e87ae49-4b64-4fa4-833e-457d29a6368d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx"
Jul 22 16:40:33.821314 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:33.821277    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx"
Jul 22 16:40:33.955982 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:33.955954    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx"]
Jul 22 16:40:33.957515 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:40:33.957488    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e87ae49_4b64_4fa4_833e_457d29a6368d.slice/crio-6d49e72df21a93d4f0acee0c28148e4b3eb7d286753035aa9dc3b483b2db939b WatchSource:0}: Error finding container 6d49e72df21a93d4f0acee0c28148e4b3eb7d286753035aa9dc3b483b2db939b: Status 404 returned error can't find the container with id 6d49e72df21a93d4f0acee0c28148e4b3eb7d286753035aa9dc3b483b2db939b
Jul 22 16:40:34.964989 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:34.964955    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx" event={"ID":"9e87ae49-4b64-4fa4-833e-457d29a6368d","Type":"ContainerStarted","Data":"76b921952e4f71f2f39574d69083d794be196bf472ae2f27999076d68e81bd6f"}
Jul 22 16:40:34.965386 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:34.965009    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx" event={"ID":"9e87ae49-4b64-4fa4-833e-457d29a6368d","Type":"ContainerStarted","Data":"6d49e72df21a93d4f0acee0c28148e4b3eb7d286753035aa9dc3b483b2db939b"}
Jul 22 16:40:34.965386 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:34.965041    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx"
Jul 22 16:40:34.990041 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:34.989980    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx" podStartSLOduration=3.501887131 podStartE2EDuration="3.989962767s" totalImagesPullingTime="488.075636ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:40:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:40:34.988717384 +0000 UTC m=+1995.119187405" watchObservedRunningTime="2026-07-22 16:40:34.989962767 +0000 UTC m=+1995.120432769"
Jul 22 16:40:36.430430 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:36.430395    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:37.113579 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:37.113545    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:37.205627 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:37.205593    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:37.526435 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:37.526399    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:40.975573 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:40.975545    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx"
Jul 22 16:40:43.336486 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:43.336446    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:43.437875 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:43.437816    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:43.692537 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:43.692500    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0bf04dbf-2hgc5"]
Jul 22 16:40:43.696794 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:43.696774    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0bf04dbf-2hgc5"
Jul 22 16:40:43.699798 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:43.699773    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-phfdj\""
Jul 22 16:40:43.704791 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:43.704766    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0bf04dbf-2hgc5"]
Jul 22 16:40:43.816623 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:43.816580    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/830286e7-c1a7-4037-85f4-6df13f755a0d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0bf04dbf-2hgc5\" (UID: \"830286e7-c1a7-4037-85f4-6df13f755a0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0bf04dbf-2hgc5"
Jul 22 16:40:43.917663 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:43.917619    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/830286e7-c1a7-4037-85f4-6df13f755a0d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0bf04dbf-2hgc5\" (UID: \"830286e7-c1a7-4037-85f4-6df13f755a0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0bf04dbf-2hgc5"
Jul 22 16:40:43.918245 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:43.918211    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/830286e7-c1a7-4037-85f4-6df13f755a0d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0bf04dbf-2hgc5\" (UID: \"830286e7-c1a7-4037-85f4-6df13f755a0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0bf04dbf-2hgc5"
Jul 22 16:40:44.007636 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:44.007540    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0bf04dbf-2hgc5"
Jul 22 16:40:44.149060 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:44.149027    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0bf04dbf-2hgc5"]
Jul 22 16:40:44.151591 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:40:44.151561    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod830286e7_c1a7_4037_85f4_6df13f755a0d.slice/crio-dad313ea9d63bc198a5e8268992cd7c53d6fe7ba49f989d2e3ead619e39dd728 WatchSource:0}: Error finding container dad313ea9d63bc198a5e8268992cd7c53d6fe7ba49f989d2e3ead619e39dd728: Status 404 returned error can't find the container with id dad313ea9d63bc198a5e8268992cd7c53d6fe7ba49f989d2e3ead619e39dd728
Jul 22 16:40:45.020492 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:45.020453    2563 generic.go:356] "Generic (PLEG): container finished" podID="830286e7-c1a7-4037-85f4-6df13f755a0d" containerID="c7e09dff96f0b0b59f22c924a76d814a647f6a60dab4b3689521756acf4b714a" exitCode=0
Jul 22 16:40:45.021008 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:45.020552    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0bf04dbf-2hgc5" event={"ID":"830286e7-c1a7-4037-85f4-6df13f755a0d","Type":"ContainerDied","Data":"c7e09dff96f0b0b59f22c924a76d814a647f6a60dab4b3689521756acf4b714a"}
Jul 22 16:40:45.021008 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:45.020640    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0bf04dbf-2hgc5" event={"ID":"830286e7-c1a7-4037-85f4-6df13f755a0d","Type":"ContainerStarted","Data":"dad313ea9d63bc198a5e8268992cd7c53d6fe7ba49f989d2e3ead619e39dd728"}
Jul 22 16:40:46.166087 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:46.166056    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0bf04dbf-2hgc5"
Jul 22 16:40:46.238843 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:46.238801    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/830286e7-c1a7-4037-85f4-6df13f755a0d-maas-api-service-ca\") pod \"830286e7-c1a7-4037-85f4-6df13f755a0d\" (UID: \"830286e7-c1a7-4037-85f4-6df13f755a0d\") "
Jul 22 16:40:46.239178 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:46.239155    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/830286e7-c1a7-4037-85f4-6df13f755a0d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "830286e7-c1a7-4037-85f4-6df13f755a0d" (UID: "830286e7-c1a7-4037-85f4-6df13f755a0d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 16:40:46.340337 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:46.340245    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/830286e7-c1a7-4037-85f4-6df13f755a0d-maas-api-service-ca\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:40:46.624512 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:46.624478    2563 patch_prober.go:28] interesting pod/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": read tcp 10.134.0.2:49594->10.134.0.41:15021: read: connection reset by peer" start-of-body=
Jul 22 16:40:46.624687 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:46.624549    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx" podUID="4950bf09-fb48-44e4-95c2-dcb6ee2e895a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": read tcp 10.134.0.2:49594->10.134.0.41:15021: read: connection reset by peer"
Jul 22 16:40:47.032375 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:47.032340    2563 generic.go:356] "Generic (PLEG): container finished" podID="4950bf09-fb48-44e4-95c2-dcb6ee2e895a" containerID="07d2090b31a1f6a5a9347ff6acd6c93dfe91a4911a29a37f00b91ae1d449d5b4" exitCode=0
Jul 22 16:40:47.032568 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:47.032412    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx" event={"ID":"4950bf09-fb48-44e4-95c2-dcb6ee2e895a","Type":"ContainerDied","Data":"07d2090b31a1f6a5a9347ff6acd6c93dfe91a4911a29a37f00b91ae1d449d5b4"}
Jul 22 16:40:47.032886 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:47.032843    2563 scope.go:117] "RemoveContainer" containerID="07d2090b31a1f6a5a9347ff6acd6c93dfe91a4911a29a37f00b91ae1d449d5b4"
Jul 22 16:40:47.033727 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:47.033707    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0bf04dbf-2hgc5"
Jul 22 16:40:47.033809 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:47.033707    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0bf04dbf-2hgc5" event={"ID":"830286e7-c1a7-4037-85f4-6df13f755a0d","Type":"ContainerDied","Data":"dad313ea9d63bc198a5e8268992cd7c53d6fe7ba49f989d2e3ead619e39dd728"}
Jul 22 16:40:47.033888 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:47.033836    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dad313ea9d63bc198a5e8268992cd7c53d6fe7ba49f989d2e3ead619e39dd728"
Jul 22 16:40:47.033936 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:47.033900    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:40:47.033995 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:47.033975    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:40:47.034047 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:47.034030    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:40:48.039950 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:48.039917    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx" event={"ID":"4950bf09-fb48-44e4-95c2-dcb6ee2e895a","Type":"ContainerStarted","Data":"d14afbdce7a2555fc11edfce58e7dc48a95b7ecb16bef81dfec2c85199a9cf31"}
Jul 22 16:40:48.305008 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:48.304909    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:48.305182 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:48.305050    2563 patch_prober.go:28] interesting pod/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 22 16:40:48.305182 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:48.305098    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx" podUID="4950bf09-fb48-44e4-95c2-dcb6ee2e895a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 22 16:40:49.304671 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:49.304636    2563 patch_prober.go:28] interesting pod/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 22 16:40:49.305161 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:49.304705    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx" podUID="4950bf09-fb48-44e4-95c2-dcb6ee2e895a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 22 16:40:50.304426 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:50.304388    2563 patch_prober.go:28] interesting pod/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 22 16:40:50.304672 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:50.304448    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx" podUID="4950bf09-fb48-44e4-95c2-dcb6ee2e895a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 22 16:40:51.304578 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:51.304542    2563 patch_prober.go:28] interesting pod/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 22 16:40:51.305008 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:51.304600    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx" podUID="4950bf09-fb48-44e4-95c2-dcb6ee2e895a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 22 16:40:52.304687 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:52.304644    2563 patch_prober.go:28] interesting pod/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 22 16:40:52.305145 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:52.304708    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx" podUID="4950bf09-fb48-44e4-95c2-dcb6ee2e895a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 22 16:40:53.308437 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:53.308402    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:53.308932 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:53.308792    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:53.309527 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:53.309507    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:40:53.798373 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:53.798344    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx"]
Jul 22 16:40:53.798562 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:53.798537    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx" podUID="9e87ae49-4b64-4fa4-833e-457d29a6368d" containerName="maas-api" containerID="cri-o://76b921952e4f71f2f39574d69083d794be196bf472ae2f27999076d68e81bd6f" gracePeriod=30
Jul 22 16:40:53.855695 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:53.855656    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:54.044826 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:54.044800    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx"
Jul 22 16:40:54.068749 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:54.068657    2563 generic.go:356] "Generic (PLEG): container finished" podID="9e87ae49-4b64-4fa4-833e-457d29a6368d" containerID="76b921952e4f71f2f39574d69083d794be196bf472ae2f27999076d68e81bd6f" exitCode=0
Jul 22 16:40:54.068749 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:54.068707    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx"
Jul 22 16:40:54.068958 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:54.068741    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx" event={"ID":"9e87ae49-4b64-4fa4-833e-457d29a6368d","Type":"ContainerDied","Data":"76b921952e4f71f2f39574d69083d794be196bf472ae2f27999076d68e81bd6f"}
Jul 22 16:40:54.068958 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:54.068800    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx" event={"ID":"9e87ae49-4b64-4fa4-833e-457d29a6368d","Type":"ContainerDied","Data":"6d49e72df21a93d4f0acee0c28148e4b3eb7d286753035aa9dc3b483b2db939b"}
Jul 22 16:40:54.068958 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:54.068828    2563 scope.go:117] "RemoveContainer" containerID="76b921952e4f71f2f39574d69083d794be196bf472ae2f27999076d68e81bd6f"
Jul 22 16:40:54.078634 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:54.078612    2563 scope.go:117] "RemoveContainer" containerID="76b921952e4f71f2f39574d69083d794be196bf472ae2f27999076d68e81bd6f"
Jul 22 16:40:54.078955 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:40:54.078935    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76b921952e4f71f2f39574d69083d794be196bf472ae2f27999076d68e81bd6f\": container with ID starting with 76b921952e4f71f2f39574d69083d794be196bf472ae2f27999076d68e81bd6f not found: ID does not exist" containerID="76b921952e4f71f2f39574d69083d794be196bf472ae2f27999076d68e81bd6f"
Jul 22 16:40:54.079030 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:54.078965    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76b921952e4f71f2f39574d69083d794be196bf472ae2f27999076d68e81bd6f"} err="failed to get container status \"76b921952e4f71f2f39574d69083d794be196bf472ae2f27999076d68e81bd6f\": rpc error: code = NotFound desc = could not find container \"76b921952e4f71f2f39574d69083d794be196bf472ae2f27999076d68e81bd6f\": container with ID starting with 76b921952e4f71f2f39574d69083d794be196bf472ae2f27999076d68e81bd6f not found: ID does not exist"
Jul 22 16:40:54.114313 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:54.114280    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e87ae49-4b64-4fa4-833e-457d29a6368d-maas-api-tls\") pod \"9e87ae49-4b64-4fa4-833e-457d29a6368d\" (UID: \"9e87ae49-4b64-4fa4-833e-457d29a6368d\") "
Jul 22 16:40:54.114455 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:54.114345    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8qmd6\" (UniqueName: \"kubernetes.io/projected/9e87ae49-4b64-4fa4-833e-457d29a6368d-kube-api-access-8qmd6\") pod \"9e87ae49-4b64-4fa4-833e-457d29a6368d\" (UID: \"9e87ae49-4b64-4fa4-833e-457d29a6368d\") "
Jul 22 16:40:54.114455 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:54.114368    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e87ae49-4b64-4fa4-833e-457d29a6368d-openshift-service-ca\") pod \"9e87ae49-4b64-4fa4-833e-457d29a6368d\" (UID: \"9e87ae49-4b64-4fa4-833e-457d29a6368d\") "
Jul 22 16:40:54.114793 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:54.114769    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e87ae49-4b64-4fa4-833e-457d29a6368d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9e87ae49-4b64-4fa4-833e-457d29a6368d" (UID: "9e87ae49-4b64-4fa4-833e-457d29a6368d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 16:40:54.116528 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:54.116497    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e87ae49-4b64-4fa4-833e-457d29a6368d-kube-api-access-8qmd6" (OuterVolumeSpecName: "kube-api-access-8qmd6") pod "9e87ae49-4b64-4fa4-833e-457d29a6368d" (UID: "9e87ae49-4b64-4fa4-833e-457d29a6368d"). InnerVolumeSpecName "kube-api-access-8qmd6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:40:54.116650 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:54.116594    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e87ae49-4b64-4fa4-833e-457d29a6368d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9e87ae49-4b64-4fa4-833e-457d29a6368d" (UID: "9e87ae49-4b64-4fa4-833e-457d29a6368d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:40:54.215220 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:54.215182    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e87ae49-4b64-4fa4-833e-457d29a6368d-openshift-service-ca\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:40:54.215220 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:54.215218    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e87ae49-4b64-4fa4-833e-457d29a6368d-maas-api-tls\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:40:54.215220 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:54.215227    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qmd6\" (UniqueName: \"kubernetes.io/projected/9e87ae49-4b64-4fa4-833e-457d29a6368d-kube-api-access-8qmd6\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:40:54.393204 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:54.393170    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx"]
Jul 22 16:40:54.397950 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:54.397925    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0bf04dbf-555975db87-zdzkx"]
Jul 22 16:40:54.447664 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:54.447636    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e87ae49-4b64-4fa4-833e-457d29a6368d" path="/var/lib/kubelet/pods/9e87ae49-4b64-4fa4-833e-457d29a6368d/volumes"
Jul 22 16:40:58.778319 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:58.778285    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0bf04dbf-2hgc5"]
Jul 22 16:40:58.781465 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:58.781437    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0bf04dbf-2hgc5"]
Jul 22 16:40:58.782053 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:58.782032    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:40:59.150447 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:40:59.150365    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:41:00.447906 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:00.447846    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="830286e7-c1a7-4037-85f4-6df13f755a0d" path="/var/lib/kubelet/pods/830286e7-c1a7-4037-85f4-6df13f755a0d/volumes"
Jul 22 16:41:05.344191 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:05.344158    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:41:05.443459 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:05.443410    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:41:05.603432 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:05.603350    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"]
Jul 22 16:41:05.603618 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:05.603590    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx" podUID="4950bf09-fb48-44e4-95c2-dcb6ee2e895a" containerName="istio-proxy" containerID="cri-o://d14afbdce7a2555fc11edfce58e7dc48a95b7ecb16bef81dfec2c85199a9cf31" gracePeriod=30
Jul 22 16:41:05.631185 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:05.631159    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:41:05.843083 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:05.843049    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:41:05.982961 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:05.982930    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:41:06.100098 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:06.100060    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:41:06.267823 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:06.267731    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:41:08.311033 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:08.310991    2563 patch_prober.go:28] interesting pod/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Jul 22 16:41:08.311391 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:08.311057    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx" podUID="4950bf09-fb48-44e4-95c2-dcb6ee2e895a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 22 16:41:10.863792 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:10.863767    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:41:11.058375 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.058335    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-istio-envoy\") pod \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") "
Jul 22 16:41:11.058375 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.058376    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-istio-data\") pod \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") "
Jul 22 16:41:11.058655 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.058411    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-workload-socket\") pod \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") "
Jul 22 16:41:11.058655 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.058433    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-istiod-ca-cert\") pod \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") "
Jul 22 16:41:11.058655 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.058465    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-istio-podinfo\") pod \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") "
Jul 22 16:41:11.058655 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.058502    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjtmp\" (UniqueName: \"kubernetes.io/projected/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-kube-api-access-zjtmp\") pod \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") "
Jul 22 16:41:11.058655 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.058525    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-istio-token\") pod \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") "
Jul 22 16:41:11.058655 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.058593    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-credential-socket\") pod \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") "
Jul 22 16:41:11.058655 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.058639    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-workload-certs\") pod \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\" (UID: \"4950bf09-fb48-44e4-95c2-dcb6ee2e895a\") "
Jul 22 16:41:11.059043 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.058702    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4950bf09-fb48-44e4-95c2-dcb6ee2e895a" (UID: "4950bf09-fb48-44e4-95c2-dcb6ee2e895a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:41:11.059043 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.058935    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-workload-socket\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:41:11.059043 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.058931    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4950bf09-fb48-44e4-95c2-dcb6ee2e895a" (UID: "4950bf09-fb48-44e4-95c2-dcb6ee2e895a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:41:11.059043 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.058982    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4950bf09-fb48-44e4-95c2-dcb6ee2e895a" (UID: "4950bf09-fb48-44e4-95c2-dcb6ee2e895a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:41:11.059256 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.059224    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4950bf09-fb48-44e4-95c2-dcb6ee2e895a" (UID: "4950bf09-fb48-44e4-95c2-dcb6ee2e895a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 16:41:11.059421 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.059400    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-istio-data" (OuterVolumeSpecName: "istio-data") pod "4950bf09-fb48-44e4-95c2-dcb6ee2e895a" (UID: "4950bf09-fb48-44e4-95c2-dcb6ee2e895a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:41:11.060955 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.060918    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4950bf09-fb48-44e4-95c2-dcb6ee2e895a" (UID: "4950bf09-fb48-44e4-95c2-dcb6ee2e895a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 16:41:11.061072 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.060988    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-kube-api-access-zjtmp" (OuterVolumeSpecName: "kube-api-access-zjtmp") pod "4950bf09-fb48-44e4-95c2-dcb6ee2e895a" (UID: "4950bf09-fb48-44e4-95c2-dcb6ee2e895a"). InnerVolumeSpecName "kube-api-access-zjtmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:41:11.061072 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.061060    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4950bf09-fb48-44e4-95c2-dcb6ee2e895a" (UID: "4950bf09-fb48-44e4-95c2-dcb6ee2e895a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:41:11.061178 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.061080    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-istio-token" (OuterVolumeSpecName: "istio-token") pod "4950bf09-fb48-44e4-95c2-dcb6ee2e895a" (UID: "4950bf09-fb48-44e4-95c2-dcb6ee2e895a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:41:11.148756 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.148719    2563 generic.go:356] "Generic (PLEG): container finished" podID="4950bf09-fb48-44e4-95c2-dcb6ee2e895a" containerID="d14afbdce7a2555fc11edfce58e7dc48a95b7ecb16bef81dfec2c85199a9cf31" exitCode=0
Jul 22 16:41:11.148982 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.148774    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx" event={"ID":"4950bf09-fb48-44e4-95c2-dcb6ee2e895a","Type":"ContainerDied","Data":"d14afbdce7a2555fc11edfce58e7dc48a95b7ecb16bef81dfec2c85199a9cf31"}
Jul 22 16:41:11.148982 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.148781    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"
Jul 22 16:41:11.148982 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.148824    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx" event={"ID":"4950bf09-fb48-44e4-95c2-dcb6ee2e895a","Type":"ContainerDied","Data":"c9125220b1cc3489230f7fe056559f5875fddabc277159b2ce3db74629d7aecd"}
Jul 22 16:41:11.148982 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.148846    2563 scope.go:117] "RemoveContainer" containerID="d14afbdce7a2555fc11edfce58e7dc48a95b7ecb16bef81dfec2c85199a9cf31"
Jul 22 16:41:11.159307 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.159280    2563 scope.go:117] "RemoveContainer" containerID="07d2090b31a1f6a5a9347ff6acd6c93dfe91a4911a29a37f00b91ae1d449d5b4"
Jul 22 16:41:11.159603 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.159580    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-istiod-ca-cert\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:41:11.159603 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.159603    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-istio-podinfo\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:41:11.159724 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.159617    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjtmp\" (UniqueName: \"kubernetes.io/projected/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-kube-api-access-zjtmp\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:41:11.159724 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.159634    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-istio-token\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:41:11.159724 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.159647    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-credential-socket\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:41:11.159724 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.159660    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-workload-certs\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:41:11.159724 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.159672    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-istio-envoy\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:41:11.159724 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.159680    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4950bf09-fb48-44e4-95c2-dcb6ee2e895a-istio-data\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:41:11.167588 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.167568    2563 scope.go:117] "RemoveContainer" containerID="d14afbdce7a2555fc11edfce58e7dc48a95b7ecb16bef81dfec2c85199a9cf31"
Jul 22 16:41:11.170978 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:41:11.170952    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d14afbdce7a2555fc11edfce58e7dc48a95b7ecb16bef81dfec2c85199a9cf31\": container with ID starting with d14afbdce7a2555fc11edfce58e7dc48a95b7ecb16bef81dfec2c85199a9cf31 not found: ID does not exist" containerID="d14afbdce7a2555fc11edfce58e7dc48a95b7ecb16bef81dfec2c85199a9cf31"
Jul 22 16:41:11.171088 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.170984    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d14afbdce7a2555fc11edfce58e7dc48a95b7ecb16bef81dfec2c85199a9cf31"} err="failed to get container status \"d14afbdce7a2555fc11edfce58e7dc48a95b7ecb16bef81dfec2c85199a9cf31\": rpc error: code = NotFound desc = could not find container \"d14afbdce7a2555fc11edfce58e7dc48a95b7ecb16bef81dfec2c85199a9cf31\": container with ID starting with d14afbdce7a2555fc11edfce58e7dc48a95b7ecb16bef81dfec2c85199a9cf31 not found: ID does not exist"
Jul 22 16:41:11.171088 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.171003    2563 scope.go:117] "RemoveContainer" containerID="07d2090b31a1f6a5a9347ff6acd6c93dfe91a4911a29a37f00b91ae1d449d5b4"
Jul 22 16:41:11.171378 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:41:11.171361    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07d2090b31a1f6a5a9347ff6acd6c93dfe91a4911a29a37f00b91ae1d449d5b4\": container with ID starting with 07d2090b31a1f6a5a9347ff6acd6c93dfe91a4911a29a37f00b91ae1d449d5b4 not found: ID does not exist" containerID="07d2090b31a1f6a5a9347ff6acd6c93dfe91a4911a29a37f00b91ae1d449d5b4"
Jul 22 16:41:11.171465 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.171382    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07d2090b31a1f6a5a9347ff6acd6c93dfe91a4911a29a37f00b91ae1d449d5b4"} err="failed to get container status \"07d2090b31a1f6a5a9347ff6acd6c93dfe91a4911a29a37f00b91ae1d449d5b4\": rpc error: code = NotFound desc = could not find container \"07d2090b31a1f6a5a9347ff6acd6c93dfe91a4911a29a37f00b91ae1d449d5b4\": container with ID starting with 07d2090b31a1f6a5a9347ff6acd6c93dfe91a4911a29a37f00b91ae1d449d5b4 not found: ID does not exist"
Jul 22 16:41:11.172748 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.172729    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"]
Jul 22 16:41:11.175523 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.175496    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-0bf04dbf-openshift-default-848475bc8c-cbbcx"]
Jul 22 16:41:11.941029 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:11.940994    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:41:12.003469 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:12.003429    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:41:12.358895 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:12.358770    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:41:12.455002 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:12.454973    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4950bf09-fb48-44e4-95c2-dcb6ee2e895a" path="/var/lib/kubelet/pods/4950bf09-fb48-44e4-95c2-dcb6ee2e895a/volumes"
Jul 22 16:41:12.459847 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:12.459825    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:41:18.159050 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:18.159018    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:41:18.213764 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:18.213732    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:41:25.076224 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.076191    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:41:25.432582 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.432540    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"]
Jul 22 16:41:25.433060 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.433044    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="830286e7-c1a7-4037-85f4-6df13f755a0d" containerName="revoke-keys"
Jul 22 16:41:25.433136 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.433063    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="830286e7-c1a7-4037-85f4-6df13f755a0d" containerName="revoke-keys"
Jul 22 16:41:25.433136 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.433078    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4950bf09-fb48-44e4-95c2-dcb6ee2e895a" containerName="istio-proxy"
Jul 22 16:41:25.433136 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.433087    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4950bf09-fb48-44e4-95c2-dcb6ee2e895a" containerName="istio-proxy"
Jul 22 16:41:25.433136 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.433105    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e87ae49-4b64-4fa4-833e-457d29a6368d" containerName="maas-api"
Jul 22 16:41:25.433136 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.433114    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e87ae49-4b64-4fa4-833e-457d29a6368d" containerName="maas-api"
Jul 22 16:41:25.433136 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.433129    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4950bf09-fb48-44e4-95c2-dcb6ee2e895a" containerName="istio-proxy"
Jul 22 16:41:25.433136 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.433137    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4950bf09-fb48-44e4-95c2-dcb6ee2e895a" containerName="istio-proxy"
Jul 22 16:41:25.433346 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.433216    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4950bf09-fb48-44e4-95c2-dcb6ee2e895a" containerName="istio-proxy"
Jul 22 16:41:25.433346 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.433226    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e87ae49-4b64-4fa4-833e-457d29a6368d" containerName="maas-api"
Jul 22 16:41:25.433346 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.433236    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="830286e7-c1a7-4037-85f4-6df13f755a0d" containerName="revoke-keys"
Jul 22 16:41:25.433346 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.433247    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4950bf09-fb48-44e4-95c2-dcb6ee2e895a" containerName="istio-proxy"
Jul 22 16:41:25.435699 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.435681    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:25.441557 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.441537    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 22 16:41:25.441699 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.441669    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-4972458a-openshift-default-dockercfg-bjj6d\""
Jul 22 16:41:25.461900 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.461873    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"]
Jul 22 16:41:25.482605 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.482577    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab7230ef-b7ad-4c53-994a-18d1cc865404-istio-podinfo\") pod \"e2e-mt-4972458a-openshift-default-67fd477847-fltk2\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") " pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:25.482790 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.482609    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab7230ef-b7ad-4c53-994a-18d1cc865404-workload-socket\") pod \"e2e-mt-4972458a-openshift-default-67fd477847-fltk2\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") " pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:25.482790 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.482630    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntlkb\" (UniqueName: \"kubernetes.io/projected/ab7230ef-b7ad-4c53-994a-18d1cc865404-kube-api-access-ntlkb\") pod \"e2e-mt-4972458a-openshift-default-67fd477847-fltk2\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") " pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:25.482790 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.482653    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab7230ef-b7ad-4c53-994a-18d1cc865404-istio-envoy\") pod \"e2e-mt-4972458a-openshift-default-67fd477847-fltk2\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") " pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:25.482790 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.482745    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab7230ef-b7ad-4c53-994a-18d1cc865404-credential-socket\") pod \"e2e-mt-4972458a-openshift-default-67fd477847-fltk2\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") " pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:25.482790 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.482786    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab7230ef-b7ad-4c53-994a-18d1cc865404-istiod-ca-cert\") pod \"e2e-mt-4972458a-openshift-default-67fd477847-fltk2\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") " pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:25.483077 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.482895    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab7230ef-b7ad-4c53-994a-18d1cc865404-istio-data\") pod \"e2e-mt-4972458a-openshift-default-67fd477847-fltk2\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") " pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:25.483077 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.482924    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab7230ef-b7ad-4c53-994a-18d1cc865404-workload-certs\") pod \"e2e-mt-4972458a-openshift-default-67fd477847-fltk2\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") " pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:25.483077 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.482967    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab7230ef-b7ad-4c53-994a-18d1cc865404-istio-token\") pod \"e2e-mt-4972458a-openshift-default-67fd477847-fltk2\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") " pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:25.491375 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.491348    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:41:25.584312 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.584277    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab7230ef-b7ad-4c53-994a-18d1cc865404-istio-data\") pod \"e2e-mt-4972458a-openshift-default-67fd477847-fltk2\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") " pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:25.584312 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.584314    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab7230ef-b7ad-4c53-994a-18d1cc865404-workload-certs\") pod \"e2e-mt-4972458a-openshift-default-67fd477847-fltk2\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") " pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:25.584597 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.584341    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab7230ef-b7ad-4c53-994a-18d1cc865404-istio-token\") pod \"e2e-mt-4972458a-openshift-default-67fd477847-fltk2\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") " pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:25.584597 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.584368    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab7230ef-b7ad-4c53-994a-18d1cc865404-istio-podinfo\") pod \"e2e-mt-4972458a-openshift-default-67fd477847-fltk2\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") " pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:25.584597 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.584395    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab7230ef-b7ad-4c53-994a-18d1cc865404-workload-socket\") pod \"e2e-mt-4972458a-openshift-default-67fd477847-fltk2\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") " pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:25.584597 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.584424    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntlkb\" (UniqueName: \"kubernetes.io/projected/ab7230ef-b7ad-4c53-994a-18d1cc865404-kube-api-access-ntlkb\") pod \"e2e-mt-4972458a-openshift-default-67fd477847-fltk2\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") " pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:25.584597 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.584457    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab7230ef-b7ad-4c53-994a-18d1cc865404-istio-envoy\") pod \"e2e-mt-4972458a-openshift-default-67fd477847-fltk2\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") " pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:25.584597 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.584487    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab7230ef-b7ad-4c53-994a-18d1cc865404-credential-socket\") pod \"e2e-mt-4972458a-openshift-default-67fd477847-fltk2\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") " pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:25.584597 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.584514    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab7230ef-b7ad-4c53-994a-18d1cc865404-istiod-ca-cert\") pod \"e2e-mt-4972458a-openshift-default-67fd477847-fltk2\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") " pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:25.585013 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.584745    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab7230ef-b7ad-4c53-994a-18d1cc865404-workload-certs\") pod \"e2e-mt-4972458a-openshift-default-67fd477847-fltk2\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") " pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:25.585013 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.584802    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab7230ef-b7ad-4c53-994a-18d1cc865404-istio-data\") pod \"e2e-mt-4972458a-openshift-default-67fd477847-fltk2\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") " pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:25.585013 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.584927    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab7230ef-b7ad-4c53-994a-18d1cc865404-workload-socket\") pod \"e2e-mt-4972458a-openshift-default-67fd477847-fltk2\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") " pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:25.585183 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.585019    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab7230ef-b7ad-4c53-994a-18d1cc865404-credential-socket\") pod \"e2e-mt-4972458a-openshift-default-67fd477847-fltk2\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") " pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:25.585365 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.585343    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab7230ef-b7ad-4c53-994a-18d1cc865404-istiod-ca-cert\") pod \"e2e-mt-4972458a-openshift-default-67fd477847-fltk2\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") " pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:25.586708 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.586691    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab7230ef-b7ad-4c53-994a-18d1cc865404-istio-envoy\") pod \"e2e-mt-4972458a-openshift-default-67fd477847-fltk2\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") " pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:25.586827 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.586809    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab7230ef-b7ad-4c53-994a-18d1cc865404-istio-podinfo\") pod \"e2e-mt-4972458a-openshift-default-67fd477847-fltk2\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") " pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:25.605007 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.604982    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab7230ef-b7ad-4c53-994a-18d1cc865404-istio-token\") pod \"e2e-mt-4972458a-openshift-default-67fd477847-fltk2\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") " pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:25.606920 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.606900    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntlkb\" (UniqueName: \"kubernetes.io/projected/ab7230ef-b7ad-4c53-994a-18d1cc865404-kube-api-access-ntlkb\") pod \"e2e-mt-4972458a-openshift-default-67fd477847-fltk2\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") " pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:25.612249 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.612223    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:41:25.746171 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.746081    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:25.892700 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.892676    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"]
Jul 22 16:41:25.894603 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:41:25.894574    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab7230ef_b7ad_4c53_994a_18d1cc865404.slice/crio-a64342328865c3dc8be384d59683ae68895d7ae03861a718714a4bb3b251dba6 WatchSource:0}: Error finding container a64342328865c3dc8be384d59683ae68895d7ae03861a718714a4bb3b251dba6: Status 404 returned error can't find the container with id a64342328865c3dc8be384d59683ae68895d7ae03861a718714a4bb3b251dba6
Jul 22 16:41:25.896736 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.896701    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:41:25.896833 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.896771    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:41:25.896833 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:25.896800    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:41:26.218357 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:26.218318    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2" event={"ID":"ab7230ef-b7ad-4c53-994a-18d1cc865404","Type":"ContainerStarted","Data":"3237c20b76786b3b89dbc2bb24f2b672626184fde2fa32fb676439aca8f5ef77"}
Jul 22 16:41:26.218732 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:26.218383    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2" event={"ID":"ab7230ef-b7ad-4c53-994a-18d1cc865404","Type":"ContainerStarted","Data":"a64342328865c3dc8be384d59683ae68895d7ae03861a718714a4bb3b251dba6"}
Jul 22 16:41:26.255103 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:26.255049    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2" podStartSLOduration=1.25503162 podStartE2EDuration="1.25503162s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:41:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:41:26.252664579 +0000 UTC m=+2046.383134582" watchObservedRunningTime="2026-07-22 16:41:26.25503162 +0000 UTC m=+2046.385501622"
Jul 22 16:41:26.746464 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:26.746425    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:26.751498 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:26.751472    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:27.223293 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:27.223263    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:27.224263 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:27.224244    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:28.200295 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:28.200259    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:41:28.296943 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:28.296914    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4972458a-5449ff67f7-brwrq"]
Jul 22 16:41:28.299303 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:28.299277    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4972458a-5449ff67f7-brwrq"
Jul 22 16:41:28.305357 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:28.305334    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-4972458a-serving-cert\""
Jul 22 16:41:28.305674 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:28.305657    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 16:41:28.306760 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:28.306732    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 22 16:41:28.306866 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:28.306790    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vghv2\""
Jul 22 16:41:28.315496 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:28.315476    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:41:28.321008 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:28.320987    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4972458a-5449ff67f7-brwrq"]
Jul 22 16:41:28.409196 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:28.409157    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbcb14f4-3198-4454-932d-24beb5573f3e-openshift-service-ca\") pod \"maas-api-e2e-mt-4972458a-5449ff67f7-brwrq\" (UID: \"cbcb14f4-3198-4454-932d-24beb5573f3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4972458a-5449ff67f7-brwrq"
Jul 22 16:41:28.409403 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:28.409202    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knm4j\" (UniqueName: \"kubernetes.io/projected/cbcb14f4-3198-4454-932d-24beb5573f3e-kube-api-access-knm4j\") pod \"maas-api-e2e-mt-4972458a-5449ff67f7-brwrq\" (UID: \"cbcb14f4-3198-4454-932d-24beb5573f3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4972458a-5449ff67f7-brwrq"
Jul 22 16:41:28.409403 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:28.409254    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbcb14f4-3198-4454-932d-24beb5573f3e-maas-api-tls\") pod \"maas-api-e2e-mt-4972458a-5449ff67f7-brwrq\" (UID: \"cbcb14f4-3198-4454-932d-24beb5573f3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4972458a-5449ff67f7-brwrq"
Jul 22 16:41:28.424474 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:28.424443    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:41:28.510813 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:28.510707    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbcb14f4-3198-4454-932d-24beb5573f3e-openshift-service-ca\") pod \"maas-api-e2e-mt-4972458a-5449ff67f7-brwrq\" (UID: \"cbcb14f4-3198-4454-932d-24beb5573f3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4972458a-5449ff67f7-brwrq"
Jul 22 16:41:28.510813 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:28.510751    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knm4j\" (UniqueName: \"kubernetes.io/projected/cbcb14f4-3198-4454-932d-24beb5573f3e-kube-api-access-knm4j\") pod \"maas-api-e2e-mt-4972458a-5449ff67f7-brwrq\" (UID: \"cbcb14f4-3198-4454-932d-24beb5573f3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4972458a-5449ff67f7-brwrq"
Jul 22 16:41:28.510813 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:28.510798    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbcb14f4-3198-4454-932d-24beb5573f3e-maas-api-tls\") pod \"maas-api-e2e-mt-4972458a-5449ff67f7-brwrq\" (UID: \"cbcb14f4-3198-4454-932d-24beb5573f3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4972458a-5449ff67f7-brwrq"
Jul 22 16:41:28.511204 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:41:28.510932    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-4972458a-serving-cert: secret "maas-api-e2e-mt-4972458a-serving-cert" not found
Jul 22 16:41:28.511204 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:41:28.511035    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbcb14f4-3198-4454-932d-24beb5573f3e-maas-api-tls podName:cbcb14f4-3198-4454-932d-24beb5573f3e nodeName:}" failed. No retries permitted until 2026-07-22 16:41:29.011010604 +0000 UTC m=+2049.141480592 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cbcb14f4-3198-4454-932d-24beb5573f3e-maas-api-tls") pod "maas-api-e2e-mt-4972458a-5449ff67f7-brwrq" (UID: "cbcb14f4-3198-4454-932d-24beb5573f3e") : secret "maas-api-e2e-mt-4972458a-serving-cert" not found
Jul 22 16:41:28.511456 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:28.511434    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbcb14f4-3198-4454-932d-24beb5573f3e-openshift-service-ca\") pod \"maas-api-e2e-mt-4972458a-5449ff67f7-brwrq\" (UID: \"cbcb14f4-3198-4454-932d-24beb5573f3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4972458a-5449ff67f7-brwrq"
Jul 22 16:41:28.521549 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:28.521523    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knm4j\" (UniqueName: \"kubernetes.io/projected/cbcb14f4-3198-4454-932d-24beb5573f3e-kube-api-access-knm4j\") pod \"maas-api-e2e-mt-4972458a-5449ff67f7-brwrq\" (UID: \"cbcb14f4-3198-4454-932d-24beb5573f3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4972458a-5449ff67f7-brwrq"
Jul 22 16:41:28.923282 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:28.923191    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:41:29.015956 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:29.015917    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbcb14f4-3198-4454-932d-24beb5573f3e-maas-api-tls\") pod \"maas-api-e2e-mt-4972458a-5449ff67f7-brwrq\" (UID: \"cbcb14f4-3198-4454-932d-24beb5573f3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4972458a-5449ff67f7-brwrq"
Jul 22 16:41:29.018459 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:29.018434    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbcb14f4-3198-4454-932d-24beb5573f3e-maas-api-tls\") pod \"maas-api-e2e-mt-4972458a-5449ff67f7-brwrq\" (UID: \"cbcb14f4-3198-4454-932d-24beb5573f3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4972458a-5449ff67f7-brwrq"
Jul 22 16:41:29.022017 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:29.021986    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:41:29.210002 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:29.209952    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4972458a-5449ff67f7-brwrq"
Jul 22 16:41:29.347663 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:29.347630    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4972458a-5449ff67f7-brwrq"]
Jul 22 16:41:29.349821 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:41:29.349780    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbcb14f4_3198_4454_932d_24beb5573f3e.slice/crio-4aecf255b5a5708989c4d7f786564656f735344046af9bd882cead549949944b WatchSource:0}: Error finding container 4aecf255b5a5708989c4d7f786564656f735344046af9bd882cead549949944b: Status 404 returned error can't find the container with id 4aecf255b5a5708989c4d7f786564656f735344046af9bd882cead549949944b
Jul 22 16:41:29.351286 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:29.351266    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 16:41:30.239991 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:30.239880    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4972458a-5449ff67f7-brwrq" event={"ID":"cbcb14f4-3198-4454-932d-24beb5573f3e","Type":"ContainerStarted","Data":"a22b902a4871cac63fcf939df01e7335b311622bd000c12c5028c18977f734d4"}
Jul 22 16:41:30.239991 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:30.239934    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4972458a-5449ff67f7-brwrq" event={"ID":"cbcb14f4-3198-4454-932d-24beb5573f3e","Type":"ContainerStarted","Data":"4aecf255b5a5708989c4d7f786564656f735344046af9bd882cead549949944b"}
Jul 22 16:41:30.239991 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:30.239962    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4972458a-5449ff67f7-brwrq"
Jul 22 16:41:30.265975 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:30.265915    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4972458a-5449ff67f7-brwrq" podStartSLOduration=1.6786751949999998 podStartE2EDuration="2.265898559s" totalImagesPullingTime="587.223364ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:41:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:41:30.263311262 +0000 UTC m=+2050.393781265" watchObservedRunningTime="2026-07-22 16:41:30.265898559 +0000 UTC m=+2050.396368587"
Jul 22 16:41:30.953088 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:30.953053    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:41:32.703112 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:32.703077    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:41:36.250160 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:36.250129    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4972458a-5449ff67f7-brwrq"
Jul 22 16:41:41.294026 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:41.293991    2563 generic.go:356] "Generic (PLEG): container finished" podID="ab7230ef-b7ad-4c53-994a-18d1cc865404" containerID="3237c20b76786b3b89dbc2bb24f2b672626184fde2fa32fb676439aca8f5ef77" exitCode=0
Jul 22 16:41:41.294458 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:41.294075    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2" event={"ID":"ab7230ef-b7ad-4c53-994a-18d1cc865404","Type":"ContainerDied","Data":"3237c20b76786b3b89dbc2bb24f2b672626184fde2fa32fb676439aca8f5ef77"}
Jul 22 16:41:41.294504 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:41.294483    2563 scope.go:117] "RemoveContainer" containerID="3237c20b76786b3b89dbc2bb24f2b672626184fde2fa32fb676439aca8f5ef77"
Jul 22 16:41:41.295499 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:41.295471    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:41:41.295628 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:41.295553    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:41:41.295628 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:41.295582    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:41:42.223762 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:42.223697    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:42.300932 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:42.300889    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2" event={"ID":"ab7230ef-b7ad-4c53-994a-18d1cc865404","Type":"ContainerStarted","Data":"7a11dcf5866536c3208d7657d3d359879e7559216fb3cbd648d60e097d9c734a"}
Jul 22 16:41:42.746426 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:42.746395    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:42.747708 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:42.747680    2563 patch_prober.go:28] interesting pod/e2e-mt-4972458a-openshift-default-67fd477847-fltk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 22 16:41:42.747845 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:42.747734    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2" podUID="ab7230ef-b7ad-4c53-994a-18d1cc865404" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 22 16:41:43.747477 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:43.747438    2563 patch_prober.go:28] interesting pod/e2e-mt-4972458a-openshift-default-67fd477847-fltk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 22 16:41:43.747909 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:43.747500    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2" podUID="ab7230ef-b7ad-4c53-994a-18d1cc865404" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 22 16:41:44.747526 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:44.747488    2563 patch_prober.go:28] interesting pod/e2e-mt-4972458a-openshift-default-67fd477847-fltk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 22 16:41:44.747987 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:44.747545    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2" podUID="ab7230ef-b7ad-4c53-994a-18d1cc865404" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 22 16:41:45.747222 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:45.747185    2563 patch_prober.go:28] interesting pod/e2e-mt-4972458a-openshift-default-67fd477847-fltk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 22 16:41:45.747412 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:45.747241    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2" podUID="ab7230ef-b7ad-4c53-994a-18d1cc865404" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 22 16:41:46.750415 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:46.750378    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:46.750818 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:46.750649    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:46.751295 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:46.751275    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:41:49.945832 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:49.945801    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp"]
Jul 22 16:41:49.948333 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:49.948306    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp"
Jul 22 16:41:49.952500 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:49.952469    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-4972458a\"/\"kube-root-ca.crt\""
Jul 22 16:41:49.952500 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:49.952476    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-4972458a\"/\"e2e-lifecycle-model-4972458a-kserve-self-signed-certs\""
Jul 22 16:41:49.952689 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:49.952503    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-4972458a\"/\"openshift-service-ca.crt\""
Jul 22 16:41:49.952689 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:49.952546    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-4972458a\"/\"default-dockercfg-2qz6t\""
Jul 22 16:41:49.959286 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:49.959262    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp"]
Jul 22 16:41:49.991952 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:49.991921    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:41:50.110770 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:50.110733    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nr5v\" (UniqueName: \"kubernetes.io/projected/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-kube-api-access-9nr5v\") pod \"e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp\" (UID: \"dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9\") " pod="ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp"
Jul 22 16:41:50.111005 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:50.110783    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-tls-certs\") pod \"e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp\" (UID: \"dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9\") " pod="ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp"
Jul 22 16:41:50.111005 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:50.110814    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-dshm\") pod \"e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp\" (UID: \"dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9\") " pod="ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp"
Jul 22 16:41:50.111005 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:50.110833    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-tmp-dir\") pod \"e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp\" (UID: \"dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9\") " pod="ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp"
Jul 22 16:41:50.111005 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:50.110928    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-model-cache\") pod \"e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp\" (UID: \"dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9\") " pod="ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp"
Jul 22 16:41:50.111005 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:50.110984    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-home\") pod \"e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp\" (UID: \"dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9\") " pod="ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp"
Jul 22 16:41:50.211607 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:50.211511    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-tls-certs\") pod \"e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp\" (UID: \"dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9\") " pod="ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp"
Jul 22 16:41:50.211607 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:50.211555    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-dshm\") pod \"e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp\" (UID: \"dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9\") " pod="ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp"
Jul 22 16:41:50.211607 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:50.211573    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-tmp-dir\") pod \"e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp\" (UID: \"dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9\") " pod="ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp"
Jul 22 16:41:50.211607 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:50.211609    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-model-cache\") pod \"e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp\" (UID: \"dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9\") " pod="ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp"
Jul 22 16:41:50.211994 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:50.211636    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-home\") pod \"e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp\" (UID: \"dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9\") " pod="ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp"
Jul 22 16:41:50.211994 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:50.211674    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nr5v\" (UniqueName: \"kubernetes.io/projected/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-kube-api-access-9nr5v\") pod \"e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp\" (UID: \"dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9\") " pod="ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp"
Jul 22 16:41:50.212108 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:50.212062    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-tmp-dir\") pod \"e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp\" (UID: \"dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9\") " pod="ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp"
Jul 22 16:41:50.212108 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:50.212075    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-model-cache\") pod \"e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp\" (UID: \"dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9\") " pod="ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp"
Jul 22 16:41:50.212184 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:50.212114    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-home\") pod \"e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp\" (UID: \"dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9\") " pod="ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp"
Jul 22 16:41:50.213996 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:50.213975    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-dshm\") pod \"e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp\" (UID: \"dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9\") " pod="ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp"
Jul 22 16:41:50.214153 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:50.214137    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-tls-certs\") pod \"e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp\" (UID: \"dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9\") " pod="ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp"
Jul 22 16:41:50.221457 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:50.221436    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nr5v\" (UniqueName: \"kubernetes.io/projected/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-kube-api-access-9nr5v\") pod \"e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp\" (UID: \"dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9\") " pod="ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp"
Jul 22 16:41:50.257982 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:50.257950    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp"
Jul 22 16:41:50.394951 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:50.394910    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp"]
Jul 22 16:41:50.398257 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:41:50.398228    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddad6e5b3_9506_4d96_a8dc_5f66ebe5c3d9.slice/crio-cbee61fbe7cec570efb94cc551603a6c066aac2ffc25e3ebbaaf0c74b8f7e943 WatchSource:0}: Error finding container cbee61fbe7cec570efb94cc551603a6c066aac2ffc25e3ebbaaf0c74b8f7e943: Status 404 returned error can't find the container with id cbee61fbe7cec570efb94cc551603a6c066aac2ffc25e3ebbaaf0c74b8f7e943
Jul 22 16:41:51.353504 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:51.353461    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp" event={"ID":"dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9","Type":"ContainerStarted","Data":"34c8a3c6cc786926745fb18acf95e1536b4880891e020bc97118ae8b5c5d4aa2"}
Jul 22 16:41:51.353504 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:51.353507    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp" event={"ID":"dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9","Type":"ContainerStarted","Data":"cbee61fbe7cec570efb94cc551603a6c066aac2ffc25e3ebbaaf0c74b8f7e943"}
Jul 22 16:41:51.382536 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:41:51.382482    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp" podStartSLOduration=2.382466329 podStartE2EDuration="2.382466329s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:41:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:41:51.38114877 +0000 UTC m=+2071.511618790" watchObservedRunningTime="2026-07-22 16:41:51.382466329 +0000 UTC m=+2071.512936331"
Jul 22 16:42:00.259127 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:00.259081    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp"
Jul 22 16:42:00.259750 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:00.259141    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp"
Jul 22 16:42:00.271682 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:00.271657    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp"
Jul 22 16:42:00.403737 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:00.403711    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp"
Jul 22 16:42:20.487237 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:20.487207    2563 scope.go:117] "RemoveContainer" containerID="02f8140c59db6de07c7fd32fdb13d791013869521e5c35ac5cff0095c03ee332"
Jul 22 16:42:26.639335 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:26.639296    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4972458a-5449ff67f7-brwrq"]
Jul 22 16:42:26.639772 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:26.639514    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4972458a-5449ff67f7-brwrq" podUID="cbcb14f4-3198-4454-932d-24beb5573f3e" containerName="maas-api" containerID="cri-o://a22b902a4871cac63fcf939df01e7335b311622bd000c12c5028c18977f734d4" gracePeriod=30
Jul 22 16:42:26.667257 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:26.667227    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:42:26.879262 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:26.879235    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4972458a-5449ff67f7-brwrq"
Jul 22 16:42:26.918541 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:26.918459    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-knm4j\" (UniqueName: \"kubernetes.io/projected/cbcb14f4-3198-4454-932d-24beb5573f3e-kube-api-access-knm4j\") pod \"cbcb14f4-3198-4454-932d-24beb5573f3e\" (UID: \"cbcb14f4-3198-4454-932d-24beb5573f3e\") "
Jul 22 16:42:26.918541 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:26.918516    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbcb14f4-3198-4454-932d-24beb5573f3e-openshift-service-ca\") pod \"cbcb14f4-3198-4454-932d-24beb5573f3e\" (UID: \"cbcb14f4-3198-4454-932d-24beb5573f3e\") "
Jul 22 16:42:26.918732 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:26.918611    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbcb14f4-3198-4454-932d-24beb5573f3e-maas-api-tls\") pod \"cbcb14f4-3198-4454-932d-24beb5573f3e\" (UID: \"cbcb14f4-3198-4454-932d-24beb5573f3e\") "
Jul 22 16:42:26.918922 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:26.918897    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbcb14f4-3198-4454-932d-24beb5573f3e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cbcb14f4-3198-4454-932d-24beb5573f3e" (UID: "cbcb14f4-3198-4454-932d-24beb5573f3e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 16:42:26.920779 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:26.920749    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbcb14f4-3198-4454-932d-24beb5573f3e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cbcb14f4-3198-4454-932d-24beb5573f3e" (UID: "cbcb14f4-3198-4454-932d-24beb5573f3e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:42:26.920913 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:26.920752    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbcb14f4-3198-4454-932d-24beb5573f3e-kube-api-access-knm4j" (OuterVolumeSpecName: "kube-api-access-knm4j") pod "cbcb14f4-3198-4454-932d-24beb5573f3e" (UID: "cbcb14f4-3198-4454-932d-24beb5573f3e"). InnerVolumeSpecName "kube-api-access-knm4j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:42:27.019664 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:27.019626    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-knm4j\" (UniqueName: \"kubernetes.io/projected/cbcb14f4-3198-4454-932d-24beb5573f3e-kube-api-access-knm4j\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:42:27.019664 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:27.019653    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbcb14f4-3198-4454-932d-24beb5573f3e-openshift-service-ca\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:42:27.019664 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:27.019662    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbcb14f4-3198-4454-932d-24beb5573f3e-maas-api-tls\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:42:27.520189 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:27.520155    2563 generic.go:356] "Generic (PLEG): container finished" podID="cbcb14f4-3198-4454-932d-24beb5573f3e" containerID="a22b902a4871cac63fcf939df01e7335b311622bd000c12c5028c18977f734d4" exitCode=0
Jul 22 16:42:27.520407 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:27.520229    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4972458a-5449ff67f7-brwrq"
Jul 22 16:42:27.520407 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:27.520236    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4972458a-5449ff67f7-brwrq" event={"ID":"cbcb14f4-3198-4454-932d-24beb5573f3e","Type":"ContainerDied","Data":"a22b902a4871cac63fcf939df01e7335b311622bd000c12c5028c18977f734d4"}
Jul 22 16:42:27.520407 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:27.520283    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4972458a-5449ff67f7-brwrq" event={"ID":"cbcb14f4-3198-4454-932d-24beb5573f3e","Type":"ContainerDied","Data":"4aecf255b5a5708989c4d7f786564656f735344046af9bd882cead549949944b"}
Jul 22 16:42:27.520407 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:27.520304    2563 scope.go:117] "RemoveContainer" containerID="a22b902a4871cac63fcf939df01e7335b311622bd000c12c5028c18977f734d4"
Jul 22 16:42:27.529104 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:27.529085    2563 scope.go:117] "RemoveContainer" containerID="a22b902a4871cac63fcf939df01e7335b311622bd000c12c5028c18977f734d4"
Jul 22 16:42:27.529353 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:42:27.529336    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a22b902a4871cac63fcf939df01e7335b311622bd000c12c5028c18977f734d4\": container with ID starting with a22b902a4871cac63fcf939df01e7335b311622bd000c12c5028c18977f734d4 not found: ID does not exist" containerID="a22b902a4871cac63fcf939df01e7335b311622bd000c12c5028c18977f734d4"
Jul 22 16:42:27.529405 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:27.529361    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a22b902a4871cac63fcf939df01e7335b311622bd000c12c5028c18977f734d4"} err="failed to get container status \"a22b902a4871cac63fcf939df01e7335b311622bd000c12c5028c18977f734d4\": rpc error: code = NotFound desc = could not find container \"a22b902a4871cac63fcf939df01e7335b311622bd000c12c5028c18977f734d4\": container with ID starting with a22b902a4871cac63fcf939df01e7335b311622bd000c12c5028c18977f734d4 not found: ID does not exist"
Jul 22 16:42:27.545132 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:27.545107    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4972458a-5449ff67f7-brwrq"]
Jul 22 16:42:27.549123 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:27.549101    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4972458a-5449ff67f7-brwrq"]
Jul 22 16:42:28.448606 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:28.448564    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbcb14f4-3198-4454-932d-24beb5573f3e" path="/var/lib/kubelet/pods/cbcb14f4-3198-4454-932d-24beb5573f3e/volumes"
Jul 22 16:42:31.258291 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:31.258220    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:42:33.725582 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:33.725549    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:42:35.452761 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:35.452725    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:42:38.716375 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:38.716335    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp"]
Jul 22 16:42:38.759989 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:38.759953    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:42:49.993646 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:49.993607    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:42:50.096129 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:50.096095    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:42:53.736455 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:53.736416    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp" podUID="dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9" containerName="main" containerID="cri-o://34c8a3c6cc786926745fb18acf95e1536b4880891e020bc97118ae8b5c5d4aa2" gracePeriod=45
Jul 22 16:42:53.992336 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:53.992270    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp"
Jul 22 16:42:54.160379 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:54.160348    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-dshm\") pod \"dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9\" (UID: \"dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9\") "
Jul 22 16:42:54.160588 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:54.160447    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-model-cache\") pod \"dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9\" (UID: \"dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9\") "
Jul 22 16:42:54.160588 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:54.160471    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9nr5v\" (UniqueName: \"kubernetes.io/projected/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-kube-api-access-9nr5v\") pod \"dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9\" (UID: \"dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9\") "
Jul 22 16:42:54.160588 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:54.160506    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-tls-certs\") pod \"dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9\" (UID: \"dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9\") "
Jul 22 16:42:54.160588 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:54.160522    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-home\") pod \"dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9\" (UID: \"dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9\") "
Jul 22 16:42:54.160588 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:54.160542    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-tmp-dir\") pod \"dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9\" (UID: \"dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9\") "
Jul 22 16:42:54.160908 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:54.160807    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-home" (OuterVolumeSpecName: "home") pod "dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9" (UID: "dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:42:54.160908 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:54.160822    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-model-cache" (OuterVolumeSpecName: "model-cache") pod "dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9" (UID: "dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:42:54.160908 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:54.160843    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9" (UID: "dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:42:54.162755 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:54.162721    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-dshm" (OuterVolumeSpecName: "dshm") pod "dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9" (UID: "dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:42:54.162918 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:54.162828    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-kube-api-access-9nr5v" (OuterVolumeSpecName: "kube-api-access-9nr5v") pod "dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9" (UID: "dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9"). InnerVolumeSpecName "kube-api-access-9nr5v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:42:54.162918 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:54.162844    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9" (UID: "dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:42:54.261622 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:54.261533    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-model-cache\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:42:54.261622 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:54.261563    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9nr5v\" (UniqueName: \"kubernetes.io/projected/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-kube-api-access-9nr5v\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:42:54.261622 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:54.261574    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-tls-certs\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:42:54.261622 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:54.261582    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-home\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:42:54.261622 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:54.261590    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-tmp-dir\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:42:54.261622 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:54.261597    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9-dshm\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:42:54.643803 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:54.643719    2563 generic.go:356] "Generic (PLEG): container finished" podID="dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9" containerID="34c8a3c6cc786926745fb18acf95e1536b4880891e020bc97118ae8b5c5d4aa2" exitCode=0
Jul 22 16:42:54.643803 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:54.643765    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp"
Jul 22 16:42:54.643803 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:54.643776    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp" event={"ID":"dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9","Type":"ContainerDied","Data":"34c8a3c6cc786926745fb18acf95e1536b4880891e020bc97118ae8b5c5d4aa2"}
Jul 22 16:42:54.644068 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:54.643805    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp" event={"ID":"dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9","Type":"ContainerDied","Data":"cbee61fbe7cec570efb94cc551603a6c066aac2ffc25e3ebbaaf0c74b8f7e943"}
Jul 22 16:42:54.644068 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:54.643824    2563 scope.go:117] "RemoveContainer" containerID="34c8a3c6cc786926745fb18acf95e1536b4880891e020bc97118ae8b5c5d4aa2"
Jul 22 16:42:54.652109 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:54.652096    2563 scope.go:117] "RemoveContainer" containerID="34c8a3c6cc786926745fb18acf95e1536b4880891e020bc97118ae8b5c5d4aa2"
Jul 22 16:42:54.652392 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:42:54.652371    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34c8a3c6cc786926745fb18acf95e1536b4880891e020bc97118ae8b5c5d4aa2\": container with ID starting with 34c8a3c6cc786926745fb18acf95e1536b4880891e020bc97118ae8b5c5d4aa2 not found: ID does not exist" containerID="34c8a3c6cc786926745fb18acf95e1536b4880891e020bc97118ae8b5c5d4aa2"
Jul 22 16:42:54.652495 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:54.652398    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34c8a3c6cc786926745fb18acf95e1536b4880891e020bc97118ae8b5c5d4aa2"} err="failed to get container status \"34c8a3c6cc786926745fb18acf95e1536b4880891e020bc97118ae8b5c5d4aa2\": rpc error: code = NotFound desc = could not find container \"34c8a3c6cc786926745fb18acf95e1536b4880891e020bc97118ae8b5c5d4aa2\": container with ID starting with 34c8a3c6cc786926745fb18acf95e1536b4880891e020bc97118ae8b5c5d4aa2 not found: ID does not exist"
Jul 22 16:42:54.662825 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:54.662799    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp"]
Jul 22 16:42:54.667116 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:54.667091    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-4972458a/e2e-lifecycle-model-4972458a-kserve-9674bcc7f-d8nzp"]
Jul 22 16:42:56.447639 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:42:56.447600    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9" path="/var/lib/kubelet/pods/dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9/volumes"
Jul 22 16:43:15.131541 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:15.131449    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:43:16.024330 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:16.024299    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:43:20.504692 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:20.504658    2563 scope.go:117] "RemoveContainer" containerID="8069d9e535ac969dec6ac8b7a0653e360034b1f4a3dd32a63a919296d58c0035"
Jul 22 16:43:21.749255 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:21.749215    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:43:21.815948 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:21.815914    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:43:21.998237 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:21.998205    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"]
Jul 22 16:43:21.998464 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:21.998420    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2" podUID="ab7230ef-b7ad-4c53-994a-18d1cc865404" containerName="istio-proxy" containerID="cri-o://7a11dcf5866536c3208d7657d3d359879e7559216fb3cbd648d60e097d9c734a" gracePeriod=30
Jul 22 16:43:22.026140 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:22.026050    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:43:22.251051 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:22.251017    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:43:23.342788 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:23.342751    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:43:23.448438 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:23.448404    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:43:23.545132 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:23.545101    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:43:24.242868 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:24.242814    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:43:24.343135 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:24.343103    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:43:24.446084 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:24.446047    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:43:25.831609 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:25.831569    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:43:27.252400 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.252374    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:43:27.337880 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.337758    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab7230ef-b7ad-4c53-994a-18d1cc865404-istio-token\") pod \"ab7230ef-b7ad-4c53-994a-18d1cc865404\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") "
Jul 22 16:43:27.337880 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.337805    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab7230ef-b7ad-4c53-994a-18d1cc865404-istio-envoy\") pod \"ab7230ef-b7ad-4c53-994a-18d1cc865404\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") "
Jul 22 16:43:27.338122 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.337894    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab7230ef-b7ad-4c53-994a-18d1cc865404-credential-socket\") pod \"ab7230ef-b7ad-4c53-994a-18d1cc865404\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") "
Jul 22 16:43:27.338122 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.337929    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab7230ef-b7ad-4c53-994a-18d1cc865404-istio-podinfo\") pod \"ab7230ef-b7ad-4c53-994a-18d1cc865404\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") "
Jul 22 16:43:27.338122 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.337956    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ntlkb\" (UniqueName: \"kubernetes.io/projected/ab7230ef-b7ad-4c53-994a-18d1cc865404-kube-api-access-ntlkb\") pod \"ab7230ef-b7ad-4c53-994a-18d1cc865404\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") "
Jul 22 16:43:27.338122 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.337977    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab7230ef-b7ad-4c53-994a-18d1cc865404-workload-socket\") pod \"ab7230ef-b7ad-4c53-994a-18d1cc865404\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") "
Jul 22 16:43:27.338122 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.338001    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab7230ef-b7ad-4c53-994a-18d1cc865404-istiod-ca-cert\") pod \"ab7230ef-b7ad-4c53-994a-18d1cc865404\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") "
Jul 22 16:43:27.338122 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.338025    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab7230ef-b7ad-4c53-994a-18d1cc865404-istio-data\") pod \"ab7230ef-b7ad-4c53-994a-18d1cc865404\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") "
Jul 22 16:43:27.338122 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.338047    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab7230ef-b7ad-4c53-994a-18d1cc865404-workload-certs\") pod \"ab7230ef-b7ad-4c53-994a-18d1cc865404\" (UID: \"ab7230ef-b7ad-4c53-994a-18d1cc865404\") "
Jul 22 16:43:27.338471 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.338257    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab7230ef-b7ad-4c53-994a-18d1cc865404-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ab7230ef-b7ad-4c53-994a-18d1cc865404" (UID: "ab7230ef-b7ad-4c53-994a-18d1cc865404"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:43:27.338471 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.338430    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab7230ef-b7ad-4c53-994a-18d1cc865404-credential-socket\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:43:27.338579 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.338489    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab7230ef-b7ad-4c53-994a-18d1cc865404-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ab7230ef-b7ad-4c53-994a-18d1cc865404" (UID: "ab7230ef-b7ad-4c53-994a-18d1cc865404"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:43:27.338579 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.338494    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab7230ef-b7ad-4c53-994a-18d1cc865404-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ab7230ef-b7ad-4c53-994a-18d1cc865404" (UID: "ab7230ef-b7ad-4c53-994a-18d1cc865404"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 16:43:27.338692 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.338555    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab7230ef-b7ad-4c53-994a-18d1cc865404-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ab7230ef-b7ad-4c53-994a-18d1cc865404" (UID: "ab7230ef-b7ad-4c53-994a-18d1cc865404"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:43:27.339155 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.339133    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab7230ef-b7ad-4c53-994a-18d1cc865404-istio-data" (OuterVolumeSpecName: "istio-data") pod "ab7230ef-b7ad-4c53-994a-18d1cc865404" (UID: "ab7230ef-b7ad-4c53-994a-18d1cc865404"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:43:27.340140 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.340116    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab7230ef-b7ad-4c53-994a-18d1cc865404-kube-api-access-ntlkb" (OuterVolumeSpecName: "kube-api-access-ntlkb") pod "ab7230ef-b7ad-4c53-994a-18d1cc865404" (UID: "ab7230ef-b7ad-4c53-994a-18d1cc865404"). InnerVolumeSpecName "kube-api-access-ntlkb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:43:27.340227 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.340144    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab7230ef-b7ad-4c53-994a-18d1cc865404-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ab7230ef-b7ad-4c53-994a-18d1cc865404" (UID: "ab7230ef-b7ad-4c53-994a-18d1cc865404"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:43:27.340415 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.340390    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ab7230ef-b7ad-4c53-994a-18d1cc865404-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ab7230ef-b7ad-4c53-994a-18d1cc865404" (UID: "ab7230ef-b7ad-4c53-994a-18d1cc865404"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 16:43:27.340517 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.340393    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab7230ef-b7ad-4c53-994a-18d1cc865404-istio-token" (OuterVolumeSpecName: "istio-token") pod "ab7230ef-b7ad-4c53-994a-18d1cc865404" (UID: "ab7230ef-b7ad-4c53-994a-18d1cc865404"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:43:27.439770 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.439730    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab7230ef-b7ad-4c53-994a-18d1cc865404-istio-token\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:43:27.439770 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.439768    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab7230ef-b7ad-4c53-994a-18d1cc865404-istio-envoy\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:43:27.439770 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.439780    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab7230ef-b7ad-4c53-994a-18d1cc865404-istio-podinfo\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:43:27.440072 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.439793    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ntlkb\" (UniqueName: \"kubernetes.io/projected/ab7230ef-b7ad-4c53-994a-18d1cc865404-kube-api-access-ntlkb\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:43:27.440072 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.439806    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab7230ef-b7ad-4c53-994a-18d1cc865404-workload-socket\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:43:27.440072 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.439821    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab7230ef-b7ad-4c53-994a-18d1cc865404-istiod-ca-cert\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:43:27.440072 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.439833    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab7230ef-b7ad-4c53-994a-18d1cc865404-istio-data\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:43:27.440072 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.439846    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab7230ef-b7ad-4c53-994a-18d1cc865404-workload-certs\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:43:27.792678 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.792637    2563 generic.go:356] "Generic (PLEG): container finished" podID="ab7230ef-b7ad-4c53-994a-18d1cc865404" containerID="7a11dcf5866536c3208d7657d3d359879e7559216fb3cbd648d60e097d9c734a" exitCode=0
Jul 22 16:43:27.792884 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.792693    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"
Jul 22 16:43:27.792884 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.792722    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2" event={"ID":"ab7230ef-b7ad-4c53-994a-18d1cc865404","Type":"ContainerDied","Data":"7a11dcf5866536c3208d7657d3d359879e7559216fb3cbd648d60e097d9c734a"}
Jul 22 16:43:27.792884 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.792788    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2" event={"ID":"ab7230ef-b7ad-4c53-994a-18d1cc865404","Type":"ContainerDied","Data":"a64342328865c3dc8be384d59683ae68895d7ae03861a718714a4bb3b251dba6"}
Jul 22 16:43:27.792884 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.792823    2563 scope.go:117] "RemoveContainer" containerID="7a11dcf5866536c3208d7657d3d359879e7559216fb3cbd648d60e097d9c734a"
Jul 22 16:43:27.802257 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.802239    2563 scope.go:117] "RemoveContainer" containerID="3237c20b76786b3b89dbc2bb24f2b672626184fde2fa32fb676439aca8f5ef77"
Jul 22 16:43:27.810460 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.810440    2563 scope.go:117] "RemoveContainer" containerID="7a11dcf5866536c3208d7657d3d359879e7559216fb3cbd648d60e097d9c734a"
Jul 22 16:43:27.810734 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:43:27.810714    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a11dcf5866536c3208d7657d3d359879e7559216fb3cbd648d60e097d9c734a\": container with ID starting with 7a11dcf5866536c3208d7657d3d359879e7559216fb3cbd648d60e097d9c734a not found: ID does not exist" containerID="7a11dcf5866536c3208d7657d3d359879e7559216fb3cbd648d60e097d9c734a"
Jul 22 16:43:27.810784 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.810744    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a11dcf5866536c3208d7657d3d359879e7559216fb3cbd648d60e097d9c734a"} err="failed to get container status \"7a11dcf5866536c3208d7657d3d359879e7559216fb3cbd648d60e097d9c734a\": rpc error: code = NotFound desc = could not find container \"7a11dcf5866536c3208d7657d3d359879e7559216fb3cbd648d60e097d9c734a\": container with ID starting with 7a11dcf5866536c3208d7657d3d359879e7559216fb3cbd648d60e097d9c734a not found: ID does not exist"
Jul 22 16:43:27.810784 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.810764    2563 scope.go:117] "RemoveContainer" containerID="3237c20b76786b3b89dbc2bb24f2b672626184fde2fa32fb676439aca8f5ef77"
Jul 22 16:43:27.811147 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:43:27.811127    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3237c20b76786b3b89dbc2bb24f2b672626184fde2fa32fb676439aca8f5ef77\": container with ID starting with 3237c20b76786b3b89dbc2bb24f2b672626184fde2fa32fb676439aca8f5ef77 not found: ID does not exist" containerID="3237c20b76786b3b89dbc2bb24f2b672626184fde2fa32fb676439aca8f5ef77"
Jul 22 16:43:27.811194 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.811157    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3237c20b76786b3b89dbc2bb24f2b672626184fde2fa32fb676439aca8f5ef77"} err="failed to get container status \"3237c20b76786b3b89dbc2bb24f2b672626184fde2fa32fb676439aca8f5ef77\": rpc error: code = NotFound desc = could not find container \"3237c20b76786b3b89dbc2bb24f2b672626184fde2fa32fb676439aca8f5ef77\": container with ID starting with 3237c20b76786b3b89dbc2bb24f2b672626184fde2fa32fb676439aca8f5ef77 not found: ID does not exist"
Jul 22 16:43:27.819291 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.819265    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"]
Jul 22 16:43:27.823068 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:27.823045    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-4972458a-openshift-default-67fd477847-fltk2"]
Jul 22 16:43:28.448052 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:28.448018    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab7230ef-b7ad-4c53-994a-18d1cc865404" path="/var/lib/kubelet/pods/ab7230ef-b7ad-4c53-994a-18d1cc865404/volumes"
Jul 22 16:43:31.662218 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:31.662179    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:43:31.762615 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:31.762580    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:43:32.353188 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:32.353150    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:43:38.460468 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:38.460436    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:43:38.569642 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:38.569613    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:43:38.660693 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:38.660655    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:43:49.179306 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:49.179269    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:43:49.521485 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:43:49.521394    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:44:10.160482 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:10.160444    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:44:10.511657 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:10.511625    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:44:16.960188 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:16.960152    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:44:17.063192 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.063156    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:44:17.320132 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.320042    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:44:17.707141 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.707112    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"]
Jul 22 16:44:17.707596 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.707580    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbcb14f4-3198-4454-932d-24beb5573f3e" containerName="maas-api"
Jul 22 16:44:17.707645 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.707600    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbcb14f4-3198-4454-932d-24beb5573f3e" containerName="maas-api"
Jul 22 16:44:17.707645 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.707615    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9" containerName="main"
Jul 22 16:44:17.707645 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.707623    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9" containerName="main"
Jul 22 16:44:17.707645 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.707640    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab7230ef-b7ad-4c53-994a-18d1cc865404" containerName="istio-proxy"
Jul 22 16:44:17.707772 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.707648    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab7230ef-b7ad-4c53-994a-18d1cc865404" containerName="istio-proxy"
Jul 22 16:44:17.707772 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.707660    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab7230ef-b7ad-4c53-994a-18d1cc865404" containerName="istio-proxy"
Jul 22 16:44:17.707772 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.707668    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab7230ef-b7ad-4c53-994a-18d1cc865404" containerName="istio-proxy"
Jul 22 16:44:17.707772 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.707751    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab7230ef-b7ad-4c53-994a-18d1cc865404" containerName="istio-proxy"
Jul 22 16:44:17.707772 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.707764    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbcb14f4-3198-4454-932d-24beb5573f3e" containerName="maas-api"
Jul 22 16:44:17.707946 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.707774    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="dad6e5b3-9506-4d96-a8dc-5f66ebe5c3d9" containerName="main"
Jul 22 16:44:17.710567 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.710546    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:17.714209 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.714171    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-fa170d-openshift-default-dockercfg-kjpvv\""
Jul 22 16:44:17.714354 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.714232    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 22 16:44:17.714608 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.714585    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:44:17.730321 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.730290    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"]
Jul 22 16:44:17.742909 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.742877    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-workload-socket\") pod \"e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") " pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:17.743055 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.742922    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-istio-envoy\") pod \"e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") " pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:17.743055 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.742968    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-credential-socket\") pod \"e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") " pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:17.743055 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.743031    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-istio-podinfo\") pod \"e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") " pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:17.743211 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.743092    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-istiod-ca-cert\") pod \"e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") " pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:17.743211 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.743179    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-workload-certs\") pod \"e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") " pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:17.743295 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.743211    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmsr9\" (UniqueName: \"kubernetes.io/projected/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-kube-api-access-qmsr9\") pod \"e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") " pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:17.743295 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.743242    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-istio-token\") pod \"e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") " pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:17.743376 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.743293    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-istio-data\") pod \"e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") " pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:17.844444 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.844403    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-workload-socket\") pod \"e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") " pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:17.844444 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.844441    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-istio-envoy\") pod \"e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") " pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:17.844712 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.844466    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-credential-socket\") pod \"e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") " pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:17.844712 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.844499    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-istio-podinfo\") pod \"e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") " pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:17.844712 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.844538    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-istiod-ca-cert\") pod \"e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") " pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:17.844712 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.844578    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-workload-certs\") pod \"e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") " pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:17.844712 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.844602    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmsr9\" (UniqueName: \"kubernetes.io/projected/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-kube-api-access-qmsr9\") pod \"e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") " pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:17.844712 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.844630    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-istio-token\") pod \"e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") " pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:17.845066 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.844909    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-credential-socket\") pod \"e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") " pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:17.845066 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.844956    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-istio-data\") pod \"e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") " pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:17.845168 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.845097    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-workload-certs\") pod \"e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") " pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:17.845277 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.845254    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-istio-data\") pod \"e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") " pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:17.845413 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.845395    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-workload-socket\") pod \"e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") " pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:17.845480 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.845395    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-istiod-ca-cert\") pod \"e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") " pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:17.846985 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.846964    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-istio-envoy\") pod \"e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") " pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:17.847206 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.847185    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-istio-podinfo\") pod \"e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") " pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:17.853363 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.853333    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-istio-token\") pod \"e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") " pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:17.853482 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.853428    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmsr9\" (UniqueName: \"kubernetes.io/projected/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-kube-api-access-qmsr9\") pod \"e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") " pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:17.894609 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:17.894580    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:44:18.021055 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:18.020965    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:18.155755 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:18.155731    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"]
Jul 22 16:44:18.158292 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:44:18.158264    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2c1917e_275b_43ec_9ed3_3d66b549e3a3.slice/crio-c39564d9caecbdb695f6824abf1b0214a27e084ef99e4fab2406e80e61da3a94 WatchSource:0}: Error finding container c39564d9caecbdb695f6824abf1b0214a27e084ef99e4fab2406e80e61da3a94: Status 404 returned error can't find the container with id c39564d9caecbdb695f6824abf1b0214a27e084ef99e4fab2406e80e61da3a94
Jul 22 16:44:18.160446 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:18.160415    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:44:18.160512 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:18.160484    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:44:18.160555 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:18.160515    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:44:19.024450 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:19.024413    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj" event={"ID":"b2c1917e-275b-43ec-9ed3-3d66b549e3a3","Type":"ContainerStarted","Data":"0e9448b71a8beeab1ce0af07a3b0083e564620dbed8736d33b21302fba6fda29"}
Jul 22 16:44:19.024845 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:19.024472    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj" event={"ID":"b2c1917e-275b-43ec-9ed3-3d66b549e3a3","Type":"ContainerStarted","Data":"c39564d9caecbdb695f6824abf1b0214a27e084ef99e4fab2406e80e61da3a94"}
Jul 22 16:44:19.048353 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:19.048299    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj" podStartSLOduration=2.048282413 podStartE2EDuration="2.048282413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:44:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:44:19.046812945 +0000 UTC m=+2219.177282948" watchObservedRunningTime="2026-07-22 16:44:19.048282413 +0000 UTC m=+2219.178752415"
Jul 22 16:44:20.021936 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:20.021897    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:20.027002 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:20.026979    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:20.030199 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:20.030178    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:20.030986 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:20.030970    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:20.529436 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:20.529404    2563 scope.go:117] "RemoveContainer" containerID="7380c2973103bcda77b6ecddb7bee000a3d58065580d71f3442f43431393c2e1"
Jul 22 16:44:21.025239 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:21.025197    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:44:21.108712 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:21.108677    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb"]
Jul 22 16:44:21.109135 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:21.109120    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab7230ef-b7ad-4c53-994a-18d1cc865404" containerName="istio-proxy"
Jul 22 16:44:21.113230 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:21.113210    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb"
Jul 22 16:44:21.116277 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:21.116254    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vghv2\""
Jul 22 16:44:21.116425 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:21.116253    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 16:44:21.117419 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:21.117391    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 22 16:44:21.117538 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:21.117418    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-fa170d-serving-cert\""
Jul 22 16:44:21.123609 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:21.123585    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb"]
Jul 22 16:44:21.152648 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:21.152620    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:44:21.175461 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:21.175425    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmsvj\" (UniqueName: \"kubernetes.io/projected/4bb34562-a2ce-4c3c-a4bd-613dff486657-kube-api-access-cmsvj\") pod \"maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb\" (UID: \"4bb34562-a2ce-4c3c-a4bd-613dff486657\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb"
Jul 22 16:44:21.175620 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:21.175552    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4bb34562-a2ce-4c3c-a4bd-613dff486657-maas-api-tls\") pod \"maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb\" (UID: \"4bb34562-a2ce-4c3c-a4bd-613dff486657\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb"
Jul 22 16:44:21.175620 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:21.175595    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bb34562-a2ce-4c3c-a4bd-613dff486657-openshift-service-ca\") pod \"maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb\" (UID: \"4bb34562-a2ce-4c3c-a4bd-613dff486657\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb"
Jul 22 16:44:21.276272 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:21.276174    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmsvj\" (UniqueName: \"kubernetes.io/projected/4bb34562-a2ce-4c3c-a4bd-613dff486657-kube-api-access-cmsvj\") pod \"maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb\" (UID: \"4bb34562-a2ce-4c3c-a4bd-613dff486657\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb"
Jul 22 16:44:21.276454 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:21.276286    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4bb34562-a2ce-4c3c-a4bd-613dff486657-maas-api-tls\") pod \"maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb\" (UID: \"4bb34562-a2ce-4c3c-a4bd-613dff486657\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb"
Jul 22 16:44:21.276454 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:21.276310    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bb34562-a2ce-4c3c-a4bd-613dff486657-openshift-service-ca\") pod \"maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb\" (UID: \"4bb34562-a2ce-4c3c-a4bd-613dff486657\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb"
Jul 22 16:44:21.276540 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:44:21.276452    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-fa170d-serving-cert: secret "maas-api-e2e-inf-a-fa170d-serving-cert" not found
Jul 22 16:44:21.276540 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:44:21.276533    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bb34562-a2ce-4c3c-a4bd-613dff486657-maas-api-tls podName:4bb34562-a2ce-4c3c-a4bd-613dff486657 nodeName:}" failed. No retries permitted until 2026-07-22 16:44:21.776508821 +0000 UTC m=+2221.906978816 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4bb34562-a2ce-4c3c-a4bd-613dff486657-maas-api-tls") pod "maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb" (UID: "4bb34562-a2ce-4c3c-a4bd-613dff486657") : secret "maas-api-e2e-inf-a-fa170d-serving-cert" not found
Jul 22 16:44:21.277055 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:21.277037    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bb34562-a2ce-4c3c-a4bd-613dff486657-openshift-service-ca\") pod \"maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb\" (UID: \"4bb34562-a2ce-4c3c-a4bd-613dff486657\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb"
Jul 22 16:44:21.285570 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:21.285536    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:44:21.286739 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:21.286721    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmsvj\" (UniqueName: \"kubernetes.io/projected/4bb34562-a2ce-4c3c-a4bd-613dff486657-kube-api-access-cmsvj\") pod \"maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb\" (UID: \"4bb34562-a2ce-4c3c-a4bd-613dff486657\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb"
Jul 22 16:44:21.391736 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:21.391701    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:44:21.682448 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:21.682419    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:44:21.781305 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:21.781263    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4bb34562-a2ce-4c3c-a4bd-613dff486657-maas-api-tls\") pod \"maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb\" (UID: \"4bb34562-a2ce-4c3c-a4bd-613dff486657\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb"
Jul 22 16:44:21.783667 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:21.783646    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4bb34562-a2ce-4c3c-a4bd-613dff486657-maas-api-tls\") pod \"maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb\" (UID: \"4bb34562-a2ce-4c3c-a4bd-613dff486657\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb"
Jul 22 16:44:21.785415 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:21.785386    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:44:22.024190 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:22.024086    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb"
Jul 22 16:44:22.167341 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:22.167310    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb"]
Jul 22 16:44:22.169943 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:44:22.169913    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bb34562_a2ce_4c3c_a4bd_613dff486657.slice/crio-e62d157d9d2c5b95c5eb16d28466529770bc118af5be33f38fd551b415b6a512 WatchSource:0}: Error finding container e62d157d9d2c5b95c5eb16d28466529770bc118af5be33f38fd551b415b6a512: Status 404 returned error can't find the container with id e62d157d9d2c5b95c5eb16d28466529770bc118af5be33f38fd551b415b6a512
Jul 22 16:44:23.047047 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:23.047014    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb" event={"ID":"4bb34562-a2ce-4c3c-a4bd-613dff486657","Type":"ContainerStarted","Data":"a42297d3759f556632f1ca83a53415e6a7e8e5b5fcff4fef8a35f3843992ef2b"}
Jul 22 16:44:23.047208 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:23.047064    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb" event={"ID":"4bb34562-a2ce-4c3c-a4bd-613dff486657","Type":"ContainerStarted","Data":"e62d157d9d2c5b95c5eb16d28466529770bc118af5be33f38fd551b415b6a512"}
Jul 22 16:44:23.047208 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:23.047111    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb"
Jul 22 16:44:23.065815 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:23.065767    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb" podStartSLOduration=1.419793893 podStartE2EDuration="2.065752222s" totalImagesPullingTime="645.958329ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:44:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:44:23.064492443 +0000 UTC m=+2223.194962445" watchObservedRunningTime="2026-07-22 16:44:23.065752222 +0000 UTC m=+2223.196222252"
Jul 22 16:44:25.524756 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:25.524722    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:44:25.886132 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:25.886048    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:44:26.272937 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:26.272897    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:44:26.408505 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:26.408465    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:44:29.059115 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:29.059082    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb"
Jul 22 16:44:29.123878 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:29.123337    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:44:29.214876 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:29.214212    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-059dda-749488c947-sfkdx"]
Jul 22 16:44:29.218440 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:29.218417    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-059dda-749488c947-sfkdx"
Jul 22 16:44:29.221125 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:29.221103    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:44:29.222149 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:29.222128    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-059dda-serving-cert\""
Jul 22 16:44:29.234645 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:29.234615    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-059dda-749488c947-sfkdx"]
Jul 22 16:44:29.296151 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:29.296116    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:44:29.345790 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:29.345685    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9tz6\" (UniqueName: \"kubernetes.io/projected/ee451d77-8af9-424b-a0d4-27ff32b54150-kube-api-access-r9tz6\") pod \"maas-api-e2e-inf-b-059dda-749488c947-sfkdx\" (UID: \"ee451d77-8af9-424b-a0d4-27ff32b54150\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-059dda-749488c947-sfkdx"
Jul 22 16:44:29.345977 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:29.345822    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee451d77-8af9-424b-a0d4-27ff32b54150-openshift-service-ca\") pod \"maas-api-e2e-inf-b-059dda-749488c947-sfkdx\" (UID: \"ee451d77-8af9-424b-a0d4-27ff32b54150\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-059dda-749488c947-sfkdx"
Jul 22 16:44:29.345977 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:29.345892    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee451d77-8af9-424b-a0d4-27ff32b54150-maas-api-tls\") pod \"maas-api-e2e-inf-b-059dda-749488c947-sfkdx\" (UID: \"ee451d77-8af9-424b-a0d4-27ff32b54150\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-059dda-749488c947-sfkdx"
Jul 22 16:44:29.446574 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:29.446539    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee451d77-8af9-424b-a0d4-27ff32b54150-openshift-service-ca\") pod \"maas-api-e2e-inf-b-059dda-749488c947-sfkdx\" (UID: \"ee451d77-8af9-424b-a0d4-27ff32b54150\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-059dda-749488c947-sfkdx"
Jul 22 16:44:29.446791 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:29.446596    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee451d77-8af9-424b-a0d4-27ff32b54150-maas-api-tls\") pod \"maas-api-e2e-inf-b-059dda-749488c947-sfkdx\" (UID: \"ee451d77-8af9-424b-a0d4-27ff32b54150\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-059dda-749488c947-sfkdx"
Jul 22 16:44:29.446791 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:44:29.446699    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-059dda-serving-cert: secret "maas-api-e2e-inf-b-059dda-serving-cert" not found
Jul 22 16:44:29.446791 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:29.446700    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9tz6\" (UniqueName: \"kubernetes.io/projected/ee451d77-8af9-424b-a0d4-27ff32b54150-kube-api-access-r9tz6\") pod \"maas-api-e2e-inf-b-059dda-749488c947-sfkdx\" (UID: \"ee451d77-8af9-424b-a0d4-27ff32b54150\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-059dda-749488c947-sfkdx"
Jul 22 16:44:29.446791 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:44:29.446760    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee451d77-8af9-424b-a0d4-27ff32b54150-maas-api-tls podName:ee451d77-8af9-424b-a0d4-27ff32b54150 nodeName:}" failed. No retries permitted until 2026-07-22 16:44:29.946738656 +0000 UTC m=+2230.077208660 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ee451d77-8af9-424b-a0d4-27ff32b54150-maas-api-tls") pod "maas-api-e2e-inf-b-059dda-749488c947-sfkdx" (UID: "ee451d77-8af9-424b-a0d4-27ff32b54150") : secret "maas-api-e2e-inf-b-059dda-serving-cert" not found
Jul 22 16:44:29.447319 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:29.447298    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee451d77-8af9-424b-a0d4-27ff32b54150-openshift-service-ca\") pod \"maas-api-e2e-inf-b-059dda-749488c947-sfkdx\" (UID: \"ee451d77-8af9-424b-a0d4-27ff32b54150\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-059dda-749488c947-sfkdx"
Jul 22 16:44:29.456203 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:29.456177    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9tz6\" (UniqueName: \"kubernetes.io/projected/ee451d77-8af9-424b-a0d4-27ff32b54150-kube-api-access-r9tz6\") pod \"maas-api-e2e-inf-b-059dda-749488c947-sfkdx\" (UID: \"ee451d77-8af9-424b-a0d4-27ff32b54150\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-059dda-749488c947-sfkdx"
Jul 22 16:44:29.609128 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:29.609029    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:44:29.796843 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:29.796808    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:44:29.900506 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:29.900422    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:44:29.951734 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:29.951694    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee451d77-8af9-424b-a0d4-27ff32b54150-maas-api-tls\") pod \"maas-api-e2e-inf-b-059dda-749488c947-sfkdx\" (UID: \"ee451d77-8af9-424b-a0d4-27ff32b54150\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-059dda-749488c947-sfkdx"
Jul 22 16:44:29.954220 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:29.954195    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee451d77-8af9-424b-a0d4-27ff32b54150-maas-api-tls\") pod \"maas-api-e2e-inf-b-059dda-749488c947-sfkdx\" (UID: \"ee451d77-8af9-424b-a0d4-27ff32b54150\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-059dda-749488c947-sfkdx"
Jul 22 16:44:30.149758 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:30.149723    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-059dda-749488c947-sfkdx"
Jul 22 16:44:30.324877 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:30.307293    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-059dda-749488c947-sfkdx"]
Jul 22 16:44:30.336290 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:44:30.336258    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee451d77_8af9_424b_a0d4_27ff32b54150.slice/crio-8d8635b2d1d9e3ba257f7ce936fa9de58becf37ec12491c76e916f3451fe2758 WatchSource:0}: Error finding container 8d8635b2d1d9e3ba257f7ce936fa9de58becf37ec12491c76e916f3451fe2758: Status 404 returned error can't find the container with id 8d8635b2d1d9e3ba257f7ce936fa9de58becf37ec12491c76e916f3451fe2758
Jul 22 16:44:31.086664 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:31.086568    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-059dda-749488c947-sfkdx" event={"ID":"ee451d77-8af9-424b-a0d4-27ff32b54150","Type":"ContainerStarted","Data":"8319eb47a31e21a4c1ca7d039598eb795bdd2b1ec94144f7b9edb4fcfe69b96f"}
Jul 22 16:44:31.086664 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:31.086632    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-059dda-749488c947-sfkdx" event={"ID":"ee451d77-8af9-424b-a0d4-27ff32b54150","Type":"ContainerStarted","Data":"8d8635b2d1d9e3ba257f7ce936fa9de58becf37ec12491c76e916f3451fe2758"}
Jul 22 16:44:31.086875 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:31.086686    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-059dda-749488c947-sfkdx"
Jul 22 16:44:31.105958 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:31.105904    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-059dda-749488c947-sfkdx" podStartSLOduration=1.642745517 podStartE2EDuration="2.105885921s" totalImagesPullingTime="463.140404ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:44:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:44:31.104759489 +0000 UTC m=+2231.235229491" watchObservedRunningTime="2026-07-22 16:44:31.105885921 +0000 UTC m=+2231.236355920"
Jul 22 16:44:33.502316 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:33.502288    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:44:35.032406 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.032316    2563 patch_prober.go:28] interesting pod/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 22 16:44:35.032406 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.032374    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj" podUID="b2c1917e-275b-43ec-9ed3-3d66b549e3a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 22 16:44:35.105691 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.105660    2563 generic.go:356] "Generic (PLEG): container finished" podID="b2c1917e-275b-43ec-9ed3-3d66b549e3a3" containerID="0e9448b71a8beeab1ce0af07a3b0083e564620dbed8736d33b21302fba6fda29" exitCode=0
Jul 22 16:44:35.105895 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.105746    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj" event={"ID":"b2c1917e-275b-43ec-9ed3-3d66b549e3a3","Type":"ContainerDied","Data":"0e9448b71a8beeab1ce0af07a3b0083e564620dbed8736d33b21302fba6fda29"}
Jul 22 16:44:35.106172 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.106159    2563 scope.go:117] "RemoveContainer" containerID="0e9448b71a8beeab1ce0af07a3b0083e564620dbed8736d33b21302fba6fda29"
Jul 22 16:44:35.107168 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.107138    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:44:35.107264 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.107205    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:44:35.107264 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.107239    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:44:35.234643 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.234611    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r"]
Jul 22 16:44:35.237228 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.237207    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r"
Jul 22 16:44:35.240310 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.240288    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-fa170d\"/\"default-dockercfg-tphx8\""
Jul 22 16:44:35.240310 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.240298    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-fa170d\"/\"kube-root-ca.crt\""
Jul 22 16:44:35.240509 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.240304    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-fa170d\"/\"test-model-fa170d-kserve-self-signed-certs\""
Jul 22 16:44:35.240709 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.240689    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-fa170d\"/\"openshift-service-ca.crt\""
Jul 22 16:44:35.248301 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.248275    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r"]
Jul 22 16:44:35.299625 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.299533    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad016f0c-8dbf-4e27-9b18-214326ce4298-home\") pod \"test-model-fa170d-kserve-dc8b6dfc9-n255r\" (UID: \"ad016f0c-8dbf-4e27-9b18-214326ce4298\") " pod="ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r"
Jul 22 16:44:35.299625 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.299566    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad016f0c-8dbf-4e27-9b18-214326ce4298-tmp-dir\") pod \"test-model-fa170d-kserve-dc8b6dfc9-n255r\" (UID: \"ad016f0c-8dbf-4e27-9b18-214326ce4298\") " pod="ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r"
Jul 22 16:44:35.299625 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.299587    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgpt4\" (UniqueName: \"kubernetes.io/projected/ad016f0c-8dbf-4e27-9b18-214326ce4298-kube-api-access-cgpt4\") pod \"test-model-fa170d-kserve-dc8b6dfc9-n255r\" (UID: \"ad016f0c-8dbf-4e27-9b18-214326ce4298\") " pod="ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r"
Jul 22 16:44:35.299625 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.299619    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad016f0c-8dbf-4e27-9b18-214326ce4298-dshm\") pod \"test-model-fa170d-kserve-dc8b6dfc9-n255r\" (UID: \"ad016f0c-8dbf-4e27-9b18-214326ce4298\") " pod="ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r"
Jul 22 16:44:35.299977 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.299726    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad016f0c-8dbf-4e27-9b18-214326ce4298-model-cache\") pod \"test-model-fa170d-kserve-dc8b6dfc9-n255r\" (UID: \"ad016f0c-8dbf-4e27-9b18-214326ce4298\") " pod="ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r"
Jul 22 16:44:35.299977 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.299755    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad016f0c-8dbf-4e27-9b18-214326ce4298-tls-certs\") pod \"test-model-fa170d-kserve-dc8b6dfc9-n255r\" (UID: \"ad016f0c-8dbf-4e27-9b18-214326ce4298\") " pod="ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r"
Jul 22 16:44:35.340659 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.340623    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:44:35.400546 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.400515    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad016f0c-8dbf-4e27-9b18-214326ce4298-tmp-dir\") pod \"test-model-fa170d-kserve-dc8b6dfc9-n255r\" (UID: \"ad016f0c-8dbf-4e27-9b18-214326ce4298\") " pod="ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r"
Jul 22 16:44:35.400546 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.400549    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgpt4\" (UniqueName: \"kubernetes.io/projected/ad016f0c-8dbf-4e27-9b18-214326ce4298-kube-api-access-cgpt4\") pod \"test-model-fa170d-kserve-dc8b6dfc9-n255r\" (UID: \"ad016f0c-8dbf-4e27-9b18-214326ce4298\") " pod="ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r"
Jul 22 16:44:35.400788 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.400577    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad016f0c-8dbf-4e27-9b18-214326ce4298-dshm\") pod \"test-model-fa170d-kserve-dc8b6dfc9-n255r\" (UID: \"ad016f0c-8dbf-4e27-9b18-214326ce4298\") " pod="ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r"
Jul 22 16:44:35.400788 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.400613    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad016f0c-8dbf-4e27-9b18-214326ce4298-model-cache\") pod \"test-model-fa170d-kserve-dc8b6dfc9-n255r\" (UID: \"ad016f0c-8dbf-4e27-9b18-214326ce4298\") " pod="ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r"
Jul 22 16:44:35.400788 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.400628    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad016f0c-8dbf-4e27-9b18-214326ce4298-tls-certs\") pod \"test-model-fa170d-kserve-dc8b6dfc9-n255r\" (UID: \"ad016f0c-8dbf-4e27-9b18-214326ce4298\") " pod="ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r"
Jul 22 16:44:35.400788 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.400713    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad016f0c-8dbf-4e27-9b18-214326ce4298-home\") pod \"test-model-fa170d-kserve-dc8b6dfc9-n255r\" (UID: \"ad016f0c-8dbf-4e27-9b18-214326ce4298\") " pod="ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r"
Jul 22 16:44:35.401107 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.401051    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad016f0c-8dbf-4e27-9b18-214326ce4298-tmp-dir\") pod \"test-model-fa170d-kserve-dc8b6dfc9-n255r\" (UID: \"ad016f0c-8dbf-4e27-9b18-214326ce4298\") " pod="ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r"
Jul 22 16:44:35.401255 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.401112    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad016f0c-8dbf-4e27-9b18-214326ce4298-model-cache\") pod \"test-model-fa170d-kserve-dc8b6dfc9-n255r\" (UID: \"ad016f0c-8dbf-4e27-9b18-214326ce4298\") " pod="ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r"
Jul 22 16:44:35.401255 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.401136    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad016f0c-8dbf-4e27-9b18-214326ce4298-home\") pod \"test-model-fa170d-kserve-dc8b6dfc9-n255r\" (UID: \"ad016f0c-8dbf-4e27-9b18-214326ce4298\") " pod="ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r"
Jul 22 16:44:35.402895 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.402837    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad016f0c-8dbf-4e27-9b18-214326ce4298-dshm\") pod \"test-model-fa170d-kserve-dc8b6dfc9-n255r\" (UID: \"ad016f0c-8dbf-4e27-9b18-214326ce4298\") " pod="ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r"
Jul 22 16:44:35.403195 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.403175    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad016f0c-8dbf-4e27-9b18-214326ce4298-tls-certs\") pod \"test-model-fa170d-kserve-dc8b6dfc9-n255r\" (UID: \"ad016f0c-8dbf-4e27-9b18-214326ce4298\") " pod="ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r"
Jul 22 16:44:35.409231 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.409209    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgpt4\" (UniqueName: \"kubernetes.io/projected/ad016f0c-8dbf-4e27-9b18-214326ce4298-kube-api-access-cgpt4\") pod \"test-model-fa170d-kserve-dc8b6dfc9-n255r\" (UID: \"ad016f0c-8dbf-4e27-9b18-214326ce4298\") " pod="ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r"
Jul 22 16:44:35.548336 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.548296    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r"
Jul 22 16:44:35.678027 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:35.677995    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r"]
Jul 22 16:44:35.679722 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:44:35.679697    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad016f0c_8dbf_4e27_9b18_214326ce4298.slice/crio-498b7a5a6980adff35ff08b899b71865f90222081fc218eaa04b339b58568ed0 WatchSource:0}: Error finding container 498b7a5a6980adff35ff08b899b71865f90222081fc218eaa04b339b58568ed0: Status 404 returned error can't find the container with id 498b7a5a6980adff35ff08b899b71865f90222081fc218eaa04b339b58568ed0
Jul 22 16:44:36.115377 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:36.115327    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj" event={"ID":"b2c1917e-275b-43ec-9ed3-3d66b549e3a3","Type":"ContainerStarted","Data":"803e5e201f599a3efd5bf05ac9def202c48fd8004f46e36b9792c38a932ac886"}
Jul 22 16:44:36.117315 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:36.117276    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r" event={"ID":"ad016f0c-8dbf-4e27-9b18-214326ce4298","Type":"ContainerStarted","Data":"a8da14402923915ac9d31f8218fa7b4a29a8235689fd5f67f91bc090e3b2134e"}
Jul 22 16:44:36.117457 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:36.117329    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r" event={"ID":"ad016f0c-8dbf-4e27-9b18-214326ce4298","Type":"ContainerStarted","Data":"498b7a5a6980adff35ff08b899b71865f90222081fc218eaa04b339b58568ed0"}
Jul 22 16:44:36.156992 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:36.156929    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r" podStartSLOduration=1.156905391 podStartE2EDuration="1.156905391s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:44:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:44:36.156710655 +0000 UTC m=+2236.287180658" watchObservedRunningTime="2026-07-22 16:44:36.156905391 +0000 UTC m=+2236.287375393"
Jul 22 16:44:37.021553 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:37.021499    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:37.021772 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:37.021671    2563 patch_prober.go:28] interesting pod/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 22 16:44:37.021772 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:37.021731    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj" podUID="b2c1917e-275b-43ec-9ed3-3d66b549e3a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 22 16:44:37.096592 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:37.096559    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-059dda-749488c947-sfkdx"
Jul 22 16:44:38.022028 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:38.021988    2563 patch_prober.go:28] interesting pod/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 22 16:44:38.022614 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:38.022580    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj" podUID="b2c1917e-275b-43ec-9ed3-3d66b549e3a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 22 16:44:39.023796 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:39.023305    2563 patch_prober.go:28] interesting pod/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 22 16:44:39.023796 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:39.023376    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj" podUID="b2c1917e-275b-43ec-9ed3-3d66b549e3a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 22 16:44:40.021904 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:40.021837    2563 patch_prober.go:28] interesting pod/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 22 16:44:40.022102 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:40.021932    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj" podUID="b2c1917e-275b-43ec-9ed3-3d66b549e3a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 22 16:44:41.025375 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:41.025338    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:41.025843 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:41.025645    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:41.026316 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:41.026301    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:44:42.455151 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:42.455116    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:44:42.556355 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:42.556302    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:44:45.549361 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:45.549319    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r"
Jul 22 16:44:45.549767 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:45.549376    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r"
Jul 22 16:44:45.561699 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:45.561675    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r"
Jul 22 16:44:46.179727 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:46.179692    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r"
Jul 22 16:44:50.565363 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:50.565329    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:44:50.641781 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:44:50.641751    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:45:01.838488 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:01.838453    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg"]
Jul 22 16:45:01.841107 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:01.841085    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg"
Jul 22 16:45:01.844413 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:01.844386    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-059dda\"/\"kube-root-ca.crt\""
Jul 22 16:45:01.845925 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:01.845896    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-059dda\"/\"default-dockercfg-mdblr\""
Jul 22 16:45:01.846139 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:01.846123    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-059dda\"/\"openshift-service-ca.crt\""
Jul 22 16:45:01.846209 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:01.846166    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-059dda\"/\"test-model-059dda-kserve-self-signed-certs\""
Jul 22 16:45:01.857111 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:01.857089    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg"]
Jul 22 16:45:01.916305 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:01.916246    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:45:01.951797 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:01.951757    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6704627-46df-45aa-b94c-3abaa74d0bc1-tls-certs\") pod \"test-model-059dda-kserve-5b9998f7f5-4nfpg\" (UID: \"b6704627-46df-45aa-b94c-3abaa74d0bc1\") " pod="ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg"
Jul 22 16:45:01.952012 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:01.951804    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxp9v\" (UniqueName: \"kubernetes.io/projected/b6704627-46df-45aa-b94c-3abaa74d0bc1-kube-api-access-pxp9v\") pod \"test-model-059dda-kserve-5b9998f7f5-4nfpg\" (UID: \"b6704627-46df-45aa-b94c-3abaa74d0bc1\") " pod="ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg"
Jul 22 16:45:01.952012 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:01.951899    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6704627-46df-45aa-b94c-3abaa74d0bc1-dshm\") pod \"test-model-059dda-kserve-5b9998f7f5-4nfpg\" (UID: \"b6704627-46df-45aa-b94c-3abaa74d0bc1\") " pod="ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg"
Jul 22 16:45:01.952012 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:01.951954    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6704627-46df-45aa-b94c-3abaa74d0bc1-home\") pod \"test-model-059dda-kserve-5b9998f7f5-4nfpg\" (UID: \"b6704627-46df-45aa-b94c-3abaa74d0bc1\") " pod="ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg"
Jul 22 16:45:01.952012 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:01.951980    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6704627-46df-45aa-b94c-3abaa74d0bc1-model-cache\") pod \"test-model-059dda-kserve-5b9998f7f5-4nfpg\" (UID: \"b6704627-46df-45aa-b94c-3abaa74d0bc1\") " pod="ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg"
Jul 22 16:45:01.952189 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:01.952017    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6704627-46df-45aa-b94c-3abaa74d0bc1-tmp-dir\") pod \"test-model-059dda-kserve-5b9998f7f5-4nfpg\" (UID: \"b6704627-46df-45aa-b94c-3abaa74d0bc1\") " pod="ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg"
Jul 22 16:45:02.053359 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:02.053328    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6704627-46df-45aa-b94c-3abaa74d0bc1-tmp-dir\") pod \"test-model-059dda-kserve-5b9998f7f5-4nfpg\" (UID: \"b6704627-46df-45aa-b94c-3abaa74d0bc1\") " pod="ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg"
Jul 22 16:45:02.053542 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:02.053391    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6704627-46df-45aa-b94c-3abaa74d0bc1-tls-certs\") pod \"test-model-059dda-kserve-5b9998f7f5-4nfpg\" (UID: \"b6704627-46df-45aa-b94c-3abaa74d0bc1\") " pod="ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg"
Jul 22 16:45:02.053542 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:02.053412    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxp9v\" (UniqueName: \"kubernetes.io/projected/b6704627-46df-45aa-b94c-3abaa74d0bc1-kube-api-access-pxp9v\") pod \"test-model-059dda-kserve-5b9998f7f5-4nfpg\" (UID: \"b6704627-46df-45aa-b94c-3abaa74d0bc1\") " pod="ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg"
Jul 22 16:45:02.053542 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:02.053444    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6704627-46df-45aa-b94c-3abaa74d0bc1-dshm\") pod \"test-model-059dda-kserve-5b9998f7f5-4nfpg\" (UID: \"b6704627-46df-45aa-b94c-3abaa74d0bc1\") " pod="ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg"
Jul 22 16:45:02.053542 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:02.053478    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6704627-46df-45aa-b94c-3abaa74d0bc1-home\") pod \"test-model-059dda-kserve-5b9998f7f5-4nfpg\" (UID: \"b6704627-46df-45aa-b94c-3abaa74d0bc1\") " pod="ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg"
Jul 22 16:45:02.053758 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:02.053608    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6704627-46df-45aa-b94c-3abaa74d0bc1-model-cache\") pod \"test-model-059dda-kserve-5b9998f7f5-4nfpg\" (UID: \"b6704627-46df-45aa-b94c-3abaa74d0bc1\") " pod="ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg"
Jul 22 16:45:02.053810 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:02.053761    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6704627-46df-45aa-b94c-3abaa74d0bc1-tmp-dir\") pod \"test-model-059dda-kserve-5b9998f7f5-4nfpg\" (UID: \"b6704627-46df-45aa-b94c-3abaa74d0bc1\") " pod="ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg"
Jul 22 16:45:02.053900 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:02.053876    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6704627-46df-45aa-b94c-3abaa74d0bc1-home\") pod \"test-model-059dda-kserve-5b9998f7f5-4nfpg\" (UID: \"b6704627-46df-45aa-b94c-3abaa74d0bc1\") " pod="ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg"
Jul 22 16:45:02.054000 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:02.053937    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6704627-46df-45aa-b94c-3abaa74d0bc1-model-cache\") pod \"test-model-059dda-kserve-5b9998f7f5-4nfpg\" (UID: \"b6704627-46df-45aa-b94c-3abaa74d0bc1\") " pod="ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg"
Jul 22 16:45:02.055701 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:02.055681    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6704627-46df-45aa-b94c-3abaa74d0bc1-dshm\") pod \"test-model-059dda-kserve-5b9998f7f5-4nfpg\" (UID: \"b6704627-46df-45aa-b94c-3abaa74d0bc1\") " pod="ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg"
Jul 22 16:45:02.055970 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:02.055954    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6704627-46df-45aa-b94c-3abaa74d0bc1-tls-certs\") pod \"test-model-059dda-kserve-5b9998f7f5-4nfpg\" (UID: \"b6704627-46df-45aa-b94c-3abaa74d0bc1\") " pod="ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg"
Jul 22 16:45:02.068281 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:02.068256    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxp9v\" (UniqueName: \"kubernetes.io/projected/b6704627-46df-45aa-b94c-3abaa74d0bc1-kube-api-access-pxp9v\") pod \"test-model-059dda-kserve-5b9998f7f5-4nfpg\" (UID: \"b6704627-46df-45aa-b94c-3abaa74d0bc1\") " pod="ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg"
Jul 22 16:45:02.151538 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:02.151442    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg"
Jul 22 16:45:02.287047 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:02.287018    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg"]
Jul 22 16:45:02.288824 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:45:02.288798    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb6704627_46df_45aa_b94c_3abaa74d0bc1.slice/crio-a293214754c7f72642a5c7b582beff7f77a4e9dc31fc8cb2914a27864370f1c7 WatchSource:0}: Error finding container a293214754c7f72642a5c7b582beff7f77a4e9dc31fc8cb2914a27864370f1c7: Status 404 returned error can't find the container with id a293214754c7f72642a5c7b582beff7f77a4e9dc31fc8cb2914a27864370f1c7
Jul 22 16:45:03.252693 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:03.252653    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg" event={"ID":"b6704627-46df-45aa-b94c-3abaa74d0bc1","Type":"ContainerStarted","Data":"66069c70ea6630fdd8dc130b2d330b3ff94f6efb25448acb8b54ed4b01703961"}
Jul 22 16:45:03.252693 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:03.252701    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg" event={"ID":"b6704627-46df-45aa-b94c-3abaa74d0bc1","Type":"ContainerStarted","Data":"a293214754c7f72642a5c7b582beff7f77a4e9dc31fc8cb2914a27864370f1c7"}
Jul 22 16:45:03.271627 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:03.271573    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg" podStartSLOduration=2.271554678 podStartE2EDuration="2.271554678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:45:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:45:03.271400947 +0000 UTC m=+2263.401870960" watchObservedRunningTime="2026-07-22 16:45:03.271554678 +0000 UTC m=+2263.402024681"
Jul 22 16:45:12.151824 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:12.151784    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg"
Jul 22 16:45:12.152253 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:12.151839    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg"
Jul 22 16:45:12.164594 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:12.164567    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg"
Jul 22 16:45:12.304404 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:12.304370    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg"
Jul 22 16:45:16.722606 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:16.722567    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:45:16.784020 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:16.783957    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:45:20.551269 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:45:20.551238    2563 scope.go:117] "RemoveContainer" containerID="460e70b7b595d73a2912102eec215a232dc8881684079ae9422ea4aa0e6104b1"
Jul 22 16:46:03.420578 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:03.420544    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r"]
Jul 22 16:46:03.476218 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:03.476183    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:46:04.419285 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:04.419240    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg"]
Jul 22 16:46:04.456754 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:04.456717    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:46:05.220251 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:05.220217    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:46:05.364714 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:05.364682    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:46:15.420624 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:15.420536    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb"]
Jul 22 16:46:15.421156 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:15.420795    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb" podUID="4bb34562-a2ce-4c3c-a4bd-613dff486657" containerName="maas-api" containerID="cri-o://a42297d3759f556632f1ca83a53415e6a7e8e5b5fcff4fef8a35f3843992ef2b" gracePeriod=30
Jul 22 16:46:15.541750 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:15.541719    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:46:15.587487 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:15.587452    2563 generic.go:356] "Generic (PLEG): container finished" podID="4bb34562-a2ce-4c3c-a4bd-613dff486657" containerID="a42297d3759f556632f1ca83a53415e6a7e8e5b5fcff4fef8a35f3843992ef2b" exitCode=0
Jul 22 16:46:15.587648 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:15.587544    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb" event={"ID":"4bb34562-a2ce-4c3c-a4bd-613dff486657","Type":"ContainerDied","Data":"a42297d3759f556632f1ca83a53415e6a7e8e5b5fcff4fef8a35f3843992ef2b"}
Jul 22 16:46:15.662422 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:15.662398    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb"
Jul 22 16:46:15.811543 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:15.811513    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4bb34562-a2ce-4c3c-a4bd-613dff486657-maas-api-tls\") pod \"4bb34562-a2ce-4c3c-a4bd-613dff486657\" (UID: \"4bb34562-a2ce-4c3c-a4bd-613dff486657\") "
Jul 22 16:46:15.811732 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:15.811622    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cmsvj\" (UniqueName: \"kubernetes.io/projected/4bb34562-a2ce-4c3c-a4bd-613dff486657-kube-api-access-cmsvj\") pod \"4bb34562-a2ce-4c3c-a4bd-613dff486657\" (UID: \"4bb34562-a2ce-4c3c-a4bd-613dff486657\") "
Jul 22 16:46:15.811732 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:15.811642    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bb34562-a2ce-4c3c-a4bd-613dff486657-openshift-service-ca\") pod \"4bb34562-a2ce-4c3c-a4bd-613dff486657\" (UID: \"4bb34562-a2ce-4c3c-a4bd-613dff486657\") "
Jul 22 16:46:15.812114 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:15.812075    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bb34562-a2ce-4c3c-a4bd-613dff486657-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4bb34562-a2ce-4c3c-a4bd-613dff486657" (UID: "4bb34562-a2ce-4c3c-a4bd-613dff486657"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 16:46:15.813694 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:15.813669    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bb34562-a2ce-4c3c-a4bd-613dff486657-kube-api-access-cmsvj" (OuterVolumeSpecName: "kube-api-access-cmsvj") pod "4bb34562-a2ce-4c3c-a4bd-613dff486657" (UID: "4bb34562-a2ce-4c3c-a4bd-613dff486657"). InnerVolumeSpecName "kube-api-access-cmsvj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:46:15.813751 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:15.813693    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4bb34562-a2ce-4c3c-a4bd-613dff486657-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4bb34562-a2ce-4c3c-a4bd-613dff486657" (UID: "4bb34562-a2ce-4c3c-a4bd-613dff486657"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:46:15.912980 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:15.912932    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4bb34562-a2ce-4c3c-a4bd-613dff486657-maas-api-tls\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:46:15.912980 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:15.912973    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cmsvj\" (UniqueName: \"kubernetes.io/projected/4bb34562-a2ce-4c3c-a4bd-613dff486657-kube-api-access-cmsvj\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:46:15.912980 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:15.912984    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bb34562-a2ce-4c3c-a4bd-613dff486657-openshift-service-ca\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:46:16.592928 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:16.592901    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb"
Jul 22 16:46:16.592928 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:16.592918    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb" event={"ID":"4bb34562-a2ce-4c3c-a4bd-613dff486657","Type":"ContainerDied","Data":"e62d157d9d2c5b95c5eb16d28466529770bc118af5be33f38fd551b415b6a512"}
Jul 22 16:46:16.593385 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:16.592982    2563 scope.go:117] "RemoveContainer" containerID="a42297d3759f556632f1ca83a53415e6a7e8e5b5fcff4fef8a35f3843992ef2b"
Jul 22 16:46:16.632109 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:16.632070    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb"]
Jul 22 16:46:16.642557 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:16.642529    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fa170d-79d6f9cb4-h6bqb"]
Jul 22 16:46:18.440998 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:18.440955    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r" podUID="ad016f0c-8dbf-4e27-9b18-214326ce4298" containerName="main" containerID="cri-o://a8da14402923915ac9d31f8218fa7b4a29a8235689fd5f67f91bc090e3b2134e" gracePeriod=45
Jul 22 16:46:18.449100 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:18.449067    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4bb34562-a2ce-4c3c-a4bd-613dff486657" path="/var/lib/kubelet/pods/4bb34562-a2ce-4c3c-a4bd-613dff486657/volumes"
Jul 22 16:46:18.604674 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:18.604643    2563 generic.go:356] "Generic (PLEG): container finished" podID="ad016f0c-8dbf-4e27-9b18-214326ce4298" containerID="a8da14402923915ac9d31f8218fa7b4a29a8235689fd5f67f91bc090e3b2134e" exitCode=0
Jul 22 16:46:18.604805 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:18.604672    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r" event={"ID":"ad016f0c-8dbf-4e27-9b18-214326ce4298","Type":"ContainerDied","Data":"a8da14402923915ac9d31f8218fa7b4a29a8235689fd5f67f91bc090e3b2134e"}
Jul 22 16:46:18.687115 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:18.687089    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r"
Jul 22 16:46:18.841032 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:18.840942    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad016f0c-8dbf-4e27-9b18-214326ce4298-tmp-dir\") pod \"ad016f0c-8dbf-4e27-9b18-214326ce4298\" (UID: \"ad016f0c-8dbf-4e27-9b18-214326ce4298\") "
Jul 22 16:46:18.841032 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:18.840986    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgpt4\" (UniqueName: \"kubernetes.io/projected/ad016f0c-8dbf-4e27-9b18-214326ce4298-kube-api-access-cgpt4\") pod \"ad016f0c-8dbf-4e27-9b18-214326ce4298\" (UID: \"ad016f0c-8dbf-4e27-9b18-214326ce4298\") "
Jul 22 16:46:18.841032 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:18.841021    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad016f0c-8dbf-4e27-9b18-214326ce4298-tls-certs\") pod \"ad016f0c-8dbf-4e27-9b18-214326ce4298\" (UID: \"ad016f0c-8dbf-4e27-9b18-214326ce4298\") "
Jul 22 16:46:18.841313 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:18.841041    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad016f0c-8dbf-4e27-9b18-214326ce4298-dshm\") pod \"ad016f0c-8dbf-4e27-9b18-214326ce4298\" (UID: \"ad016f0c-8dbf-4e27-9b18-214326ce4298\") "
Jul 22 16:46:18.841313 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:18.841061    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad016f0c-8dbf-4e27-9b18-214326ce4298-model-cache\") pod \"ad016f0c-8dbf-4e27-9b18-214326ce4298\" (UID: \"ad016f0c-8dbf-4e27-9b18-214326ce4298\") "
Jul 22 16:46:18.841313 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:18.841151    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad016f0c-8dbf-4e27-9b18-214326ce4298-home\") pod \"ad016f0c-8dbf-4e27-9b18-214326ce4298\" (UID: \"ad016f0c-8dbf-4e27-9b18-214326ce4298\") "
Jul 22 16:46:18.841313 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:18.841252    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad016f0c-8dbf-4e27-9b18-214326ce4298-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ad016f0c-8dbf-4e27-9b18-214326ce4298" (UID: "ad016f0c-8dbf-4e27-9b18-214326ce4298"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:46:18.841521 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:18.841454    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad016f0c-8dbf-4e27-9b18-214326ce4298-model-cache" (OuterVolumeSpecName: "model-cache") pod "ad016f0c-8dbf-4e27-9b18-214326ce4298" (UID: "ad016f0c-8dbf-4e27-9b18-214326ce4298"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:46:18.841521 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:18.841469    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad016f0c-8dbf-4e27-9b18-214326ce4298-home" (OuterVolumeSpecName: "home") pod "ad016f0c-8dbf-4e27-9b18-214326ce4298" (UID: "ad016f0c-8dbf-4e27-9b18-214326ce4298"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:46:18.841521 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:18.841478    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad016f0c-8dbf-4e27-9b18-214326ce4298-tmp-dir\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:46:18.843320 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:18.843298    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad016f0c-8dbf-4e27-9b18-214326ce4298-dshm" (OuterVolumeSpecName: "dshm") pod "ad016f0c-8dbf-4e27-9b18-214326ce4298" (UID: "ad016f0c-8dbf-4e27-9b18-214326ce4298"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:46:18.843408 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:18.843346    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad016f0c-8dbf-4e27-9b18-214326ce4298-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ad016f0c-8dbf-4e27-9b18-214326ce4298" (UID: "ad016f0c-8dbf-4e27-9b18-214326ce4298"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:46:18.843446 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:18.843425    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad016f0c-8dbf-4e27-9b18-214326ce4298-kube-api-access-cgpt4" (OuterVolumeSpecName: "kube-api-access-cgpt4") pod "ad016f0c-8dbf-4e27-9b18-214326ce4298" (UID: "ad016f0c-8dbf-4e27-9b18-214326ce4298"). InnerVolumeSpecName "kube-api-access-cgpt4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:46:18.942840 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:18.942798    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad016f0c-8dbf-4e27-9b18-214326ce4298-home\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:46:18.942840 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:18.942832    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgpt4\" (UniqueName: \"kubernetes.io/projected/ad016f0c-8dbf-4e27-9b18-214326ce4298-kube-api-access-cgpt4\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:46:18.942840 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:18.942841    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad016f0c-8dbf-4e27-9b18-214326ce4298-tls-certs\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:46:18.942840 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:18.942870    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad016f0c-8dbf-4e27-9b18-214326ce4298-dshm\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:46:18.942840 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:18.942878    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad016f0c-8dbf-4e27-9b18-214326ce4298-model-cache\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:46:19.439014 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:19.438975    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg" podUID="b6704627-46df-45aa-b94c-3abaa74d0bc1" containerName="main" containerID="cri-o://66069c70ea6630fdd8dc130b2d330b3ff94f6efb25448acb8b54ed4b01703961" gracePeriod=45
Jul 22 16:46:19.611143 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:19.611111    2563 generic.go:356] "Generic (PLEG): container finished" podID="b6704627-46df-45aa-b94c-3abaa74d0bc1" containerID="66069c70ea6630fdd8dc130b2d330b3ff94f6efb25448acb8b54ed4b01703961" exitCode=0
Jul 22 16:46:19.611599 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:19.611180    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg" event={"ID":"b6704627-46df-45aa-b94c-3abaa74d0bc1","Type":"ContainerDied","Data":"66069c70ea6630fdd8dc130b2d330b3ff94f6efb25448acb8b54ed4b01703961"}
Jul 22 16:46:19.612633 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:19.612610    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r" event={"ID":"ad016f0c-8dbf-4e27-9b18-214326ce4298","Type":"ContainerDied","Data":"498b7a5a6980adff35ff08b899b71865f90222081fc218eaa04b339b58568ed0"}
Jul 22 16:46:19.612781 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:19.612647    2563 scope.go:117] "RemoveContainer" containerID="a8da14402923915ac9d31f8218fa7b4a29a8235689fd5f67f91bc090e3b2134e"
Jul 22 16:46:19.612781 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:19.612652    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r"
Jul 22 16:46:19.656888 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:19.656840    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r"]
Jul 22 16:46:19.668243 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:19.668212    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-fa170d/test-model-fa170d-kserve-dc8b6dfc9-n255r"]
Jul 22 16:46:19.688798 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:19.688772    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg"
Jul 22 16:46:19.851221 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:19.851125    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6704627-46df-45aa-b94c-3abaa74d0bc1-tls-certs\") pod \"b6704627-46df-45aa-b94c-3abaa74d0bc1\" (UID: \"b6704627-46df-45aa-b94c-3abaa74d0bc1\") "
Jul 22 16:46:19.851221 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:19.851191    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6704627-46df-45aa-b94c-3abaa74d0bc1-tmp-dir\") pod \"b6704627-46df-45aa-b94c-3abaa74d0bc1\" (UID: \"b6704627-46df-45aa-b94c-3abaa74d0bc1\") "
Jul 22 16:46:19.851452 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:19.851228    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6704627-46df-45aa-b94c-3abaa74d0bc1-model-cache\") pod \"b6704627-46df-45aa-b94c-3abaa74d0bc1\" (UID: \"b6704627-46df-45aa-b94c-3abaa74d0bc1\") "
Jul 22 16:46:19.851452 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:19.851301    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxp9v\" (UniqueName: \"kubernetes.io/projected/b6704627-46df-45aa-b94c-3abaa74d0bc1-kube-api-access-pxp9v\") pod \"b6704627-46df-45aa-b94c-3abaa74d0bc1\" (UID: \"b6704627-46df-45aa-b94c-3abaa74d0bc1\") "
Jul 22 16:46:19.851452 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:19.851359    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6704627-46df-45aa-b94c-3abaa74d0bc1-dshm\") pod \"b6704627-46df-45aa-b94c-3abaa74d0bc1\" (UID: \"b6704627-46df-45aa-b94c-3abaa74d0bc1\") "
Jul 22 16:46:19.851452 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:19.851383    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6704627-46df-45aa-b94c-3abaa74d0bc1-home\") pod \"b6704627-46df-45aa-b94c-3abaa74d0bc1\" (UID: \"b6704627-46df-45aa-b94c-3abaa74d0bc1\") "
Jul 22 16:46:19.851668 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:19.851477    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6704627-46df-45aa-b94c-3abaa74d0bc1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b6704627-46df-45aa-b94c-3abaa74d0bc1" (UID: "b6704627-46df-45aa-b94c-3abaa74d0bc1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:46:19.851668 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:19.851544    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6704627-46df-45aa-b94c-3abaa74d0bc1-model-cache" (OuterVolumeSpecName: "model-cache") pod "b6704627-46df-45aa-b94c-3abaa74d0bc1" (UID: "b6704627-46df-45aa-b94c-3abaa74d0bc1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:46:19.851765 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:19.851691    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6704627-46df-45aa-b94c-3abaa74d0bc1-tmp-dir\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:46:19.851765 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:19.851697    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6704627-46df-45aa-b94c-3abaa74d0bc1-home" (OuterVolumeSpecName: "home") pod "b6704627-46df-45aa-b94c-3abaa74d0bc1" (UID: "b6704627-46df-45aa-b94c-3abaa74d0bc1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:46:19.851765 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:19.851706    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6704627-46df-45aa-b94c-3abaa74d0bc1-model-cache\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:46:19.853536 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:19.853511    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6704627-46df-45aa-b94c-3abaa74d0bc1-dshm" (OuterVolumeSpecName: "dshm") pod "b6704627-46df-45aa-b94c-3abaa74d0bc1" (UID: "b6704627-46df-45aa-b94c-3abaa74d0bc1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:46:19.853648 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:19.853574    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6704627-46df-45aa-b94c-3abaa74d0bc1-kube-api-access-pxp9v" (OuterVolumeSpecName: "kube-api-access-pxp9v") pod "b6704627-46df-45aa-b94c-3abaa74d0bc1" (UID: "b6704627-46df-45aa-b94c-3abaa74d0bc1"). InnerVolumeSpecName "kube-api-access-pxp9v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:46:19.853648 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:19.853590    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6704627-46df-45aa-b94c-3abaa74d0bc1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b6704627-46df-45aa-b94c-3abaa74d0bc1" (UID: "b6704627-46df-45aa-b94c-3abaa74d0bc1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:46:19.952402 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:19.952351    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxp9v\" (UniqueName: \"kubernetes.io/projected/b6704627-46df-45aa-b94c-3abaa74d0bc1-kube-api-access-pxp9v\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:46:19.952402 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:19.952397    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6704627-46df-45aa-b94c-3abaa74d0bc1-dshm\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:46:19.952402 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:19.952407    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6704627-46df-45aa-b94c-3abaa74d0bc1-home\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:46:19.952402 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:19.952415    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6704627-46df-45aa-b94c-3abaa74d0bc1-tls-certs\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:46:20.449276 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:20.449246    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad016f0c-8dbf-4e27-9b18-214326ce4298" path="/var/lib/kubelet/pods/ad016f0c-8dbf-4e27-9b18-214326ce4298/volumes"
Jul 22 16:46:20.495450 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:20.495421    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:46:20.617833 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:20.617804    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg"
Jul 22 16:46:20.617833 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:20.617815    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg" event={"ID":"b6704627-46df-45aa-b94c-3abaa74d0bc1","Type":"ContainerDied","Data":"a293214754c7f72642a5c7b582beff7f77a4e9dc31fc8cb2914a27864370f1c7"}
Jul 22 16:46:20.618349 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:20.617890    2563 scope.go:117] "RemoveContainer" containerID="66069c70ea6630fdd8dc130b2d330b3ff94f6efb25448acb8b54ed4b01703961"
Jul 22 16:46:20.629000 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:20.628971    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:46:20.644913 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:20.644881    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg"]
Jul 22 16:46:20.649917 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:20.649879    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-059dda/test-model-059dda-kserve-5b9998f7f5-4nfpg"]
Jul 22 16:46:22.447922 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:22.447889    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6704627-46df-45aa-b94c-3abaa74d0bc1" path="/var/lib/kubelet/pods/b6704627-46df-45aa-b94c-3abaa74d0bc1/volumes"
Jul 22 16:46:27.472830 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:27.472797    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:46:27.617497 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:27.617467    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:46:27.709575 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:27.709546    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"]
Jul 22 16:46:27.709793 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:27.709762    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj" podUID="b2c1917e-275b-43ec-9ed3-3d66b549e3a3" containerName="istio-proxy" containerID="cri-o://803e5e201f599a3efd5bf05ac9def202c48fd8004f46e36b9792c38a932ac886" gracePeriod=30
Jul 22 16:46:27.775802 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:27.775683    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:46:27.961516 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:27.961484    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:46:29.933543 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:29.933504    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:46:30.073952 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:30.073916    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:46:32.965301 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:32.965273    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:46:33.059620 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.059535    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qmsr9\" (UniqueName: \"kubernetes.io/projected/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-kube-api-access-qmsr9\") pod \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") "
Jul 22 16:46:33.059620 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.059582    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-workload-certs\") pod \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") "
Jul 22 16:46:33.059620 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.059608    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-istiod-ca-cert\") pod \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") "
Jul 22 16:46:33.059888 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.059626    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-istio-podinfo\") pod \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") "
Jul 22 16:46:33.059888 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.059759    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-istio-data\") pod \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") "
Jul 22 16:46:33.059888 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.059804    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-workload-socket\") pod \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") "
Jul 22 16:46:33.059888 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.059830    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-credential-socket\") pod \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") "
Jul 22 16:46:33.060123 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.059947    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b2c1917e-275b-43ec-9ed3-3d66b549e3a3" (UID: "b2c1917e-275b-43ec-9ed3-3d66b549e3a3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:46:33.060123 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.059954    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-istio-envoy\") pod \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") "
Jul 22 16:46:33.060123 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.060008    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-istio-token\") pod \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\" (UID: \"b2c1917e-275b-43ec-9ed3-3d66b549e3a3\") "
Jul 22 16:46:33.060276 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.060144    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b2c1917e-275b-43ec-9ed3-3d66b549e3a3" (UID: "b2c1917e-275b-43ec-9ed3-3d66b549e3a3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 16:46:33.060276 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.060152    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b2c1917e-275b-43ec-9ed3-3d66b549e3a3" (UID: "b2c1917e-275b-43ec-9ed3-3d66b549e3a3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:46:33.060390 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.060280    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-workload-certs\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:46:33.060390 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.060301    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-istiod-ca-cert\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:46:33.060390 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.060317    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-workload-socket\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:46:33.060558 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.060484    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b2c1917e-275b-43ec-9ed3-3d66b549e3a3" (UID: "b2c1917e-275b-43ec-9ed3-3d66b549e3a3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:46:33.060914 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.060882    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-istio-data" (OuterVolumeSpecName: "istio-data") pod "b2c1917e-275b-43ec-9ed3-3d66b549e3a3" (UID: "b2c1917e-275b-43ec-9ed3-3d66b549e3a3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:46:33.061995 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.061971    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-kube-api-access-qmsr9" (OuterVolumeSpecName: "kube-api-access-qmsr9") pod "b2c1917e-275b-43ec-9ed3-3d66b549e3a3" (UID: "b2c1917e-275b-43ec-9ed3-3d66b549e3a3"). InnerVolumeSpecName "kube-api-access-qmsr9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:46:33.062200 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.062181    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-istio-token" (OuterVolumeSpecName: "istio-token") pod "b2c1917e-275b-43ec-9ed3-3d66b549e3a3" (UID: "b2c1917e-275b-43ec-9ed3-3d66b549e3a3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:46:33.062401 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.062378    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b2c1917e-275b-43ec-9ed3-3d66b549e3a3" (UID: "b2c1917e-275b-43ec-9ed3-3d66b549e3a3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:46:33.062472 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.062454    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b2c1917e-275b-43ec-9ed3-3d66b549e3a3" (UID: "b2c1917e-275b-43ec-9ed3-3d66b549e3a3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 16:46:33.160710 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.160675    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmsr9\" (UniqueName: \"kubernetes.io/projected/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-kube-api-access-qmsr9\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:46:33.160710 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.160703    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-istio-podinfo\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:46:33.160710 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.160712    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-istio-data\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:46:33.160710 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.160720    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-credential-socket\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:46:33.161008 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.160730    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-istio-envoy\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:46:33.161008 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.160738    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2c1917e-275b-43ec-9ed3-3d66b549e3a3-istio-token\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:46:33.677246 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.677212    2563 generic.go:356] "Generic (PLEG): container finished" podID="b2c1917e-275b-43ec-9ed3-3d66b549e3a3" containerID="803e5e201f599a3efd5bf05ac9def202c48fd8004f46e36b9792c38a932ac886" exitCode=0
Jul 22 16:46:33.677515 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.677269    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"
Jul 22 16:46:33.677515 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.677298    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj" event={"ID":"b2c1917e-275b-43ec-9ed3-3d66b549e3a3","Type":"ContainerDied","Data":"803e5e201f599a3efd5bf05ac9def202c48fd8004f46e36b9792c38a932ac886"}
Jul 22 16:46:33.677515 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.677365    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj" event={"ID":"b2c1917e-275b-43ec-9ed3-3d66b549e3a3","Type":"ContainerDied","Data":"c39564d9caecbdb695f6824abf1b0214a27e084ef99e4fab2406e80e61da3a94"}
Jul 22 16:46:33.677515 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.677395    2563 scope.go:117] "RemoveContainer" containerID="803e5e201f599a3efd5bf05ac9def202c48fd8004f46e36b9792c38a932ac886"
Jul 22 16:46:33.686925 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.686902    2563 scope.go:117] "RemoveContainer" containerID="0e9448b71a8beeab1ce0af07a3b0083e564620dbed8736d33b21302fba6fda29"
Jul 22 16:46:33.695429 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.695409    2563 scope.go:117] "RemoveContainer" containerID="803e5e201f599a3efd5bf05ac9def202c48fd8004f46e36b9792c38a932ac886"
Jul 22 16:46:33.695723 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:46:33.695703    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"803e5e201f599a3efd5bf05ac9def202c48fd8004f46e36b9792c38a932ac886\": container with ID starting with 803e5e201f599a3efd5bf05ac9def202c48fd8004f46e36b9792c38a932ac886 not found: ID does not exist" containerID="803e5e201f599a3efd5bf05ac9def202c48fd8004f46e36b9792c38a932ac886"
Jul 22 16:46:33.695806 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.695736    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"803e5e201f599a3efd5bf05ac9def202c48fd8004f46e36b9792c38a932ac886"} err="failed to get container status \"803e5e201f599a3efd5bf05ac9def202c48fd8004f46e36b9792c38a932ac886\": rpc error: code = NotFound desc = could not find container \"803e5e201f599a3efd5bf05ac9def202c48fd8004f46e36b9792c38a932ac886\": container with ID starting with 803e5e201f599a3efd5bf05ac9def202c48fd8004f46e36b9792c38a932ac886 not found: ID does not exist"
Jul 22 16:46:33.695806 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.695761    2563 scope.go:117] "RemoveContainer" containerID="0e9448b71a8beeab1ce0af07a3b0083e564620dbed8736d33b21302fba6fda29"
Jul 22 16:46:33.696048 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:46:33.696031    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e9448b71a8beeab1ce0af07a3b0083e564620dbed8736d33b21302fba6fda29\": container with ID starting with 0e9448b71a8beeab1ce0af07a3b0083e564620dbed8736d33b21302fba6fda29 not found: ID does not exist" containerID="0e9448b71a8beeab1ce0af07a3b0083e564620dbed8736d33b21302fba6fda29"
Jul 22 16:46:33.696115 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.696056    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e9448b71a8beeab1ce0af07a3b0083e564620dbed8736d33b21302fba6fda29"} err="failed to get container status \"0e9448b71a8beeab1ce0af07a3b0083e564620dbed8736d33b21302fba6fda29\": rpc error: code = NotFound desc = could not find container \"0e9448b71a8beeab1ce0af07a3b0083e564620dbed8736d33b21302fba6fda29\": container with ID starting with 0e9448b71a8beeab1ce0af07a3b0083e564620dbed8736d33b21302fba6fda29 not found: ID does not exist"
Jul 22 16:46:33.716494 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.716462    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"]
Jul 22 16:46:33.723380 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:33.723356    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-fa170d-openshift-default-5c78db4d7f-ttpjj"]
Jul 22 16:46:34.448220 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:34.448189    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2c1917e-275b-43ec-9ed3-3d66b549e3a3" path="/var/lib/kubelet/pods/b2c1917e-275b-43ec-9ed3-3d66b549e3a3/volumes"
Jul 22 16:46:38.659974 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:38.659933    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-059dda-749488c947-sfkdx"]
Jul 22 16:46:38.660579 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:38.660274    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-059dda-749488c947-sfkdx" podUID="ee451d77-8af9-424b-a0d4-27ff32b54150" containerName="maas-api" containerID="cri-o://8319eb47a31e21a4c1ca7d039598eb795bdd2b1ec94144f7b9edb4fcfe69b96f" gracePeriod=30
Jul 22 16:46:38.748320 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:38.748284    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:46:38.903036 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:38.903009    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-059dda-749488c947-sfkdx"
Jul 22 16:46:39.003534 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:39.003494    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee451d77-8af9-424b-a0d4-27ff32b54150-maas-api-tls\") pod \"ee451d77-8af9-424b-a0d4-27ff32b54150\" (UID: \"ee451d77-8af9-424b-a0d4-27ff32b54150\") "
Jul 22 16:46:39.003689 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:39.003554    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee451d77-8af9-424b-a0d4-27ff32b54150-openshift-service-ca\") pod \"ee451d77-8af9-424b-a0d4-27ff32b54150\" (UID: \"ee451d77-8af9-424b-a0d4-27ff32b54150\") "
Jul 22 16:46:39.003689 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:39.003616    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9tz6\" (UniqueName: \"kubernetes.io/projected/ee451d77-8af9-424b-a0d4-27ff32b54150-kube-api-access-r9tz6\") pod \"ee451d77-8af9-424b-a0d4-27ff32b54150\" (UID: \"ee451d77-8af9-424b-a0d4-27ff32b54150\") "
Jul 22 16:46:39.003954 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:39.003916    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee451d77-8af9-424b-a0d4-27ff32b54150-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ee451d77-8af9-424b-a0d4-27ff32b54150" (UID: "ee451d77-8af9-424b-a0d4-27ff32b54150"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 16:46:39.005613 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:39.005589    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee451d77-8af9-424b-a0d4-27ff32b54150-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ee451d77-8af9-424b-a0d4-27ff32b54150" (UID: "ee451d77-8af9-424b-a0d4-27ff32b54150"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:46:39.005709 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:39.005647    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee451d77-8af9-424b-a0d4-27ff32b54150-kube-api-access-r9tz6" (OuterVolumeSpecName: "kube-api-access-r9tz6") pod "ee451d77-8af9-424b-a0d4-27ff32b54150" (UID: "ee451d77-8af9-424b-a0d4-27ff32b54150"). InnerVolumeSpecName "kube-api-access-r9tz6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:46:39.104472 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:39.104426    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee451d77-8af9-424b-a0d4-27ff32b54150-openshift-service-ca\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:46:39.104472 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:39.104462    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9tz6\" (UniqueName: \"kubernetes.io/projected/ee451d77-8af9-424b-a0d4-27ff32b54150-kube-api-access-r9tz6\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:46:39.104472 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:39.104476    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee451d77-8af9-424b-a0d4-27ff32b54150-maas-api-tls\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:46:39.703933 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:39.703899    2563 generic.go:356] "Generic (PLEG): container finished" podID="ee451d77-8af9-424b-a0d4-27ff32b54150" containerID="8319eb47a31e21a4c1ca7d039598eb795bdd2b1ec94144f7b9edb4fcfe69b96f" exitCode=0
Jul 22 16:46:39.704370 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:39.703942    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-059dda-749488c947-sfkdx"
Jul 22 16:46:39.704370 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:39.703984    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-059dda-749488c947-sfkdx" event={"ID":"ee451d77-8af9-424b-a0d4-27ff32b54150","Type":"ContainerDied","Data":"8319eb47a31e21a4c1ca7d039598eb795bdd2b1ec94144f7b9edb4fcfe69b96f"}
Jul 22 16:46:39.704370 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:39.704033    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-059dda-749488c947-sfkdx" event={"ID":"ee451d77-8af9-424b-a0d4-27ff32b54150","Type":"ContainerDied","Data":"8d8635b2d1d9e3ba257f7ce936fa9de58becf37ec12491c76e916f3451fe2758"}
Jul 22 16:46:39.704370 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:39.704054    2563 scope.go:117] "RemoveContainer" containerID="8319eb47a31e21a4c1ca7d039598eb795bdd2b1ec94144f7b9edb4fcfe69b96f"
Jul 22 16:46:39.713042 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:39.713024    2563 scope.go:117] "RemoveContainer" containerID="8319eb47a31e21a4c1ca7d039598eb795bdd2b1ec94144f7b9edb4fcfe69b96f"
Jul 22 16:46:39.713298 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:46:39.713279    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8319eb47a31e21a4c1ca7d039598eb795bdd2b1ec94144f7b9edb4fcfe69b96f\": container with ID starting with 8319eb47a31e21a4c1ca7d039598eb795bdd2b1ec94144f7b9edb4fcfe69b96f not found: ID does not exist" containerID="8319eb47a31e21a4c1ca7d039598eb795bdd2b1ec94144f7b9edb4fcfe69b96f"
Jul 22 16:46:39.713365 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:39.713313    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8319eb47a31e21a4c1ca7d039598eb795bdd2b1ec94144f7b9edb4fcfe69b96f"} err="failed to get container status \"8319eb47a31e21a4c1ca7d039598eb795bdd2b1ec94144f7b9edb4fcfe69b96f\": rpc error: code = NotFound desc = could not find container \"8319eb47a31e21a4c1ca7d039598eb795bdd2b1ec94144f7b9edb4fcfe69b96f\": container with ID starting with 8319eb47a31e21a4c1ca7d039598eb795bdd2b1ec94144f7b9edb4fcfe69b96f not found: ID does not exist"
Jul 22 16:46:39.741307 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:39.741271    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-059dda-749488c947-sfkdx"]
Jul 22 16:46:39.748272 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:39.748242    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-059dda-749488c947-sfkdx"]
Jul 22 16:46:40.448869 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:40.448815    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee451d77-8af9-424b-a0d4-27ff32b54150" path="/var/lib/kubelet/pods/ee451d77-8af9-424b-a0d4-27ff32b54150/volumes"
Jul 22 16:46:43.235780 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:43.235749    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:46:43.608488 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:43.608380    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:46:46.996322 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:46.996286    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:46:51.009517 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:51.009476    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:46:51.257489 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:51.257456    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:46:51.423643 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:51.423560    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:46:51.726126 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:51.726087    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:46:52.194650 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:52.194622    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:46:52.841186 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:52.841150    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"]
Jul 22 16:46:52.841469 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:52.841457    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2c1917e-275b-43ec-9ed3-3d66b549e3a3" containerName="istio-proxy"
Jul 22 16:46:52.841469 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:52.841470    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2c1917e-275b-43ec-9ed3-3d66b549e3a3" containerName="istio-proxy"
Jul 22 16:46:52.841587 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:52.841482    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bb34562-a2ce-4c3c-a4bd-613dff486657" containerName="maas-api"
Jul 22 16:46:52.841587 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:52.841487    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bb34562-a2ce-4c3c-a4bd-613dff486657" containerName="maas-api"
Jul 22 16:46:52.841587 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:52.841498    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad016f0c-8dbf-4e27-9b18-214326ce4298" containerName="main"
Jul 22 16:46:52.841587 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:52.841504    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad016f0c-8dbf-4e27-9b18-214326ce4298" containerName="main"
Jul 22 16:46:52.841587 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:52.841511    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee451d77-8af9-424b-a0d4-27ff32b54150" containerName="maas-api"
Jul 22 16:46:52.841587 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:52.841516    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee451d77-8af9-424b-a0d4-27ff32b54150" containerName="maas-api"
Jul 22 16:46:52.841587 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:52.841525    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6704627-46df-45aa-b94c-3abaa74d0bc1" containerName="main"
Jul 22 16:46:52.841587 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:52.841530    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6704627-46df-45aa-b94c-3abaa74d0bc1" containerName="main"
Jul 22 16:46:52.841587 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:52.841582    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2c1917e-275b-43ec-9ed3-3d66b549e3a3" containerName="istio-proxy"
Jul 22 16:46:52.841587 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:52.841590    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad016f0c-8dbf-4e27-9b18-214326ce4298" containerName="main"
Jul 22 16:46:52.842124 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:52.841599    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bb34562-a2ce-4c3c-a4bd-613dff486657" containerName="maas-api"
Jul 22 16:46:52.842124 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:52.841605    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6704627-46df-45aa-b94c-3abaa74d0bc1" containerName="main"
Jul 22 16:46:52.842124 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:52.841611    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee451d77-8af9-424b-a0d4-27ff32b54150" containerName="maas-api"
Jul 22 16:46:52.844302 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:52.844283    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:46:52.849098 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:52.849075    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 22 16:46:52.850059 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:52.850040    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-f0cd70-openshift-default-dockercfg-rtbck\""
Jul 22 16:46:52.871871 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:52.871816    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:46:52.920674 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:52.920642    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flp2d\" (UniqueName: \"kubernetes.io/projected/7efd9d04-351f-4e16-98c6-a8acf3076b07-kube-api-access-flp2d\") pod \"e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") " pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:46:52.920887 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:52.920679    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7efd9d04-351f-4e16-98c6-a8acf3076b07-istio-envoy\") pod \"e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") " pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:46:52.920887 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:52.920728    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7efd9d04-351f-4e16-98c6-a8acf3076b07-workload-socket\") pod \"e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") " pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:46:52.920887 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:52.920774    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7efd9d04-351f-4e16-98c6-a8acf3076b07-istio-podinfo\") pod \"e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") " pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:46:52.920887 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:52.920863    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7efd9d04-351f-4e16-98c6-a8acf3076b07-credential-socket\") pod \"e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") " pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:46:52.921049 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:52.920899    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7efd9d04-351f-4e16-98c6-a8acf3076b07-istio-data\") pod \"e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") " pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:46:52.921049 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:52.920950    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7efd9d04-351f-4e16-98c6-a8acf3076b07-istiod-ca-cert\") pod \"e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") " pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:46:52.921049 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:52.920977    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7efd9d04-351f-4e16-98c6-a8acf3076b07-workload-certs\") pod \"e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") " pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:46:52.921049 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:52.921009    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7efd9d04-351f-4e16-98c6-a8acf3076b07-istio-token\") pod \"e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") " pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:46:52.995387 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:52.995345    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"]
Jul 22 16:46:53.021791 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:53.021749    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7efd9d04-351f-4e16-98c6-a8acf3076b07-workload-socket\") pod \"e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") " pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:46:53.021791 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:53.021790    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7efd9d04-351f-4e16-98c6-a8acf3076b07-istio-podinfo\") pod \"e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") " pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:46:53.022055 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:53.021816    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7efd9d04-351f-4e16-98c6-a8acf3076b07-credential-socket\") pod \"e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") " pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:46:53.022055 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:53.021842    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7efd9d04-351f-4e16-98c6-a8acf3076b07-istio-data\") pod \"e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") " pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:46:53.022055 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:53.021908    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7efd9d04-351f-4e16-98c6-a8acf3076b07-istiod-ca-cert\") pod \"e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") " pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:46:53.022055 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:53.021932    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7efd9d04-351f-4e16-98c6-a8acf3076b07-workload-certs\") pod \"e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") " pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:46:53.022055 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:53.021964    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7efd9d04-351f-4e16-98c6-a8acf3076b07-istio-token\") pod \"e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") " pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:46:53.022055 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:53.021998    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flp2d\" (UniqueName: \"kubernetes.io/projected/7efd9d04-351f-4e16-98c6-a8acf3076b07-kube-api-access-flp2d\") pod \"e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") " pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:46:53.022055 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:53.022024    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7efd9d04-351f-4e16-98c6-a8acf3076b07-istio-envoy\") pod \"e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") " pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:46:53.022422 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:53.022310    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7efd9d04-351f-4e16-98c6-a8acf3076b07-workload-socket\") pod \"e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") " pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:46:53.022555 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:53.022515    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7efd9d04-351f-4e16-98c6-a8acf3076b07-workload-certs\") pod \"e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") " pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:46:53.022656 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:53.022638    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7efd9d04-351f-4e16-98c6-a8acf3076b07-istio-data\") pod \"e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") " pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:46:53.023036 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:53.023004    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7efd9d04-351f-4e16-98c6-a8acf3076b07-credential-socket\") pod \"e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") " pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:46:53.023036 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:53.023026    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7efd9d04-351f-4e16-98c6-a8acf3076b07-istiod-ca-cert\") pod \"e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") " pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:46:53.024426 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:53.024404    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7efd9d04-351f-4e16-98c6-a8acf3076b07-istio-envoy\") pod \"e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") " pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:46:53.024986 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:53.024959    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7efd9d04-351f-4e16-98c6-a8acf3076b07-istio-podinfo\") pod \"e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") " pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:46:53.047254 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:53.047225    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flp2d\" (UniqueName: \"kubernetes.io/projected/7efd9d04-351f-4e16-98c6-a8acf3076b07-kube-api-access-flp2d\") pod \"e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") " pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:46:53.047361 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:53.047283    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7efd9d04-351f-4e16-98c6-a8acf3076b07-istio-token\") pod \"e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") " pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:46:53.157528 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:53.157429    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:46:53.322015 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:53.321990    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"]
Jul 22 16:46:53.323795 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:46:53.323762    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7efd9d04_351f_4e16_98c6_a8acf3076b07.slice/crio-0d23f77a9b0e39067e94d57bf5f712ea2837b96d981182dd54c80a249c6a3cc8 WatchSource:0}: Error finding container 0d23f77a9b0e39067e94d57bf5f712ea2837b96d981182dd54c80a249c6a3cc8: Status 404 returned error can't find the container with id 0d23f77a9b0e39067e94d57bf5f712ea2837b96d981182dd54c80a249c6a3cc8
Jul 22 16:46:53.325742 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:53.325711    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:46:53.325831 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:53.325782    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:46:53.325831 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:53.325811    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:46:53.396693 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:53.396654    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:46:53.768635 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:53.768597    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g" event={"ID":"7efd9d04-351f-4e16-98c6-a8acf3076b07","Type":"ContainerStarted","Data":"6149cbfa74f820b6ef41c360a89521ece73ac717535e2d8c486c834c5ba9bddf"}
Jul 22 16:46:53.768872 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:53.768654    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g" event={"ID":"7efd9d04-351f-4e16-98c6-a8acf3076b07","Type":"ContainerStarted","Data":"0d23f77a9b0e39067e94d57bf5f712ea2837b96d981182dd54c80a249c6a3cc8"}
Jul 22 16:46:53.823510 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:53.823459    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g" podStartSLOduration=1.8234422399999999 podStartE2EDuration="1.82344224s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:46:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:46:53.82338908 +0000 UTC m=+2373.953859084" watchObservedRunningTime="2026-07-22 16:46:53.82344224 +0000 UTC m=+2373.953912242"
Jul 22 16:46:54.158143 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:54.158055    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:46:54.163043 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:54.163016    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:46:54.773453 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:54.773420    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:46:54.774503 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:54.774483    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:46:56.196767 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:56.196724    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:46:56.389230 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:56.389194    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f0cd70-79b459d86-2cb22"]
Jul 22 16:46:56.389666 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:56.389652    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2c1917e-275b-43ec-9ed3-3d66b549e3a3" containerName="istio-proxy"
Jul 22 16:46:56.389711 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:56.389669    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2c1917e-275b-43ec-9ed3-3d66b549e3a3" containerName="istio-proxy"
Jul 22 16:46:56.389778 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:56.389768    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2c1917e-275b-43ec-9ed3-3d66b549e3a3" containerName="istio-proxy"
Jul 22 16:46:56.392044 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:56.392019    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f0cd70-79b459d86-2cb22"
Jul 22 16:46:56.395428 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:56.395406    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-f0cd70-serving-cert\""
Jul 22 16:46:56.395662 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:56.395647    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vghv2\""
Jul 22 16:46:56.396774 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:56.396750    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 16:46:56.403284 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:56.403260    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 22 16:46:56.414468 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:56.414441    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f0cd70-79b459d86-2cb22"]
Jul 22 16:46:56.422618 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:56.422565    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:46:56.452134 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:56.452051    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60ee3390-8856-4856-b154-4438760e8d89-maas-api-tls\") pod \"maas-api-e2e-api-a-f0cd70-79b459d86-2cb22\" (UID: \"60ee3390-8856-4856-b154-4438760e8d89\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f0cd70-79b459d86-2cb22"
Jul 22 16:46:56.452268 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:56.452173    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60ee3390-8856-4856-b154-4438760e8d89-openshift-service-ca\") pod \"maas-api-e2e-api-a-f0cd70-79b459d86-2cb22\" (UID: \"60ee3390-8856-4856-b154-4438760e8d89\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f0cd70-79b459d86-2cb22"
Jul 22 16:46:56.452268 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:56.452206    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpvh4\" (UniqueName: \"kubernetes.io/projected/60ee3390-8856-4856-b154-4438760e8d89-kube-api-access-lpvh4\") pod \"maas-api-e2e-api-a-f0cd70-79b459d86-2cb22\" (UID: \"60ee3390-8856-4856-b154-4438760e8d89\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f0cd70-79b459d86-2cb22"
Jul 22 16:46:56.553080 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:56.553042    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60ee3390-8856-4856-b154-4438760e8d89-maas-api-tls\") pod \"maas-api-e2e-api-a-f0cd70-79b459d86-2cb22\" (UID: \"60ee3390-8856-4856-b154-4438760e8d89\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f0cd70-79b459d86-2cb22"
Jul 22 16:46:56.553415 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:56.553394    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60ee3390-8856-4856-b154-4438760e8d89-openshift-service-ca\") pod \"maas-api-e2e-api-a-f0cd70-79b459d86-2cb22\" (UID: \"60ee3390-8856-4856-b154-4438760e8d89\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f0cd70-79b459d86-2cb22"
Jul 22 16:46:56.553527 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:56.553453    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpvh4\" (UniqueName: \"kubernetes.io/projected/60ee3390-8856-4856-b154-4438760e8d89-kube-api-access-lpvh4\") pod \"maas-api-e2e-api-a-f0cd70-79b459d86-2cb22\" (UID: \"60ee3390-8856-4856-b154-4438760e8d89\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f0cd70-79b459d86-2cb22"
Jul 22 16:46:56.554470 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:56.554441    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60ee3390-8856-4856-b154-4438760e8d89-openshift-service-ca\") pod \"maas-api-e2e-api-a-f0cd70-79b459d86-2cb22\" (UID: \"60ee3390-8856-4856-b154-4438760e8d89\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f0cd70-79b459d86-2cb22"
Jul 22 16:46:56.555807 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:56.555778    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60ee3390-8856-4856-b154-4438760e8d89-maas-api-tls\") pod \"maas-api-e2e-api-a-f0cd70-79b459d86-2cb22\" (UID: \"60ee3390-8856-4856-b154-4438760e8d89\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f0cd70-79b459d86-2cb22"
Jul 22 16:46:56.567784 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:56.567761    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpvh4\" (UniqueName: \"kubernetes.io/projected/60ee3390-8856-4856-b154-4438760e8d89-kube-api-access-lpvh4\") pod \"maas-api-e2e-api-a-f0cd70-79b459d86-2cb22\" (UID: \"60ee3390-8856-4856-b154-4438760e8d89\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f0cd70-79b459d86-2cb22"
Jul 22 16:46:56.618462 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:56.618425    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:46:56.703183 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:56.703095    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f0cd70-79b459d86-2cb22"
Jul 22 16:46:56.834590 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:56.834555    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:46:56.863324 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:56.863293    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f0cd70-79b459d86-2cb22"]
Jul 22 16:46:56.866651 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:46:56.866625    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60ee3390_8856_4856_b154_4438760e8d89.slice/crio-dc74a243fd0ef8187f421b51a6865c2af44378689f1a1068c06c1098f04c6caa WatchSource:0}: Error finding container dc74a243fd0ef8187f421b51a6865c2af44378689f1a1068c06c1098f04c6caa: Status 404 returned error can't find the container with id dc74a243fd0ef8187f421b51a6865c2af44378689f1a1068c06c1098f04c6caa
Jul 22 16:46:56.868252 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:56.868234    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 16:46:57.743591 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:57.743556    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:46:57.791368 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:57.791334    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f0cd70-79b459d86-2cb22" event={"ID":"60ee3390-8856-4856-b154-4438760e8d89","Type":"ContainerStarted","Data":"ec3ad8e88a85b0cf929034428eb3ffd6dce77235c192793faa140a0125a960b7"}
Jul 22 16:46:57.791538 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:57.791385    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f0cd70-79b459d86-2cb22" event={"ID":"60ee3390-8856-4856-b154-4438760e8d89","Type":"ContainerStarted","Data":"dc74a243fd0ef8187f421b51a6865c2af44378689f1a1068c06c1098f04c6caa"}
Jul 22 16:46:57.791538 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:57.791496    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f0cd70-79b459d86-2cb22"
Jul 22 16:46:57.991993 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:46:57.991960    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:47:00.519162 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:00.519130    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:47:01.258649 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:01.258617    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:47:01.944477 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:01.944437    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:47:02.320906 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:02.320772    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:47:03.801752 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:03.801722    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f0cd70-79b459d86-2cb22"
Jul 22 16:47:03.877480 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:03.877425    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f0cd70-79b459d86-2cb22" podStartSLOduration=7.443074685 podStartE2EDuration="7.877409412s" totalImagesPullingTime="434.334727ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:46:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:46:57.897370744 +0000 UTC m=+2378.027840742" watchObservedRunningTime="2026-07-22 16:47:03.877409412 +0000 UTC m=+2384.007879413"
Jul 22 16:47:06.301226 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:06.301126    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:47:06.313094 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:06.313061    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-285a31-779c874df9-bp7jd"]
Jul 22 16:47:06.315448 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:06.315424    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-285a31-779c874df9-bp7jd"
Jul 22 16:47:06.319585 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:06.319554    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-285a31-serving-cert\""
Jul 22 16:47:06.345971 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:06.345939    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-285a31-779c874df9-bp7jd"]
Jul 22 16:47:06.449732 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:06.449697    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:47:06.460021 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:06.459988    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b666c3e8-fc5f-4854-90aa-35591fc60cd9-openshift-service-ca\") pod \"maas-api-e2e-api-b-285a31-779c874df9-bp7jd\" (UID: \"b666c3e8-fc5f-4854-90aa-35591fc60cd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-285a31-779c874df9-bp7jd"
Jul 22 16:47:06.460202 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:06.460033    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hz794\" (UniqueName: \"kubernetes.io/projected/b666c3e8-fc5f-4854-90aa-35591fc60cd9-kube-api-access-hz794\") pod \"maas-api-e2e-api-b-285a31-779c874df9-bp7jd\" (UID: \"b666c3e8-fc5f-4854-90aa-35591fc60cd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-285a31-779c874df9-bp7jd"
Jul 22 16:47:06.460202 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:06.460074    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b666c3e8-fc5f-4854-90aa-35591fc60cd9-maas-api-tls\") pod \"maas-api-e2e-api-b-285a31-779c874df9-bp7jd\" (UID: \"b666c3e8-fc5f-4854-90aa-35591fc60cd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-285a31-779c874df9-bp7jd"
Jul 22 16:47:06.561519 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:06.561427    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b666c3e8-fc5f-4854-90aa-35591fc60cd9-openshift-service-ca\") pod \"maas-api-e2e-api-b-285a31-779c874df9-bp7jd\" (UID: \"b666c3e8-fc5f-4854-90aa-35591fc60cd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-285a31-779c874df9-bp7jd"
Jul 22 16:47:06.561519 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:06.561469    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hz794\" (UniqueName: \"kubernetes.io/projected/b666c3e8-fc5f-4854-90aa-35591fc60cd9-kube-api-access-hz794\") pod \"maas-api-e2e-api-b-285a31-779c874df9-bp7jd\" (UID: \"b666c3e8-fc5f-4854-90aa-35591fc60cd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-285a31-779c874df9-bp7jd"
Jul 22 16:47:06.561758 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:06.561604    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b666c3e8-fc5f-4854-90aa-35591fc60cd9-maas-api-tls\") pod \"maas-api-e2e-api-b-285a31-779c874df9-bp7jd\" (UID: \"b666c3e8-fc5f-4854-90aa-35591fc60cd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-285a31-779c874df9-bp7jd"
Jul 22 16:47:06.562263 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:06.562240    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b666c3e8-fc5f-4854-90aa-35591fc60cd9-openshift-service-ca\") pod \"maas-api-e2e-api-b-285a31-779c874df9-bp7jd\" (UID: \"b666c3e8-fc5f-4854-90aa-35591fc60cd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-285a31-779c874df9-bp7jd"
Jul 22 16:47:06.564148 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:06.564120    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b666c3e8-fc5f-4854-90aa-35591fc60cd9-maas-api-tls\") pod \"maas-api-e2e-api-b-285a31-779c874df9-bp7jd\" (UID: \"b666c3e8-fc5f-4854-90aa-35591fc60cd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-285a31-779c874df9-bp7jd"
Jul 22 16:47:06.578968 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:06.578937    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hz794\" (UniqueName: \"kubernetes.io/projected/b666c3e8-fc5f-4854-90aa-35591fc60cd9-kube-api-access-hz794\") pod \"maas-api-e2e-api-b-285a31-779c874df9-bp7jd\" (UID: \"b666c3e8-fc5f-4854-90aa-35591fc60cd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-285a31-779c874df9-bp7jd"
Jul 22 16:47:06.589592 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:06.589563    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:47:06.630734 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:06.630698    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-285a31-779c874df9-bp7jd"
Jul 22 16:47:06.824977 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:06.824952    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-285a31-779c874df9-bp7jd"]
Jul 22 16:47:06.827625 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:47:06.827597    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb666c3e8_fc5f_4854_90aa_35591fc60cd9.slice/crio-e44e4f67f5310c92459dfc8afeb6e3f69426e9de5af4f17ea48010fbdf74d260 WatchSource:0}: Error finding container e44e4f67f5310c92459dfc8afeb6e3f69426e9de5af4f17ea48010fbdf74d260: Status 404 returned error can't find the container with id e44e4f67f5310c92459dfc8afeb6e3f69426e9de5af4f17ea48010fbdf74d260
Jul 22 16:47:06.841171 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:06.841137    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-285a31-779c874df9-bp7jd" event={"ID":"b666c3e8-fc5f-4854-90aa-35591fc60cd9","Type":"ContainerStarted","Data":"e44e4f67f5310c92459dfc8afeb6e3f69426e9de5af4f17ea48010fbdf74d260"}
Jul 22 16:47:07.399819 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:07.399787    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:47:07.619920 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:07.619874    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:47:07.849381 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:07.849341    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-285a31-779c874df9-bp7jd" event={"ID":"b666c3e8-fc5f-4854-90aa-35591fc60cd9","Type":"ContainerStarted","Data":"3dcd2c148299ade45793b4c63a3373f0a58c38180ad6aa27b94878461f5a7d07"}
Jul 22 16:47:07.849524 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:07.849490    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-285a31-779c874df9-bp7jd"
Jul 22 16:47:08.854785 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:08.854703    2563 generic.go:356] "Generic (PLEG): container finished" podID="7efd9d04-351f-4e16-98c6-a8acf3076b07" containerID="6149cbfa74f820b6ef41c360a89521ece73ac717535e2d8c486c834c5ba9bddf" exitCode=0
Jul 22 16:47:08.854785 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:08.854773    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g" event={"ID":"7efd9d04-351f-4e16-98c6-a8acf3076b07","Type":"ContainerDied","Data":"6149cbfa74f820b6ef41c360a89521ece73ac717535e2d8c486c834c5ba9bddf"}
Jul 22 16:47:08.855367 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:08.855205    2563 scope.go:117] "RemoveContainer" containerID="6149cbfa74f820b6ef41c360a89521ece73ac717535e2d8c486c834c5ba9bddf"
Jul 22 16:47:08.856148 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:08.856122    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:47:08.856215 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:08.856194    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:47:08.856253 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:08.856230    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:47:08.914716 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:08.914669    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-285a31-779c874df9-bp7jd" podStartSLOduration=2.573504958 podStartE2EDuration="2.914652284s" totalImagesPullingTime="341.147326ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:47:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:47:07.895471116 +0000 UTC m=+2388.025941130" watchObservedRunningTime="2026-07-22 16:47:08.914652284 +0000 UTC m=+2389.045122285"
Jul 22 16:47:09.774291 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:09.774252    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:47:09.860805 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:09.860763    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g" event={"ID":"7efd9d04-351f-4e16-98c6-a8acf3076b07","Type":"ContainerStarted","Data":"48945bc4a12e96c2f414a7564f91531fbaf85266a5961175eedb47bb5977fbae"}
Jul 22 16:47:10.157703 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:10.157615    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:47:10.158818 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:10.158785    2563 patch_prober.go:28] interesting pod/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 22 16:47:10.158982 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:10.158842    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g" podUID="7efd9d04-351f-4e16-98c6-a8acf3076b07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 22 16:47:10.648030 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:10.647995    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:47:11.158812 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:11.158772    2563 patch_prober.go:28] interesting pod/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 22 16:47:11.159232 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:11.158833    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g" podUID="7efd9d04-351f-4e16-98c6-a8acf3076b07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 22 16:47:12.158823 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:12.158785    2563 patch_prober.go:28] interesting pod/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 22 16:47:12.159251 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:12.158843    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g" podUID="7efd9d04-351f-4e16-98c6-a8acf3076b07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 22 16:47:13.158516 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:13.158479    2563 patch_prober.go:28] interesting pod/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 22 16:47:13.158696 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:13.158535    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g" podUID="7efd9d04-351f-4e16-98c6-a8acf3076b07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 22 16:47:13.860975 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:13.860948    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-285a31-779c874df9-bp7jd"
Jul 22 16:47:14.162469 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:14.162396    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:47:14.162766 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:14.162744    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:47:14.163396 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:14.163377    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:47:19.663722 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:19.663690    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:47:19.863446 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:19.863412    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:47:20.587626 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:20.587585    2563 scope.go:117] "RemoveContainer" containerID="c7e09dff96f0b0b59f22c924a76d814a647f6a60dab4b3689521756acf4b714a"
Jul 22 16:47:28.804725 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:28.804688    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:47:29.022617 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:29.022577    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:47:34.271077 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:34.271035    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f0cd70-79b459d86-2cb22"]
Jul 22 16:47:34.271549 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:34.271281    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f0cd70-79b459d86-2cb22" podUID="60ee3390-8856-4856-b154-4438760e8d89" containerName="maas-api" containerID="cri-o://ec3ad8e88a85b0cf929034428eb3ffd6dce77235c192793faa140a0125a960b7" gracePeriod=30
Jul 22 16:47:34.342102 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:34.342060    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:47:34.518527 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:34.518503    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f0cd70-79b459d86-2cb22"
Jul 22 16:47:34.605168 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:34.605085    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60ee3390-8856-4856-b154-4438760e8d89-openshift-service-ca\") pod \"60ee3390-8856-4856-b154-4438760e8d89\" (UID: \"60ee3390-8856-4856-b154-4438760e8d89\") "
Jul 22 16:47:34.605168 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:34.605136    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60ee3390-8856-4856-b154-4438760e8d89-maas-api-tls\") pod \"60ee3390-8856-4856-b154-4438760e8d89\" (UID: \"60ee3390-8856-4856-b154-4438760e8d89\") "
Jul 22 16:47:34.605168 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:34.605160    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpvh4\" (UniqueName: \"kubernetes.io/projected/60ee3390-8856-4856-b154-4438760e8d89-kube-api-access-lpvh4\") pod \"60ee3390-8856-4856-b154-4438760e8d89\" (UID: \"60ee3390-8856-4856-b154-4438760e8d89\") "
Jul 22 16:47:34.605519 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:34.605492    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60ee3390-8856-4856-b154-4438760e8d89-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "60ee3390-8856-4856-b154-4438760e8d89" (UID: "60ee3390-8856-4856-b154-4438760e8d89"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 16:47:34.607283 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:34.607257    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60ee3390-8856-4856-b154-4438760e8d89-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "60ee3390-8856-4856-b154-4438760e8d89" (UID: "60ee3390-8856-4856-b154-4438760e8d89"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:47:34.607411 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:34.607365    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60ee3390-8856-4856-b154-4438760e8d89-kube-api-access-lpvh4" (OuterVolumeSpecName: "kube-api-access-lpvh4") pod "60ee3390-8856-4856-b154-4438760e8d89" (UID: "60ee3390-8856-4856-b154-4438760e8d89"). InnerVolumeSpecName "kube-api-access-lpvh4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:47:34.706245 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:34.706196    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60ee3390-8856-4856-b154-4438760e8d89-maas-api-tls\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:47:34.706245 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:34.706239    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpvh4\" (UniqueName: \"kubernetes.io/projected/60ee3390-8856-4856-b154-4438760e8d89-kube-api-access-lpvh4\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:47:34.706245 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:34.706250    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60ee3390-8856-4856-b154-4438760e8d89-openshift-service-ca\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:47:34.973261 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:34.973227    2563 generic.go:356] "Generic (PLEG): container finished" podID="60ee3390-8856-4856-b154-4438760e8d89" containerID="ec3ad8e88a85b0cf929034428eb3ffd6dce77235c192793faa140a0125a960b7" exitCode=0
Jul 22 16:47:34.973445 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:34.973292    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f0cd70-79b459d86-2cb22"
Jul 22 16:47:34.973445 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:34.973313    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f0cd70-79b459d86-2cb22" event={"ID":"60ee3390-8856-4856-b154-4438760e8d89","Type":"ContainerDied","Data":"ec3ad8e88a85b0cf929034428eb3ffd6dce77235c192793faa140a0125a960b7"}
Jul 22 16:47:34.973445 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:34.973368    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f0cd70-79b459d86-2cb22" event={"ID":"60ee3390-8856-4856-b154-4438760e8d89","Type":"ContainerDied","Data":"dc74a243fd0ef8187f421b51a6865c2af44378689f1a1068c06c1098f04c6caa"}
Jul 22 16:47:34.973445 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:34.973392    2563 scope.go:117] "RemoveContainer" containerID="ec3ad8e88a85b0cf929034428eb3ffd6dce77235c192793faa140a0125a960b7"
Jul 22 16:47:34.982781 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:34.982764    2563 scope.go:117] "RemoveContainer" containerID="ec3ad8e88a85b0cf929034428eb3ffd6dce77235c192793faa140a0125a960b7"
Jul 22 16:47:34.983085 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:47:34.983067    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec3ad8e88a85b0cf929034428eb3ffd6dce77235c192793faa140a0125a960b7\": container with ID starting with ec3ad8e88a85b0cf929034428eb3ffd6dce77235c192793faa140a0125a960b7 not found: ID does not exist" containerID="ec3ad8e88a85b0cf929034428eb3ffd6dce77235c192793faa140a0125a960b7"
Jul 22 16:47:34.983148 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:34.983094    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec3ad8e88a85b0cf929034428eb3ffd6dce77235c192793faa140a0125a960b7"} err="failed to get container status \"ec3ad8e88a85b0cf929034428eb3ffd6dce77235c192793faa140a0125a960b7\": rpc error: code = NotFound desc = could not find container \"ec3ad8e88a85b0cf929034428eb3ffd6dce77235c192793faa140a0125a960b7\": container with ID starting with ec3ad8e88a85b0cf929034428eb3ffd6dce77235c192793faa140a0125a960b7 not found: ID does not exist"
Jul 22 16:47:35.004435 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:35.004403    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f0cd70-79b459d86-2cb22"]
Jul 22 16:47:35.010044 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:35.010022    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f0cd70-79b459d86-2cb22"]
Jul 22 16:47:36.447617 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:36.447579    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60ee3390-8856-4856-b154-4438760e8d89" path="/var/lib/kubelet/pods/60ee3390-8856-4856-b154-4438760e8d89/volumes"
Jul 22 16:47:39.168264 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:39.168231    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:47:39.651786 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:39.651754    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:47:47.420923 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:47.420820    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:47:47.671070 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:47.670982    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:47:47.760514 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:47.760483    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"]
Jul 22 16:47:47.760716 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:47.760693    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g" podUID="7efd9d04-351f-4e16-98c6-a8acf3076b07" containerName="istio-proxy" containerID="cri-o://48945bc4a12e96c2f414a7564f91531fbaf85266a5961175eedb47bb5977fbae" gracePeriod=30
Jul 22 16:47:47.860665 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:47.860630    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:47:48.027370 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:48.027337    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:47:50.856517 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:50.856482    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:47:50.959968 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:50.959931    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:47:53.008829 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.008803    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:47:53.052245 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.052157    2563 generic.go:356] "Generic (PLEG): container finished" podID="7efd9d04-351f-4e16-98c6-a8acf3076b07" containerID="48945bc4a12e96c2f414a7564f91531fbaf85266a5961175eedb47bb5977fbae" exitCode=0
Jul 22 16:47:53.052245 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.052201    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"
Jul 22 16:47:53.052445 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.052239    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g" event={"ID":"7efd9d04-351f-4e16-98c6-a8acf3076b07","Type":"ContainerDied","Data":"48945bc4a12e96c2f414a7564f91531fbaf85266a5961175eedb47bb5977fbae"}
Jul 22 16:47:53.052445 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.052294    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g" event={"ID":"7efd9d04-351f-4e16-98c6-a8acf3076b07","Type":"ContainerDied","Data":"0d23f77a9b0e39067e94d57bf5f712ea2837b96d981182dd54c80a249c6a3cc8"}
Jul 22 16:47:53.052445 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.052328    2563 scope.go:117] "RemoveContainer" containerID="48945bc4a12e96c2f414a7564f91531fbaf85266a5961175eedb47bb5977fbae"
Jul 22 16:47:53.060668 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.060648    2563 scope.go:117] "RemoveContainer" containerID="6149cbfa74f820b6ef41c360a89521ece73ac717535e2d8c486c834c5ba9bddf"
Jul 22 16:47:53.068188 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.068170    2563 scope.go:117] "RemoveContainer" containerID="48945bc4a12e96c2f414a7564f91531fbaf85266a5961175eedb47bb5977fbae"
Jul 22 16:47:53.068460 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:47:53.068437    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48945bc4a12e96c2f414a7564f91531fbaf85266a5961175eedb47bb5977fbae\": container with ID starting with 48945bc4a12e96c2f414a7564f91531fbaf85266a5961175eedb47bb5977fbae not found: ID does not exist" containerID="48945bc4a12e96c2f414a7564f91531fbaf85266a5961175eedb47bb5977fbae"
Jul 22 16:47:53.068531 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.068474    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48945bc4a12e96c2f414a7564f91531fbaf85266a5961175eedb47bb5977fbae"} err="failed to get container status \"48945bc4a12e96c2f414a7564f91531fbaf85266a5961175eedb47bb5977fbae\": rpc error: code = NotFound desc = could not find container \"48945bc4a12e96c2f414a7564f91531fbaf85266a5961175eedb47bb5977fbae\": container with ID starting with 48945bc4a12e96c2f414a7564f91531fbaf85266a5961175eedb47bb5977fbae not found: ID does not exist"
Jul 22 16:47:53.068531 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.068497    2563 scope.go:117] "RemoveContainer" containerID="6149cbfa74f820b6ef41c360a89521ece73ac717535e2d8c486c834c5ba9bddf"
Jul 22 16:47:53.068744 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:47:53.068728    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6149cbfa74f820b6ef41c360a89521ece73ac717535e2d8c486c834c5ba9bddf\": container with ID starting with 6149cbfa74f820b6ef41c360a89521ece73ac717535e2d8c486c834c5ba9bddf not found: ID does not exist" containerID="6149cbfa74f820b6ef41c360a89521ece73ac717535e2d8c486c834c5ba9bddf"
Jul 22 16:47:53.068781 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.068752    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6149cbfa74f820b6ef41c360a89521ece73ac717535e2d8c486c834c5ba9bddf"} err="failed to get container status \"6149cbfa74f820b6ef41c360a89521ece73ac717535e2d8c486c834c5ba9bddf\": rpc error: code = NotFound desc = could not find container \"6149cbfa74f820b6ef41c360a89521ece73ac717535e2d8c486c834c5ba9bddf\": container with ID starting with 6149cbfa74f820b6ef41c360a89521ece73ac717535e2d8c486c834c5ba9bddf not found: ID does not exist"
Jul 22 16:47:53.071052 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.071037    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7efd9d04-351f-4e16-98c6-a8acf3076b07-credential-socket\") pod \"7efd9d04-351f-4e16-98c6-a8acf3076b07\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") "
Jul 22 16:47:53.071111 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.071068    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-flp2d\" (UniqueName: \"kubernetes.io/projected/7efd9d04-351f-4e16-98c6-a8acf3076b07-kube-api-access-flp2d\") pod \"7efd9d04-351f-4e16-98c6-a8acf3076b07\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") "
Jul 22 16:47:53.071111 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.071090    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7efd9d04-351f-4e16-98c6-a8acf3076b07-istio-token\") pod \"7efd9d04-351f-4e16-98c6-a8acf3076b07\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") "
Jul 22 16:47:53.071218 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.071116    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7efd9d04-351f-4e16-98c6-a8acf3076b07-istiod-ca-cert\") pod \"7efd9d04-351f-4e16-98c6-a8acf3076b07\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") "
Jul 22 16:47:53.071218 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.071143    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7efd9d04-351f-4e16-98c6-a8acf3076b07-workload-socket\") pod \"7efd9d04-351f-4e16-98c6-a8acf3076b07\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") "
Jul 22 16:47:53.071218 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.071172    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7efd9d04-351f-4e16-98c6-a8acf3076b07-workload-certs\") pod \"7efd9d04-351f-4e16-98c6-a8acf3076b07\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") "
Jul 22 16:47:53.071369 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.071223    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7efd9d04-351f-4e16-98c6-a8acf3076b07-istio-envoy\") pod \"7efd9d04-351f-4e16-98c6-a8acf3076b07\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") "
Jul 22 16:47:53.071369 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.071247    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7efd9d04-351f-4e16-98c6-a8acf3076b07-istio-podinfo\") pod \"7efd9d04-351f-4e16-98c6-a8acf3076b07\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") "
Jul 22 16:47:53.071369 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.071278    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7efd9d04-351f-4e16-98c6-a8acf3076b07-istio-data\") pod \"7efd9d04-351f-4e16-98c6-a8acf3076b07\" (UID: \"7efd9d04-351f-4e16-98c6-a8acf3076b07\") "
Jul 22 16:47:53.071518 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.071382    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7efd9d04-351f-4e16-98c6-a8acf3076b07-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7efd9d04-351f-4e16-98c6-a8acf3076b07" (UID: "7efd9d04-351f-4e16-98c6-a8acf3076b07"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:47:53.071518 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.071455    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7efd9d04-351f-4e16-98c6-a8acf3076b07-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7efd9d04-351f-4e16-98c6-a8acf3076b07" (UID: "7efd9d04-351f-4e16-98c6-a8acf3076b07"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:47:53.071518 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.071473    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7efd9d04-351f-4e16-98c6-a8acf3076b07-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7efd9d04-351f-4e16-98c6-a8acf3076b07" (UID: "7efd9d04-351f-4e16-98c6-a8acf3076b07"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:47:53.071681 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.071642    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7efd9d04-351f-4e16-98c6-a8acf3076b07-workload-socket\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:47:53.071681 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.071663    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7efd9d04-351f-4e16-98c6-a8acf3076b07-workload-certs\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:47:53.071681 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.071655    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7efd9d04-351f-4e16-98c6-a8acf3076b07-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7efd9d04-351f-4e16-98c6-a8acf3076b07" (UID: "7efd9d04-351f-4e16-98c6-a8acf3076b07"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 16:47:53.071681 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.071678    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7efd9d04-351f-4e16-98c6-a8acf3076b07-credential-socket\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:47:53.072212 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.072192    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7efd9d04-351f-4e16-98c6-a8acf3076b07-istio-data" (OuterVolumeSpecName: "istio-data") pod "7efd9d04-351f-4e16-98c6-a8acf3076b07" (UID: "7efd9d04-351f-4e16-98c6-a8acf3076b07"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:47:53.073313 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.073289    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7efd9d04-351f-4e16-98c6-a8acf3076b07-istio-token" (OuterVolumeSpecName: "istio-token") pod "7efd9d04-351f-4e16-98c6-a8acf3076b07" (UID: "7efd9d04-351f-4e16-98c6-a8acf3076b07"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:47:53.073607 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.073586    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7efd9d04-351f-4e16-98c6-a8acf3076b07-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7efd9d04-351f-4e16-98c6-a8acf3076b07" (UID: "7efd9d04-351f-4e16-98c6-a8acf3076b07"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 16:47:53.073778 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.073756    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7efd9d04-351f-4e16-98c6-a8acf3076b07-kube-api-access-flp2d" (OuterVolumeSpecName: "kube-api-access-flp2d") pod "7efd9d04-351f-4e16-98c6-a8acf3076b07" (UID: "7efd9d04-351f-4e16-98c6-a8acf3076b07"). InnerVolumeSpecName "kube-api-access-flp2d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:47:53.073778 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.073767    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7efd9d04-351f-4e16-98c6-a8acf3076b07-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7efd9d04-351f-4e16-98c6-a8acf3076b07" (UID: "7efd9d04-351f-4e16-98c6-a8acf3076b07"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:47:53.172705 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.172672    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7efd9d04-351f-4e16-98c6-a8acf3076b07-istio-token\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:47:53.172705 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.172700    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7efd9d04-351f-4e16-98c6-a8acf3076b07-istiod-ca-cert\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:47:53.172705 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.172711    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7efd9d04-351f-4e16-98c6-a8acf3076b07-istio-envoy\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:47:53.172965 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.172719    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7efd9d04-351f-4e16-98c6-a8acf3076b07-istio-podinfo\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:47:53.172965 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.172727    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7efd9d04-351f-4e16-98c6-a8acf3076b07-istio-data\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:47:53.172965 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.172736    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-flp2d\" (UniqueName: \"kubernetes.io/projected/7efd9d04-351f-4e16-98c6-a8acf3076b07-kube-api-access-flp2d\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:47:53.376683 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.376643    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"]
Jul 22 16:47:53.381219 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:53.381190    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-f0cd70-openshift-default-6d76499c99-l959g"]
Jul 22 16:47:54.447682 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:54.447652    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7efd9d04-351f-4e16-98c6-a8acf3076b07" path="/var/lib/kubelet/pods/7efd9d04-351f-4e16-98c6-a8acf3076b07/volumes"
Jul 22 16:47:58.851226 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:58.851189    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:47:58.955126 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:47:58.955087    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:48:04.098909 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:04.098869    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-285a31-779c874df9-bp7jd"]
Jul 22 16:48:04.100295 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:04.099136    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-285a31-779c874df9-bp7jd" podUID="b666c3e8-fc5f-4854-90aa-35591fc60cd9" containerName="maas-api" containerID="cri-o://3dcd2c148299ade45793b4c63a3373f0a58c38180ad6aa27b94878461f5a7d07" gracePeriod=30
Jul 22 16:48:04.134168 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:04.134138    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:48:04.346225 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:04.346200    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-285a31-779c874df9-bp7jd"
Jul 22 16:48:04.464551 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:04.464515    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b666c3e8-fc5f-4854-90aa-35591fc60cd9-openshift-service-ca\") pod \"b666c3e8-fc5f-4854-90aa-35591fc60cd9\" (UID: \"b666c3e8-fc5f-4854-90aa-35591fc60cd9\") "
Jul 22 16:48:04.464724 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:04.464563    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hz794\" (UniqueName: \"kubernetes.io/projected/b666c3e8-fc5f-4854-90aa-35591fc60cd9-kube-api-access-hz794\") pod \"b666c3e8-fc5f-4854-90aa-35591fc60cd9\" (UID: \"b666c3e8-fc5f-4854-90aa-35591fc60cd9\") "
Jul 22 16:48:04.464724 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:04.464606    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b666c3e8-fc5f-4854-90aa-35591fc60cd9-maas-api-tls\") pod \"b666c3e8-fc5f-4854-90aa-35591fc60cd9\" (UID: \"b666c3e8-fc5f-4854-90aa-35591fc60cd9\") "
Jul 22 16:48:04.464926 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:04.464896    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b666c3e8-fc5f-4854-90aa-35591fc60cd9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b666c3e8-fc5f-4854-90aa-35591fc60cd9" (UID: "b666c3e8-fc5f-4854-90aa-35591fc60cd9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 16:48:04.466602 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:04.466569    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b666c3e8-fc5f-4854-90aa-35591fc60cd9-kube-api-access-hz794" (OuterVolumeSpecName: "kube-api-access-hz794") pod "b666c3e8-fc5f-4854-90aa-35591fc60cd9" (UID: "b666c3e8-fc5f-4854-90aa-35591fc60cd9"). InnerVolumeSpecName "kube-api-access-hz794". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:48:04.466703 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:04.466653    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b666c3e8-fc5f-4854-90aa-35591fc60cd9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b666c3e8-fc5f-4854-90aa-35591fc60cd9" (UID: "b666c3e8-fc5f-4854-90aa-35591fc60cd9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:48:04.565234 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:04.565198    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b666c3e8-fc5f-4854-90aa-35591fc60cd9-maas-api-tls\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:48:04.565234 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:04.565230    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b666c3e8-fc5f-4854-90aa-35591fc60cd9-openshift-service-ca\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:48:04.565234 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:04.565240    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hz794\" (UniqueName: \"kubernetes.io/projected/b666c3e8-fc5f-4854-90aa-35591fc60cd9-kube-api-access-hz794\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:48:05.107097 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:05.107065    2563 generic.go:356] "Generic (PLEG): container finished" podID="b666c3e8-fc5f-4854-90aa-35591fc60cd9" containerID="3dcd2c148299ade45793b4c63a3373f0a58c38180ad6aa27b94878461f5a7d07" exitCode=0
Jul 22 16:48:05.107607 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:05.107110    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-285a31-779c874df9-bp7jd"
Jul 22 16:48:05.107607 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:05.107158    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-285a31-779c874df9-bp7jd" event={"ID":"b666c3e8-fc5f-4854-90aa-35591fc60cd9","Type":"ContainerDied","Data":"3dcd2c148299ade45793b4c63a3373f0a58c38180ad6aa27b94878461f5a7d07"}
Jul 22 16:48:05.107607 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:05.107213    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-285a31-779c874df9-bp7jd" event={"ID":"b666c3e8-fc5f-4854-90aa-35591fc60cd9","Type":"ContainerDied","Data":"e44e4f67f5310c92459dfc8afeb6e3f69426e9de5af4f17ea48010fbdf74d260"}
Jul 22 16:48:05.107607 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:05.107236    2563 scope.go:117] "RemoveContainer" containerID="3dcd2c148299ade45793b4c63a3373f0a58c38180ad6aa27b94878461f5a7d07"
Jul 22 16:48:05.116576 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:05.116560    2563 scope.go:117] "RemoveContainer" containerID="3dcd2c148299ade45793b4c63a3373f0a58c38180ad6aa27b94878461f5a7d07"
Jul 22 16:48:05.116840 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:48:05.116821    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dcd2c148299ade45793b4c63a3373f0a58c38180ad6aa27b94878461f5a7d07\": container with ID starting with 3dcd2c148299ade45793b4c63a3373f0a58c38180ad6aa27b94878461f5a7d07 not found: ID does not exist" containerID="3dcd2c148299ade45793b4c63a3373f0a58c38180ad6aa27b94878461f5a7d07"
Jul 22 16:48:05.116916 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:05.116866    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dcd2c148299ade45793b4c63a3373f0a58c38180ad6aa27b94878461f5a7d07"} err="failed to get container status \"3dcd2c148299ade45793b4c63a3373f0a58c38180ad6aa27b94878461f5a7d07\": rpc error: code = NotFound desc = could not find container \"3dcd2c148299ade45793b4c63a3373f0a58c38180ad6aa27b94878461f5a7d07\": container with ID starting with 3dcd2c148299ade45793b4c63a3373f0a58c38180ad6aa27b94878461f5a7d07 not found: ID does not exist"
Jul 22 16:48:05.128776 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:05.128750    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-285a31-779c874df9-bp7jd"]
Jul 22 16:48:05.132904 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:05.132879    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-285a31-779c874df9-bp7jd"]
Jul 22 16:48:06.449015 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:06.448981    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b666c3e8-fc5f-4854-90aa-35591fc60cd9" path="/var/lib/kubelet/pods/b666c3e8-fc5f-4854-90aa-35591fc60cd9/volumes"
Jul 22 16:48:08.224787 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:08.224754    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:48:08.631225 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:08.631140    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:48:14.501818 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:14.501772    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:48:14.568710 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:14.568676    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:48:14.777684 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:14.777597    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:48:15.008188 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.008157    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:48:15.356413 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.356377    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:48:15.679569 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.679539    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"]
Jul 22 16:48:15.680061 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.679845    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7efd9d04-351f-4e16-98c6-a8acf3076b07" containerName="istio-proxy"
Jul 22 16:48:15.680061 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.679881    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7efd9d04-351f-4e16-98c6-a8acf3076b07" containerName="istio-proxy"
Jul 22 16:48:15.680061 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.679894    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7efd9d04-351f-4e16-98c6-a8acf3076b07" containerName="istio-proxy"
Jul 22 16:48:15.680061 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.679900    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7efd9d04-351f-4e16-98c6-a8acf3076b07" containerName="istio-proxy"
Jul 22 16:48:15.680061 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.679912    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b666c3e8-fc5f-4854-90aa-35591fc60cd9" containerName="maas-api"
Jul 22 16:48:15.680061 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.679917    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b666c3e8-fc5f-4854-90aa-35591fc60cd9" containerName="maas-api"
Jul 22 16:48:15.680061 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.679929    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60ee3390-8856-4856-b154-4438760e8d89" containerName="maas-api"
Jul 22 16:48:15.680061 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.679934    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="60ee3390-8856-4856-b154-4438760e8d89" containerName="maas-api"
Jul 22 16:48:15.680061 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.679984    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="60ee3390-8856-4856-b154-4438760e8d89" containerName="maas-api"
Jul 22 16:48:15.680061 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.679992    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b666c3e8-fc5f-4854-90aa-35591fc60cd9" containerName="maas-api"
Jul 22 16:48:15.680061 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.679999    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7efd9d04-351f-4e16-98c6-a8acf3076b07" containerName="istio-proxy"
Jul 22 16:48:15.684392 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.684374    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:15.688974 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.688950    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 22 16:48:15.689223 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.689208    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-ff2a05-openshift-default-dockercfg-hm57s\""
Jul 22 16:48:15.697674 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.697653    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"]
Jul 22 16:48:15.723464 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.723423    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:48:15.862826 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.862790    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/482e44f8-dec4-4797-89f0-6804ec7b3738-istiod-ca-cert\") pod \"e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") " pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:15.863062 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.862835    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/482e44f8-dec4-4797-89f0-6804ec7b3738-workload-socket\") pod \"e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") " pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:15.863062 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.862885    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/482e44f8-dec4-4797-89f0-6804ec7b3738-credential-socket\") pod \"e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") " pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:15.863062 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.862908    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/482e44f8-dec4-4797-89f0-6804ec7b3738-istio-data\") pod \"e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") " pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:15.863062 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.862938    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/482e44f8-dec4-4797-89f0-6804ec7b3738-istio-envoy\") pod \"e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") " pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:15.863062 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.863005    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/482e44f8-dec4-4797-89f0-6804ec7b3738-istio-token\") pod \"e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") " pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:15.863062 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.863052    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/482e44f8-dec4-4797-89f0-6804ec7b3738-workload-certs\") pod \"e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") " pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:15.863388 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.863077    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/482e44f8-dec4-4797-89f0-6804ec7b3738-istio-podinfo\") pod \"e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") " pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:15.863388 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.863099    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs6hd\" (UniqueName: \"kubernetes.io/projected/482e44f8-dec4-4797-89f0-6804ec7b3738-kube-api-access-zs6hd\") pod \"e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") " pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:15.964476 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.964385    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/482e44f8-dec4-4797-89f0-6804ec7b3738-istiod-ca-cert\") pod \"e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") " pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:15.964476 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.964446    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/482e44f8-dec4-4797-89f0-6804ec7b3738-workload-socket\") pod \"e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") " pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:15.964476 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.964476    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/482e44f8-dec4-4797-89f0-6804ec7b3738-credential-socket\") pod \"e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") " pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:15.964755 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.964506    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/482e44f8-dec4-4797-89f0-6804ec7b3738-istio-data\") pod \"e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") " pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:15.964755 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.964543    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/482e44f8-dec4-4797-89f0-6804ec7b3738-istio-envoy\") pod \"e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") " pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:15.964755 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.964598    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/482e44f8-dec4-4797-89f0-6804ec7b3738-istio-token\") pod \"e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") " pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:15.964755 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.964628    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/482e44f8-dec4-4797-89f0-6804ec7b3738-workload-certs\") pod \"e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") " pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:15.964755 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.964653    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/482e44f8-dec4-4797-89f0-6804ec7b3738-istio-podinfo\") pod \"e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") " pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:15.964755 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.964677    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs6hd\" (UniqueName: \"kubernetes.io/projected/482e44f8-dec4-4797-89f0-6804ec7b3738-kube-api-access-zs6hd\") pod \"e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") " pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:15.965073 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.964966    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/482e44f8-dec4-4797-89f0-6804ec7b3738-workload-socket\") pod \"e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") " pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:15.965230 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.965209    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/482e44f8-dec4-4797-89f0-6804ec7b3738-istio-data\") pod \"e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") " pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:15.965314 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.965278    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/482e44f8-dec4-4797-89f0-6804ec7b3738-credential-socket\") pod \"e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") " pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:15.965366 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.965329    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/482e44f8-dec4-4797-89f0-6804ec7b3738-istiod-ca-cert\") pod \"e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") " pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:15.965405 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.965383    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/482e44f8-dec4-4797-89f0-6804ec7b3738-workload-certs\") pod \"e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") " pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:15.967064 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.967045    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/482e44f8-dec4-4797-89f0-6804ec7b3738-istio-envoy\") pod \"e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") " pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:15.967201 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.967178    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/482e44f8-dec4-4797-89f0-6804ec7b3738-istio-podinfo\") pod \"e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") " pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:15.977337 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.977312    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/482e44f8-dec4-4797-89f0-6804ec7b3738-istio-token\") pod \"e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") " pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:15.977466 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.977447    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs6hd\" (UniqueName: \"kubernetes.io/projected/482e44f8-dec4-4797-89f0-6804ec7b3738-kube-api-access-zs6hd\") pod \"e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") " pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:15.994368 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:15.994342    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:16.152510 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:16.152482    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"]
Jul 22 16:48:16.154037 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:48:16.154009    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod482e44f8_dec4_4797_89f0_6804ec7b3738.slice/crio-273bdc1e8d5b9c02f16d77c62ec8da1ce3957e4bf29ddecef78710490fa22c03 WatchSource:0}: Error finding container 273bdc1e8d5b9c02f16d77c62ec8da1ce3957e4bf29ddecef78710490fa22c03: Status 404 returned error can't find the container with id 273bdc1e8d5b9c02f16d77c62ec8da1ce3957e4bf29ddecef78710490fa22c03
Jul 22 16:48:16.156232 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:16.156196    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:48:16.156325 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:16.156274    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:48:16.156325 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:16.156304    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:48:16.160061 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:16.160037    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj" event={"ID":"482e44f8-dec4-4797-89f0-6804ec7b3738","Type":"ContainerStarted","Data":"273bdc1e8d5b9c02f16d77c62ec8da1ce3957e4bf29ddecef78710490fa22c03"}
Jul 22 16:48:16.267741 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:16.267702    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:48:17.166223 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:17.166192    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj" event={"ID":"482e44f8-dec4-4797-89f0-6804ec7b3738","Type":"ContainerStarted","Data":"791533f07cf02379fa7501a826b1b2088de091ecb01c0d7b01d58da84184cd7f"}
Jul 22 16:48:17.188865 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:17.188802    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj" podStartSLOduration=2.188782981 podStartE2EDuration="2.188782981s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:48:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:48:17.188509685 +0000 UTC m=+2457.318979687" watchObservedRunningTime="2026-07-22 16:48:17.188782981 +0000 UTC m=+2457.319252986"
Jul 22 16:48:17.994862 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:17.994825    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:17.999758 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:17.999733    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:18.170424 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:18.170393    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:18.171498 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:18.171478    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:19.139777 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:19.139745    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:48:19.251881 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:19.251839    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:48:19.258448 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:19.258415    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc"]
Jul 22 16:48:19.258885 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:19.258866    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7efd9d04-351f-4e16-98c6-a8acf3076b07" containerName="istio-proxy"
Jul 22 16:48:19.261969 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:19.261948    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc"
Jul 22 16:48:19.266179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:19.266155    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 22 16:48:19.266305 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:19.266160    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 16:48:19.269287 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:19.269037    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-ff2a05-serving-cert\""
Jul 22 16:48:19.269287 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:19.269188    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vghv2\""
Jul 22 16:48:19.282394 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:19.282367    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc"]
Jul 22 16:48:19.293815 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:19.293788    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a187da1-a115-4464-9e71-f49a19c1e112-openshift-service-ca\") pod \"maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc\" (UID: \"7a187da1-a115-4464-9e71-f49a19c1e112\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc"
Jul 22 16:48:19.293940 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:19.293829    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a187da1-a115-4464-9e71-f49a19c1e112-maas-api-tls\") pod \"maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc\" (UID: \"7a187da1-a115-4464-9e71-f49a19c1e112\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc"
Jul 22 16:48:19.293983 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:19.293945    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljdp4\" (UniqueName: \"kubernetes.io/projected/7a187da1-a115-4464-9e71-f49a19c1e112-kube-api-access-ljdp4\") pod \"maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc\" (UID: \"7a187da1-a115-4464-9e71-f49a19c1e112\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc"
Jul 22 16:48:19.345522 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:19.345484    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:48:19.395065 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:19.394973    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljdp4\" (UniqueName: \"kubernetes.io/projected/7a187da1-a115-4464-9e71-f49a19c1e112-kube-api-access-ljdp4\") pod \"maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc\" (UID: \"7a187da1-a115-4464-9e71-f49a19c1e112\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc"
Jul 22 16:48:19.395065 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:19.395022    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a187da1-a115-4464-9e71-f49a19c1e112-openshift-service-ca\") pod \"maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc\" (UID: \"7a187da1-a115-4464-9e71-f49a19c1e112\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc"
Jul 22 16:48:19.395065 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:19.395052    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a187da1-a115-4464-9e71-f49a19c1e112-maas-api-tls\") pod \"maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc\" (UID: \"7a187da1-a115-4464-9e71-f49a19c1e112\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc"
Jul 22 16:48:19.395757 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:19.395734    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a187da1-a115-4464-9e71-f49a19c1e112-openshift-service-ca\") pod \"maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc\" (UID: \"7a187da1-a115-4464-9e71-f49a19c1e112\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc"
Jul 22 16:48:19.397590 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:19.397567    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a187da1-a115-4464-9e71-f49a19c1e112-maas-api-tls\") pod \"maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc\" (UID: \"7a187da1-a115-4464-9e71-f49a19c1e112\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc"
Jul 22 16:48:19.408481 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:19.408455    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljdp4\" (UniqueName: \"kubernetes.io/projected/7a187da1-a115-4464-9e71-f49a19c1e112-kube-api-access-ljdp4\") pod \"maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc\" (UID: \"7a187da1-a115-4464-9e71-f49a19c1e112\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc"
Jul 22 16:48:19.441607 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:19.441568    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:48:19.573267 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:19.573209    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc"
Jul 22 16:48:19.711311 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:19.711285    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc"]
Jul 22 16:48:19.714040 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:48:19.714014    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a187da1_a115_4464_9e71_f49a19c1e112.slice/crio-85c85ef14a4b99ca051b0f8405170a36a74046b0b6a272881ec2a5faac02c8fd WatchSource:0}: Error finding container 85c85ef14a4b99ca051b0f8405170a36a74046b0b6a272881ec2a5faac02c8fd: Status 404 returned error can't find the container with id 85c85ef14a4b99ca051b0f8405170a36a74046b0b6a272881ec2a5faac02c8fd
Jul 22 16:48:19.876299 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:19.876263    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:48:19.975478 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:19.975449    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:48:20.181766 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:20.181736    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc" event={"ID":"7a187da1-a115-4464-9e71-f49a19c1e112","Type":"ContainerStarted","Data":"ec0d446043992b926b677c5e3644f279af96a68b8f6514190af185254b8f516a"}
Jul 22 16:48:20.181967 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:20.181786    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc" event={"ID":"7a187da1-a115-4464-9e71-f49a19c1e112","Type":"ContainerStarted","Data":"85c85ef14a4b99ca051b0f8405170a36a74046b0b6a272881ec2a5faac02c8fd"}
Jul 22 16:48:20.200551 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:20.200506    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc" podStartSLOduration=0.805449266 podStartE2EDuration="1.200490234s" totalImagesPullingTime="395.040968ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:48:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:48:20.199365875 +0000 UTC m=+2460.329835874" watchObservedRunningTime="2026-07-22 16:48:20.200490234 +0000 UTC m=+2460.330960237"
Jul 22 16:48:21.186426 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:21.186396    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc"
Jul 22 16:48:23.633265 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:23.633233    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:48:23.954374 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:23.954341    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:48:24.269670 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:24.269592    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:48:24.437418 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:24.437388    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:48:27.165824 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:27.165792    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:48:27.196349 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:27.196312    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc"
Jul 22 16:48:27.274443 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:27.274404    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:48:27.279327 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:27.279297    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr"]
Jul 22 16:48:27.282546 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:27.282526    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr"
Jul 22 16:48:27.285239 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:27.285215    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-afe7a0-serving-cert\""
Jul 22 16:48:27.294257 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:27.294234    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr"]
Jul 22 16:48:27.365189 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:27.365146    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzx2f\" (UniqueName: \"kubernetes.io/projected/7c251985-c1ec-4c42-b249-518d75a4870a-kube-api-access-kzx2f\") pod \"maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr\" (UID: \"7c251985-c1ec-4c42-b249-518d75a4870a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr"
Jul 22 16:48:27.365189 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:27.365196    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c251985-c1ec-4c42-b249-518d75a4870a-maas-api-tls\") pod \"maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr\" (UID: \"7c251985-c1ec-4c42-b249-518d75a4870a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr"
Jul 22 16:48:27.365456 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:27.365295    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c251985-c1ec-4c42-b249-518d75a4870a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr\" (UID: \"7c251985-c1ec-4c42-b249-518d75a4870a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr"
Jul 22 16:48:27.410945 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:27.410895    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:48:27.466687 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:27.466653    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c251985-c1ec-4c42-b249-518d75a4870a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr\" (UID: \"7c251985-c1ec-4c42-b249-518d75a4870a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr"
Jul 22 16:48:27.466941 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:27.466704    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzx2f\" (UniqueName: \"kubernetes.io/projected/7c251985-c1ec-4c42-b249-518d75a4870a-kube-api-access-kzx2f\") pod \"maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr\" (UID: \"7c251985-c1ec-4c42-b249-518d75a4870a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr"
Jul 22 16:48:27.466941 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:27.466742    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c251985-c1ec-4c42-b249-518d75a4870a-maas-api-tls\") pod \"maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr\" (UID: \"7c251985-c1ec-4c42-b249-518d75a4870a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr"
Jul 22 16:48:27.467443 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:27.467418    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c251985-c1ec-4c42-b249-518d75a4870a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr\" (UID: \"7c251985-c1ec-4c42-b249-518d75a4870a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr"
Jul 22 16:48:27.469396 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:27.469374    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c251985-c1ec-4c42-b249-518d75a4870a-maas-api-tls\") pod \"maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr\" (UID: \"7c251985-c1ec-4c42-b249-518d75a4870a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr"
Jul 22 16:48:27.476785 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:27.476762    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzx2f\" (UniqueName: \"kubernetes.io/projected/7c251985-c1ec-4c42-b249-518d75a4870a-kube-api-access-kzx2f\") pod \"maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr\" (UID: \"7c251985-c1ec-4c42-b249-518d75a4870a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr"
Jul 22 16:48:27.517146 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:27.517051    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:48:27.594124 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:27.594092    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr"
Jul 22 16:48:27.778101 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:27.778071    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr"]
Jul 22 16:48:27.779228 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:48:27.779207    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c251985_c1ec_4c42_b249_518d75a4870a.slice/crio-bcc1c6c4da79130290823d799d445f31964844e694e7aed56357be076dd42c81 WatchSource:0}: Error finding container bcc1c6c4da79130290823d799d445f31964844e694e7aed56357be076dd42c81: Status 404 returned error can't find the container with id bcc1c6c4da79130290823d799d445f31964844e694e7aed56357be076dd42c81
Jul 22 16:48:28.030686 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:28.030561    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:48:28.132247 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:28.132202    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:48:28.222133 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:28.222046    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr" event={"ID":"7c251985-c1ec-4c42-b249-518d75a4870a","Type":"ContainerStarted","Data":"bcc1c6c4da79130290823d799d445f31964844e694e7aed56357be076dd42c81"}
Jul 22 16:48:29.231125 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:29.231072    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr" event={"ID":"7c251985-c1ec-4c42-b249-518d75a4870a","Type":"ContainerStarted","Data":"bea3905a2a003d6597522f51df1790993faee84f0dccdd16d018e3db19ca92fe"}
Jul 22 16:48:29.232680 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:29.232656    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr"
Jul 22 16:48:29.252179 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:29.252131    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr" podStartSLOduration=1.879231124 podStartE2EDuration="2.252114575s" totalImagesPullingTime="372.883451ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:48:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:48:29.251248491 +0000 UTC m=+2469.381718511" watchObservedRunningTime="2026-07-22 16:48:29.252114575 +0000 UTC m=+2469.382584581"
Jul 22 16:48:31.648047 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:31.648016    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:48:34.211065 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:34.211021    2563 patch_prober.go:28] interesting pod/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 22 16:48:34.211495 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:34.211094    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj" podUID="482e44f8-dec4-4797-89f0-6804ec7b3738" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 22 16:48:35.259567 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:35.259535    2563 generic.go:356] "Generic (PLEG): container finished" podID="482e44f8-dec4-4797-89f0-6804ec7b3738" containerID="791533f07cf02379fa7501a826b1b2088de091ecb01c0d7b01d58da84184cd7f" exitCode=0
Jul 22 16:48:35.260007 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:35.259609    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj" event={"ID":"482e44f8-dec4-4797-89f0-6804ec7b3738","Type":"ContainerDied","Data":"791533f07cf02379fa7501a826b1b2088de091ecb01c0d7b01d58da84184cd7f"}
Jul 22 16:48:35.260007 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:35.259998    2563 scope.go:117] "RemoveContainer" containerID="791533f07cf02379fa7501a826b1b2088de091ecb01c0d7b01d58da84184cd7f"
Jul 22 16:48:35.261084 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:35.261052    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:48:35.261161 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:35.261142    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:48:35.261201 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:35.261189    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 22 16:48:36.246140 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:36.246107    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr"
Jul 22 16:48:36.267321 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:36.267285    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj" event={"ID":"482e44f8-dec4-4797-89f0-6804ec7b3738","Type":"ContainerStarted","Data":"130f21c8f31412f47461f094c9e5362e2831871cea9f8aec2f07fbce30aba019"}
Jul 22 16:48:36.995074 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:36.995027    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:36.995325 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:36.995301    2563 patch_prober.go:28] interesting pod/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 22 16:48:36.995418 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:36.995359    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj" podUID="482e44f8-dec4-4797-89f0-6804ec7b3738" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 22 16:48:37.995182 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:37.995145    2563 patch_prober.go:28] interesting pod/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 22 16:48:37.995604 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:37.995203    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj" podUID="482e44f8-dec4-4797-89f0-6804ec7b3738" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 22 16:48:38.995017 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:38.994986    2563 patch_prober.go:28] interesting pod/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 22 16:48:38.995178 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:38.995040    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj" podUID="482e44f8-dec4-4797-89f0-6804ec7b3738" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 22 16:48:39.995304 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:39.995267    2563 patch_prober.go:28] interesting pod/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 22 16:48:39.995773 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:39.995337    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj" podUID="482e44f8-dec4-4797-89f0-6804ec7b3738" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 22 16:48:40.998947 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:40.998913    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:40.999419 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:40.999312    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:40.999947 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:40.999928    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:48:41.631490 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:41.631439    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:48:41.730514 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:41.730484    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:48:41.838183 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:41.838152    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf"]
Jul 22 16:48:41.840478 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:41.840458    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf"
Jul 22 16:48:41.845061 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:41.845035    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ff2a05\"/\"auth-test-model-ff2a05-kserve-self-signed-certs\""
Jul 22 16:48:41.845061 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:41.845053    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ff2a05\"/\"default-dockercfg-ghqpn\""
Jul 22 16:48:41.845232 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:41.845035    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ff2a05\"/\"kube-root-ca.crt\""
Jul 22 16:48:41.845232 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:41.845063    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ff2a05\"/\"openshift-service-ca.crt\""
Jul 22 16:48:41.852667 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:41.852641    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf"]
Jul 22 16:48:41.866618 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:41.866594    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:48:41.895050 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:41.894963    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d5502cc-8450-43d2-8dcf-df2cb65ac928-home\") pod \"auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf\" (UID: \"5d5502cc-8450-43d2-8dcf-df2cb65ac928\") " pod="ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf"
Jul 22 16:48:41.895050 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:41.894996    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d5502cc-8450-43d2-8dcf-df2cb65ac928-model-cache\") pod \"auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf\" (UID: \"5d5502cc-8450-43d2-8dcf-df2cb65ac928\") " pod="ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf"
Jul 22 16:48:41.895050 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:41.895032    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d5502cc-8450-43d2-8dcf-df2cb65ac928-dshm\") pod \"auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf\" (UID: \"5d5502cc-8450-43d2-8dcf-df2cb65ac928\") " pod="ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf"
Jul 22 16:48:41.895282 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:41.895113    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5fsw\" (UniqueName: \"kubernetes.io/projected/5d5502cc-8450-43d2-8dcf-df2cb65ac928-kube-api-access-w5fsw\") pod \"auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf\" (UID: \"5d5502cc-8450-43d2-8dcf-df2cb65ac928\") " pod="ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf"
Jul 22 16:48:41.895282 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:41.895153    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d5502cc-8450-43d2-8dcf-df2cb65ac928-tmp-dir\") pod \"auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf\" (UID: \"5d5502cc-8450-43d2-8dcf-df2cb65ac928\") " pod="ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf"
Jul 22 16:48:41.895282 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:41.895205    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d5502cc-8450-43d2-8dcf-df2cb65ac928-tls-certs\") pod \"auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf\" (UID: \"5d5502cc-8450-43d2-8dcf-df2cb65ac928\") " pod="ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf"
Jul 22 16:48:41.996561 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:41.996522    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d5502cc-8450-43d2-8dcf-df2cb65ac928-tls-certs\") pod \"auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf\" (UID: \"5d5502cc-8450-43d2-8dcf-df2cb65ac928\") " pod="ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf"
Jul 22 16:48:41.996763 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:41.996583    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d5502cc-8450-43d2-8dcf-df2cb65ac928-home\") pod \"auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf\" (UID: \"5d5502cc-8450-43d2-8dcf-df2cb65ac928\") " pod="ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf"
Jul 22 16:48:41.996763 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:41.996599    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d5502cc-8450-43d2-8dcf-df2cb65ac928-model-cache\") pod \"auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf\" (UID: \"5d5502cc-8450-43d2-8dcf-df2cb65ac928\") " pod="ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf"
Jul 22 16:48:41.996763 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:41.996635    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d5502cc-8450-43d2-8dcf-df2cb65ac928-dshm\") pod \"auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf\" (UID: \"5d5502cc-8450-43d2-8dcf-df2cb65ac928\") " pod="ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf"
Jul 22 16:48:41.996763 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:41.996666    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5fsw\" (UniqueName: \"kubernetes.io/projected/5d5502cc-8450-43d2-8dcf-df2cb65ac928-kube-api-access-w5fsw\") pod \"auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf\" (UID: \"5d5502cc-8450-43d2-8dcf-df2cb65ac928\") " pod="ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf"
Jul 22 16:48:41.996763 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:41.996701    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d5502cc-8450-43d2-8dcf-df2cb65ac928-tmp-dir\") pod \"auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf\" (UID: \"5d5502cc-8450-43d2-8dcf-df2cb65ac928\") " pod="ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf"
Jul 22 16:48:41.997212 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:41.997185    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d5502cc-8450-43d2-8dcf-df2cb65ac928-model-cache\") pod \"auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf\" (UID: \"5d5502cc-8450-43d2-8dcf-df2cb65ac928\") " pod="ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf"
Jul 22 16:48:41.997212 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:41.997206    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d5502cc-8450-43d2-8dcf-df2cb65ac928-tmp-dir\") pod \"auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf\" (UID: \"5d5502cc-8450-43d2-8dcf-df2cb65ac928\") " pod="ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf"
Jul 22 16:48:41.997425 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:41.997285    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d5502cc-8450-43d2-8dcf-df2cb65ac928-home\") pod \"auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf\" (UID: \"5d5502cc-8450-43d2-8dcf-df2cb65ac928\") " pod="ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf"
Jul 22 16:48:41.999004 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:41.998981    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d5502cc-8450-43d2-8dcf-df2cb65ac928-dshm\") pod \"auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf\" (UID: \"5d5502cc-8450-43d2-8dcf-df2cb65ac928\") " pod="ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf"
Jul 22 16:48:41.999314 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:41.999160    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d5502cc-8450-43d2-8dcf-df2cb65ac928-tls-certs\") pod \"auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf\" (UID: \"5d5502cc-8450-43d2-8dcf-df2cb65ac928\") " pod="ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf"
Jul 22 16:48:42.008491 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:42.008468    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5fsw\" (UniqueName: \"kubernetes.io/projected/5d5502cc-8450-43d2-8dcf-df2cb65ac928-kube-api-access-w5fsw\") pod \"auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf\" (UID: \"5d5502cc-8450-43d2-8dcf-df2cb65ac928\") " pod="ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf"
Jul 22 16:48:42.150536 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:42.150445    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf"
Jul 22 16:48:42.284910 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:42.284884    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf"]
Jul 22 16:48:42.286948 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:48:42.286917    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5d5502cc_8450_43d2_8dcf_df2cb65ac928.slice/crio-5730707d4d577b3b17dd0018c3cf1d07c73cc342f60172f67c74cc806cbbcc8d WatchSource:0}: Error finding container 5730707d4d577b3b17dd0018c3cf1d07c73cc342f60172f67c74cc806cbbcc8d: Status 404 returned error can't find the container with id 5730707d4d577b3b17dd0018c3cf1d07c73cc342f60172f67c74cc806cbbcc8d
Jul 22 16:48:42.295634 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:42.295610    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf" event={"ID":"5d5502cc-8450-43d2-8dcf-df2cb65ac928","Type":"ContainerStarted","Data":"5730707d4d577b3b17dd0018c3cf1d07c73cc342f60172f67c74cc806cbbcc8d"}
Jul 22 16:48:43.301329 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:43.301294    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf" event={"ID":"5d5502cc-8450-43d2-8dcf-df2cb65ac928","Type":"ContainerStarted","Data":"f4b1d45add56548df882955b59ac55d570b17ec6e15d03387b0e64375b99e3cf"}
Jul 22 16:48:43.323348 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:43.323288    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf" podStartSLOduration=2.323270808 podStartE2EDuration="2.323270808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:48:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:48:43.321740443 +0000 UTC m=+2483.452210445" watchObservedRunningTime="2026-07-22 16:48:43.323270808 +0000 UTC m=+2483.453740810"
Jul 22 16:48:49.039951 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:49.039873    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:48:49.141088 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:49.141050    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:48:52.151050 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:52.151014    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf"
Jul 22 16:48:52.151657 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:52.151062    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf"
Jul 22 16:48:52.163521 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:52.163489    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf"
Jul 22 16:48:52.352124 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:48:52.352092    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf"
Jul 22 16:49:30.245873 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:30.245769    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c"]
Jul 22 16:49:30.248383 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:30.248367    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c"
Jul 22 16:49:30.251884 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:30.251843    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-afe7a0\"/\"kube-root-ca.crt\""
Jul 22 16:49:30.254048 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:30.254024    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-afe7a0\"/\"default-dockercfg-zlhgh\""
Jul 22 16:49:30.254048 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:30.254038    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-afe7a0\"/\"openshift-service-ca.crt\""
Jul 22 16:49:30.254282 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:30.254177    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-afe7a0\"/\"auth-test-model-afe7a0-kserve-self-signed-certs\""
Jul 22 16:49:30.261727 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:30.261703    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c"]
Jul 22 16:49:30.271981 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:30.271924    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:49:30.329969 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:30.329934    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e933847-77e2-461e-9f99-8da31690dbc4-model-cache\") pod \"auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c\" (UID: \"3e933847-77e2-461e-9f99-8da31690dbc4\") " pod="ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c"
Jul 22 16:49:30.330150 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:30.329986    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e933847-77e2-461e-9f99-8da31690dbc4-tls-certs\") pod \"auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c\" (UID: \"3e933847-77e2-461e-9f99-8da31690dbc4\") " pod="ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c"
Jul 22 16:49:30.330150 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:30.330075    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e933847-77e2-461e-9f99-8da31690dbc4-home\") pod \"auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c\" (UID: \"3e933847-77e2-461e-9f99-8da31690dbc4\") " pod="ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c"
Jul 22 16:49:30.330150 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:30.330099    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e933847-77e2-461e-9f99-8da31690dbc4-dshm\") pod \"auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c\" (UID: \"3e933847-77e2-461e-9f99-8da31690dbc4\") " pod="ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c"
Jul 22 16:49:30.330150 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:30.330137    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e933847-77e2-461e-9f99-8da31690dbc4-tmp-dir\") pod \"auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c\" (UID: \"3e933847-77e2-461e-9f99-8da31690dbc4\") " pod="ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c"
Jul 22 16:49:30.330319 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:30.330174    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vmdj\" (UniqueName: \"kubernetes.io/projected/3e933847-77e2-461e-9f99-8da31690dbc4-kube-api-access-6vmdj\") pod \"auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c\" (UID: \"3e933847-77e2-461e-9f99-8da31690dbc4\") " pod="ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c"
Jul 22 16:49:30.431341 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:30.431298    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e933847-77e2-461e-9f99-8da31690dbc4-tls-certs\") pod \"auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c\" (UID: \"3e933847-77e2-461e-9f99-8da31690dbc4\") " pod="ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c"
Jul 22 16:49:30.431527 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:30.431355    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e933847-77e2-461e-9f99-8da31690dbc4-home\") pod \"auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c\" (UID: \"3e933847-77e2-461e-9f99-8da31690dbc4\") " pod="ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c"
Jul 22 16:49:30.431527 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:30.431476    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e933847-77e2-461e-9f99-8da31690dbc4-dshm\") pod \"auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c\" (UID: \"3e933847-77e2-461e-9f99-8da31690dbc4\") " pod="ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c"
Jul 22 16:49:30.431527 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:30.431522    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e933847-77e2-461e-9f99-8da31690dbc4-tmp-dir\") pod \"auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c\" (UID: \"3e933847-77e2-461e-9f99-8da31690dbc4\") " pod="ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c"
Jul 22 16:49:30.431659 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:30.431581    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vmdj\" (UniqueName: \"kubernetes.io/projected/3e933847-77e2-461e-9f99-8da31690dbc4-kube-api-access-6vmdj\") pod \"auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c\" (UID: \"3e933847-77e2-461e-9f99-8da31690dbc4\") " pod="ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c"
Jul 22 16:49:30.431787 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:30.431736    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e933847-77e2-461e-9f99-8da31690dbc4-model-cache\") pod \"auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c\" (UID: \"3e933847-77e2-461e-9f99-8da31690dbc4\") " pod="ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c"
Jul 22 16:49:30.431787 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:30.431761    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e933847-77e2-461e-9f99-8da31690dbc4-home\") pod \"auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c\" (UID: \"3e933847-77e2-461e-9f99-8da31690dbc4\") " pod="ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c"
Jul 22 16:49:30.432050 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:30.431929    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e933847-77e2-461e-9f99-8da31690dbc4-tmp-dir\") pod \"auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c\" (UID: \"3e933847-77e2-461e-9f99-8da31690dbc4\") " pod="ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c"
Jul 22 16:49:30.432050 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:30.432026    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e933847-77e2-461e-9f99-8da31690dbc4-model-cache\") pod \"auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c\" (UID: \"3e933847-77e2-461e-9f99-8da31690dbc4\") " pod="ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c"
Jul 22 16:49:30.433771 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:30.433749    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e933847-77e2-461e-9f99-8da31690dbc4-dshm\") pod \"auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c\" (UID: \"3e933847-77e2-461e-9f99-8da31690dbc4\") " pod="ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c"
Jul 22 16:49:30.434115 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:30.434084    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e933847-77e2-461e-9f99-8da31690dbc4-tls-certs\") pod \"auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c\" (UID: \"3e933847-77e2-461e-9f99-8da31690dbc4\") " pod="ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c"
Jul 22 16:49:30.441146 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:30.441123    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vmdj\" (UniqueName: \"kubernetes.io/projected/3e933847-77e2-461e-9f99-8da31690dbc4-kube-api-access-6vmdj\") pod \"auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c\" (UID: \"3e933847-77e2-461e-9f99-8da31690dbc4\") " pod="ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c"
Jul 22 16:49:30.558003 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:30.557911    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c"
Jul 22 16:49:30.693826 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:30.693800    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c"]
Jul 22 16:49:30.695992 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:49:30.695964    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3e933847_77e2_461e_9f99_8da31690dbc4.slice/crio-f1661cbfbcf8d539b2e1fc3071795f2208ab99fc7ec1c400a93c20db869f0834 WatchSource:0}: Error finding container f1661cbfbcf8d539b2e1fc3071795f2208ab99fc7ec1c400a93c20db869f0834: Status 404 returned error can't find the container with id f1661cbfbcf8d539b2e1fc3071795f2208ab99fc7ec1c400a93c20db869f0834
Jul 22 16:49:31.524082 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:31.524050    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c" event={"ID":"3e933847-77e2-461e-9f99-8da31690dbc4","Type":"ContainerStarted","Data":"ed5caff59d88b6722efd31444bb24982e8e07b2b55c9ae82cb44c37096f74dd3"}
Jul 22 16:49:31.524529 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:31.524093    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c" event={"ID":"3e933847-77e2-461e-9f99-8da31690dbc4","Type":"ContainerStarted","Data":"f1661cbfbcf8d539b2e1fc3071795f2208ab99fc7ec1c400a93c20db869f0834"}
Jul 22 16:49:31.545314 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:31.545263    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c" podStartSLOduration=1.545249027 podStartE2EDuration="1.545249027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:49:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:49:31.54328653 +0000 UTC m=+2531.673756543" watchObservedRunningTime="2026-07-22 16:49:31.545249027 +0000 UTC m=+2531.675719076"
Jul 22 16:49:40.558798 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:40.558761    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c"
Jul 22 16:49:40.558798 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:40.558806    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c"
Jul 22 16:49:40.572008 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:40.571977    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c"
Jul 22 16:49:40.583521 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:40.583489    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c"
Jul 22 16:49:56.239090 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:49:56.239058    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:51:16.552250 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:16.552118    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:51:35.346909 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:35.346877    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf"]
Jul 22 16:51:35.359074 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:35.359039    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:51:36.218596 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:36.218563    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c"]
Jul 22 16:51:36.246927 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:36.246899    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:51:37.889503 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:37.889464    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:51:37.989832 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:37.989794    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:51:41.965719 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:41.965681    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:51:50.367234 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:50.367172    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf" podUID="5d5502cc-8450-43d2-8dcf-df2cb65ac928" containerName="main" containerID="cri-o://f4b1d45add56548df882955b59ac55d570b17ec6e15d03387b0e64375b99e3cf" gracePeriod=45
Jul 22 16:51:50.621550 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:50.621486    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf"
Jul 22 16:51:50.680155 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:50.680123    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d5502cc-8450-43d2-8dcf-df2cb65ac928-model-cache\") pod \"5d5502cc-8450-43d2-8dcf-df2cb65ac928\" (UID: \"5d5502cc-8450-43d2-8dcf-df2cb65ac928\") "
Jul 22 16:51:50.680155 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:50.680160    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d5502cc-8450-43d2-8dcf-df2cb65ac928-dshm\") pod \"5d5502cc-8450-43d2-8dcf-df2cb65ac928\" (UID: \"5d5502cc-8450-43d2-8dcf-df2cb65ac928\") "
Jul 22 16:51:50.680397 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:50.680181    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w5fsw\" (UniqueName: \"kubernetes.io/projected/5d5502cc-8450-43d2-8dcf-df2cb65ac928-kube-api-access-w5fsw\") pod \"5d5502cc-8450-43d2-8dcf-df2cb65ac928\" (UID: \"5d5502cc-8450-43d2-8dcf-df2cb65ac928\") "
Jul 22 16:51:50.680397 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:50.680220    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d5502cc-8450-43d2-8dcf-df2cb65ac928-tls-certs\") pod \"5d5502cc-8450-43d2-8dcf-df2cb65ac928\" (UID: \"5d5502cc-8450-43d2-8dcf-df2cb65ac928\") "
Jul 22 16:51:50.680397 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:50.680243    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d5502cc-8450-43d2-8dcf-df2cb65ac928-home\") pod \"5d5502cc-8450-43d2-8dcf-df2cb65ac928\" (UID: \"5d5502cc-8450-43d2-8dcf-df2cb65ac928\") "
Jul 22 16:51:50.680572 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:50.680395    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d5502cc-8450-43d2-8dcf-df2cb65ac928-model-cache" (OuterVolumeSpecName: "model-cache") pod "5d5502cc-8450-43d2-8dcf-df2cb65ac928" (UID: "5d5502cc-8450-43d2-8dcf-df2cb65ac928"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:51:50.680572 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:50.680488    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d5502cc-8450-43d2-8dcf-df2cb65ac928-model-cache\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:51:50.680658 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:50.680575    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d5502cc-8450-43d2-8dcf-df2cb65ac928-home" (OuterVolumeSpecName: "home") pod "5d5502cc-8450-43d2-8dcf-df2cb65ac928" (UID: "5d5502cc-8450-43d2-8dcf-df2cb65ac928"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:51:50.682365 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:50.682332    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d5502cc-8450-43d2-8dcf-df2cb65ac928-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5d5502cc-8450-43d2-8dcf-df2cb65ac928" (UID: "5d5502cc-8450-43d2-8dcf-df2cb65ac928"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:51:50.682467 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:50.682406    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d5502cc-8450-43d2-8dcf-df2cb65ac928-kube-api-access-w5fsw" (OuterVolumeSpecName: "kube-api-access-w5fsw") pod "5d5502cc-8450-43d2-8dcf-df2cb65ac928" (UID: "5d5502cc-8450-43d2-8dcf-df2cb65ac928"). InnerVolumeSpecName "kube-api-access-w5fsw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:51:50.682467 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:50.682409    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d5502cc-8450-43d2-8dcf-df2cb65ac928-dshm" (OuterVolumeSpecName: "dshm") pod "5d5502cc-8450-43d2-8dcf-df2cb65ac928" (UID: "5d5502cc-8450-43d2-8dcf-df2cb65ac928"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:51:50.781159 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:50.781120    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d5502cc-8450-43d2-8dcf-df2cb65ac928-tmp-dir\") pod \"5d5502cc-8450-43d2-8dcf-df2cb65ac928\" (UID: \"5d5502cc-8450-43d2-8dcf-df2cb65ac928\") "
Jul 22 16:51:50.781339 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:50.781299    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d5502cc-8450-43d2-8dcf-df2cb65ac928-tls-certs\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:51:50.781339 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:50.781310    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d5502cc-8450-43d2-8dcf-df2cb65ac928-home\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:51:50.781339 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:50.781319    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d5502cc-8450-43d2-8dcf-df2cb65ac928-dshm\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:51:50.781339 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:50.781328    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w5fsw\" (UniqueName: \"kubernetes.io/projected/5d5502cc-8450-43d2-8dcf-df2cb65ac928-kube-api-access-w5fsw\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:51:50.781523 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:50.781401    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d5502cc-8450-43d2-8dcf-df2cb65ac928-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5d5502cc-8450-43d2-8dcf-df2cb65ac928" (UID: "5d5502cc-8450-43d2-8dcf-df2cb65ac928"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:51:50.882523 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:50.882431    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d5502cc-8450-43d2-8dcf-df2cb65ac928-tmp-dir\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:51:51.156146 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:51.156045    2563 generic.go:356] "Generic (PLEG): container finished" podID="5d5502cc-8450-43d2-8dcf-df2cb65ac928" containerID="f4b1d45add56548df882955b59ac55d570b17ec6e15d03387b0e64375b99e3cf" exitCode=0
Jul 22 16:51:51.156146 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:51.156111    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf"
Jul 22 16:51:51.156344 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:51.156136    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf" event={"ID":"5d5502cc-8450-43d2-8dcf-df2cb65ac928","Type":"ContainerDied","Data":"f4b1d45add56548df882955b59ac55d570b17ec6e15d03387b0e64375b99e3cf"}
Jul 22 16:51:51.156344 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:51.156182    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf" event={"ID":"5d5502cc-8450-43d2-8dcf-df2cb65ac928","Type":"ContainerDied","Data":"5730707d4d577b3b17dd0018c3cf1d07c73cc342f60172f67c74cc806cbbcc8d"}
Jul 22 16:51:51.156344 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:51.156200    2563 scope.go:117] "RemoveContainer" containerID="f4b1d45add56548df882955b59ac55d570b17ec6e15d03387b0e64375b99e3cf"
Jul 22 16:51:51.165316 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:51.165296    2563 scope.go:117] "RemoveContainer" containerID="f4b1d45add56548df882955b59ac55d570b17ec6e15d03387b0e64375b99e3cf"
Jul 22 16:51:51.165656 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:51:51.165629    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4b1d45add56548df882955b59ac55d570b17ec6e15d03387b0e64375b99e3cf\": container with ID starting with f4b1d45add56548df882955b59ac55d570b17ec6e15d03387b0e64375b99e3cf not found: ID does not exist" containerID="f4b1d45add56548df882955b59ac55d570b17ec6e15d03387b0e64375b99e3cf"
Jul 22 16:51:51.165731 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:51.165672    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4b1d45add56548df882955b59ac55d570b17ec6e15d03387b0e64375b99e3cf"} err="failed to get container status \"f4b1d45add56548df882955b59ac55d570b17ec6e15d03387b0e64375b99e3cf\": rpc error: code = NotFound desc = could not find container \"f4b1d45add56548df882955b59ac55d570b17ec6e15d03387b0e64375b99e3cf\": container with ID starting with f4b1d45add56548df882955b59ac55d570b17ec6e15d03387b0e64375b99e3cf not found: ID does not exist"
Jul 22 16:51:51.178326 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:51.178297    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf"]
Jul 22 16:51:51.185015 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:51.184994    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-ff2a05/auth-test-model-ff2a05-kserve-5b5bc9dc5-gnglf"]
Jul 22 16:51:51.239221 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:51.239185    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c" podUID="3e933847-77e2-461e-9f99-8da31690dbc4" containerName="main" containerID="cri-o://ed5caff59d88b6722efd31444bb24982e8e07b2b55c9ae82cb44c37096f74dd3" gracePeriod=45
Jul 22 16:51:51.489224 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:51.489197    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c"
Jul 22 16:51:51.586530 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:51.586493    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e933847-77e2-461e-9f99-8da31690dbc4-model-cache\") pod \"3e933847-77e2-461e-9f99-8da31690dbc4\" (UID: \"3e933847-77e2-461e-9f99-8da31690dbc4\") "
Jul 22 16:51:51.586704 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:51.586557    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e933847-77e2-461e-9f99-8da31690dbc4-home\") pod \"3e933847-77e2-461e-9f99-8da31690dbc4\" (UID: \"3e933847-77e2-461e-9f99-8da31690dbc4\") "
Jul 22 16:51:51.586704 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:51.586594    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vmdj\" (UniqueName: \"kubernetes.io/projected/3e933847-77e2-461e-9f99-8da31690dbc4-kube-api-access-6vmdj\") pod \"3e933847-77e2-461e-9f99-8da31690dbc4\" (UID: \"3e933847-77e2-461e-9f99-8da31690dbc4\") "
Jul 22 16:51:51.586704 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:51.586641    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e933847-77e2-461e-9f99-8da31690dbc4-tmp-dir\") pod \"3e933847-77e2-461e-9f99-8da31690dbc4\" (UID: \"3e933847-77e2-461e-9f99-8da31690dbc4\") "
Jul 22 16:51:51.586704 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:51.586668    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e933847-77e2-461e-9f99-8da31690dbc4-tls-certs\") pod \"3e933847-77e2-461e-9f99-8da31690dbc4\" (UID: \"3e933847-77e2-461e-9f99-8da31690dbc4\") "
Jul 22 16:51:51.586704 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:51.586687    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e933847-77e2-461e-9f99-8da31690dbc4-dshm\") pod \"3e933847-77e2-461e-9f99-8da31690dbc4\" (UID: \"3e933847-77e2-461e-9f99-8da31690dbc4\") "
Jul 22 16:51:51.587035 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:51.586752    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e933847-77e2-461e-9f99-8da31690dbc4-model-cache" (OuterVolumeSpecName: "model-cache") pod "3e933847-77e2-461e-9f99-8da31690dbc4" (UID: "3e933847-77e2-461e-9f99-8da31690dbc4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:51:51.587035 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:51.586831    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e933847-77e2-461e-9f99-8da31690dbc4-home" (OuterVolumeSpecName: "home") pod "3e933847-77e2-461e-9f99-8da31690dbc4" (UID: "3e933847-77e2-461e-9f99-8da31690dbc4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:51:51.587035 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:51.586889    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e933847-77e2-461e-9f99-8da31690dbc4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3e933847-77e2-461e-9f99-8da31690dbc4" (UID: "3e933847-77e2-461e-9f99-8da31690dbc4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:51:51.587035 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:51.586950    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e933847-77e2-461e-9f99-8da31690dbc4-tmp-dir\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:51:51.587035 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:51.586963    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e933847-77e2-461e-9f99-8da31690dbc4-model-cache\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:51:51.587035 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:51.586971    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e933847-77e2-461e-9f99-8da31690dbc4-home\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:51:51.588897 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:51.588873    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e933847-77e2-461e-9f99-8da31690dbc4-dshm" (OuterVolumeSpecName: "dshm") pod "3e933847-77e2-461e-9f99-8da31690dbc4" (UID: "3e933847-77e2-461e-9f99-8da31690dbc4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:51:51.589022 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:51.588921    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e933847-77e2-461e-9f99-8da31690dbc4-kube-api-access-6vmdj" (OuterVolumeSpecName: "kube-api-access-6vmdj") pod "3e933847-77e2-461e-9f99-8da31690dbc4" (UID: "3e933847-77e2-461e-9f99-8da31690dbc4"). InnerVolumeSpecName "kube-api-access-6vmdj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:51:51.589022 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:51.588925    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e933847-77e2-461e-9f99-8da31690dbc4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3e933847-77e2-461e-9f99-8da31690dbc4" (UID: "3e933847-77e2-461e-9f99-8da31690dbc4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:51:51.687525 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:51.687487    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e933847-77e2-461e-9f99-8da31690dbc4-tls-certs\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:51:51.687525 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:51.687516    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e933847-77e2-461e-9f99-8da31690dbc4-dshm\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:51:51.687525 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:51.687526    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vmdj\" (UniqueName: \"kubernetes.io/projected/3e933847-77e2-461e-9f99-8da31690dbc4-kube-api-access-6vmdj\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:51:52.162364 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:52.162327    2563 generic.go:356] "Generic (PLEG): container finished" podID="3e933847-77e2-461e-9f99-8da31690dbc4" containerID="ed5caff59d88b6722efd31444bb24982e8e07b2b55c9ae82cb44c37096f74dd3" exitCode=0
Jul 22 16:51:52.162549 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:52.162369    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c"
Jul 22 16:51:52.162549 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:52.162415    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c" event={"ID":"3e933847-77e2-461e-9f99-8da31690dbc4","Type":"ContainerDied","Data":"ed5caff59d88b6722efd31444bb24982e8e07b2b55c9ae82cb44c37096f74dd3"}
Jul 22 16:51:52.162549 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:52.162457    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c" event={"ID":"3e933847-77e2-461e-9f99-8da31690dbc4","Type":"ContainerDied","Data":"f1661cbfbcf8d539b2e1fc3071795f2208ab99fc7ec1c400a93c20db869f0834"}
Jul 22 16:51:52.162549 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:52.162478    2563 scope.go:117] "RemoveContainer" containerID="ed5caff59d88b6722efd31444bb24982e8e07b2b55c9ae82cb44c37096f74dd3"
Jul 22 16:51:52.170940 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:52.170921    2563 scope.go:117] "RemoveContainer" containerID="ed5caff59d88b6722efd31444bb24982e8e07b2b55c9ae82cb44c37096f74dd3"
Jul 22 16:51:52.171186 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:51:52.171167    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed5caff59d88b6722efd31444bb24982e8e07b2b55c9ae82cb44c37096f74dd3\": container with ID starting with ed5caff59d88b6722efd31444bb24982e8e07b2b55c9ae82cb44c37096f74dd3 not found: ID does not exist" containerID="ed5caff59d88b6722efd31444bb24982e8e07b2b55c9ae82cb44c37096f74dd3"
Jul 22 16:51:52.171233 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:52.171194    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed5caff59d88b6722efd31444bb24982e8e07b2b55c9ae82cb44c37096f74dd3"} err="failed to get container status \"ed5caff59d88b6722efd31444bb24982e8e07b2b55c9ae82cb44c37096f74dd3\": rpc error: code = NotFound desc = could not find container \"ed5caff59d88b6722efd31444bb24982e8e07b2b55c9ae82cb44c37096f74dd3\": container with ID starting with ed5caff59d88b6722efd31444bb24982e8e07b2b55c9ae82cb44c37096f74dd3 not found: ID does not exist"
Jul 22 16:51:52.183767 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:52.183739    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c"]
Jul 22 16:51:52.186127 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:52.186102    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-afe7a0/auth-test-model-afe7a0-kserve-7d944c99bc-m9m7c"]
Jul 22 16:51:52.449184 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:52.449154    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e933847-77e2-461e-9f99-8da31690dbc4" path="/var/lib/kubelet/pods/3e933847-77e2-461e-9f99-8da31690dbc4/volumes"
Jul 22 16:51:52.449539 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:52.449526    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d5502cc-8450-43d2-8dcf-df2cb65ac928" path="/var/lib/kubelet/pods/5d5502cc-8450-43d2-8dcf-df2cb65ac928/volumes"
Jul 22 16:51:55.438765 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:55.438728    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c"]
Jul 22 16:51:55.439187 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:55.439076    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e933847-77e2-461e-9f99-8da31690dbc4" containerName="main"
Jul 22 16:51:55.439187 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:55.439088    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e933847-77e2-461e-9f99-8da31690dbc4" containerName="main"
Jul 22 16:51:55.439187 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:55.439106    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d5502cc-8450-43d2-8dcf-df2cb65ac928" containerName="main"
Jul 22 16:51:55.439187 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:55.439112    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d5502cc-8450-43d2-8dcf-df2cb65ac928" containerName="main"
Jul 22 16:51:55.439187 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:55.439170    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e933847-77e2-461e-9f99-8da31690dbc4" containerName="main"
Jul 22 16:51:55.439187 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:55.439179    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d5502cc-8450-43d2-8dcf-df2cb65ac928" containerName="main"
Jul 22 16:51:55.443591 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:55.443571    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c"
Jul 22 16:51:55.446622 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:55.446599    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ff2a05\"/\"kube-root-ca.crt\""
Jul 22 16:51:55.448113 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:55.448087    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ff2a05\"/\"sub-test-model-ff2a05-kserve-self-signed-certs\""
Jul 22 16:51:55.448113 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:55.448104    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ff2a05\"/\"openshift-service-ca.crt\""
Jul 22 16:51:55.448290 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:55.448113    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ff2a05\"/\"default-dockercfg-ghqpn\""
Jul 22 16:51:55.455254 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:55.455232    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c"]
Jul 22 16:51:55.467001 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:55.466978    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:51:55.519700 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:55.519661    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-home\") pod \"sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c\" (UID: \"587cd3ba-463b-4c3f-9b5c-d8ef96422dca\") " pod="ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c"
Jul 22 16:51:55.519912 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:55.519703    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-tmp-dir\") pod \"sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c\" (UID: \"587cd3ba-463b-4c3f-9b5c-d8ef96422dca\") " pod="ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c"
Jul 22 16:51:55.519912 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:55.519818    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-dshm\") pod \"sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c\" (UID: \"587cd3ba-463b-4c3f-9b5c-d8ef96422dca\") " pod="ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c"
Jul 22 16:51:55.519912 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:55.519848    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jblcf\" (UniqueName: \"kubernetes.io/projected/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-kube-api-access-jblcf\") pod \"sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c\" (UID: \"587cd3ba-463b-4c3f-9b5c-d8ef96422dca\") " pod="ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c"
Jul 22 16:51:55.520126 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:55.519939    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-tls-certs\") pod \"sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c\" (UID: \"587cd3ba-463b-4c3f-9b5c-d8ef96422dca\") " pod="ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c"
Jul 22 16:51:55.520126 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:55.519980    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-model-cache\") pod \"sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c\" (UID: \"587cd3ba-463b-4c3f-9b5c-d8ef96422dca\") " pod="ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c"
Jul 22 16:51:55.621165 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:55.621121    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-dshm\") pod \"sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c\" (UID: \"587cd3ba-463b-4c3f-9b5c-d8ef96422dca\") " pod="ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c"
Jul 22 16:51:55.621165 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:55.621167    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jblcf\" (UniqueName: \"kubernetes.io/projected/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-kube-api-access-jblcf\") pod \"sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c\" (UID: \"587cd3ba-463b-4c3f-9b5c-d8ef96422dca\") " pod="ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c"
Jul 22 16:51:55.621444 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:55.621205    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-tls-certs\") pod \"sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c\" (UID: \"587cd3ba-463b-4c3f-9b5c-d8ef96422dca\") " pod="ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c"
Jul 22 16:51:55.621444 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:55.621231    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-model-cache\") pod \"sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c\" (UID: \"587cd3ba-463b-4c3f-9b5c-d8ef96422dca\") " pod="ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c"
Jul 22 16:51:55.621444 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:55.621309    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-home\") pod \"sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c\" (UID: \"587cd3ba-463b-4c3f-9b5c-d8ef96422dca\") " pod="ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c"
Jul 22 16:51:55.621444 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:55.621339    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-tmp-dir\") pod \"sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c\" (UID: \"587cd3ba-463b-4c3f-9b5c-d8ef96422dca\") " pod="ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c"
Jul 22 16:51:55.621699 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:55.621675    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-model-cache\") pod \"sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c\" (UID: \"587cd3ba-463b-4c3f-9b5c-d8ef96422dca\") " pod="ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c"
Jul 22 16:51:55.621770 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:55.621717    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-home\") pod \"sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c\" (UID: \"587cd3ba-463b-4c3f-9b5c-d8ef96422dca\") " pod="ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c"
Jul 22 16:51:55.621770 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:55.621741    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-tmp-dir\") pod \"sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c\" (UID: \"587cd3ba-463b-4c3f-9b5c-d8ef96422dca\") " pod="ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c"
Jul 22 16:51:55.623536 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:55.623510    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-dshm\") pod \"sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c\" (UID: \"587cd3ba-463b-4c3f-9b5c-d8ef96422dca\") " pod="ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c"
Jul 22 16:51:55.623639 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:55.623617    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-tls-certs\") pod \"sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c\" (UID: \"587cd3ba-463b-4c3f-9b5c-d8ef96422dca\") " pod="ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c"
Jul 22 16:51:55.630332 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:55.630309    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jblcf\" (UniqueName: \"kubernetes.io/projected/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-kube-api-access-jblcf\") pod \"sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c\" (UID: \"587cd3ba-463b-4c3f-9b5c-d8ef96422dca\") " pod="ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c"
Jul 22 16:51:55.752790 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:55.752685    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c"
Jul 22 16:51:55.888401 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:55.888351    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c"]
Jul 22 16:51:55.895130 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:51:55.895103    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod587cd3ba_463b_4c3f_9b5c_d8ef96422dca.slice/crio-b2db6504fe4403307858cf99532d4cb66672d8a40d9c098fbc2a86d2ed9f9e4f WatchSource:0}: Error finding container b2db6504fe4403307858cf99532d4cb66672d8a40d9c098fbc2a86d2ed9f9e4f: Status 404 returned error can't find the container with id b2db6504fe4403307858cf99532d4cb66672d8a40d9c098fbc2a86d2ed9f9e4f
Jul 22 16:51:56.185771 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:56.185739    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c" event={"ID":"587cd3ba-463b-4c3f-9b5c-d8ef96422dca","Type":"ContainerStarted","Data":"8b378a2483e7bf257940dd2f8c2a610d38522da74ddb531ba3d0856568d114ac"}
Jul 22 16:51:56.185988 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:56.185787    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c" event={"ID":"587cd3ba-463b-4c3f-9b5c-d8ef96422dca","Type":"ContainerStarted","Data":"b2db6504fe4403307858cf99532d4cb66672d8a40d9c098fbc2a86d2ed9f9e4f"}
Jul 22 16:51:56.203503 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:51:56.203436    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c" podStartSLOduration=1.203415357 podStartE2EDuration="1.203415357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:51:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:51:56.20336903 +0000 UTC m=+2676.333839043" watchObservedRunningTime="2026-07-22 16:51:56.203415357 +0000 UTC m=+2676.333885360"
Jul 22 16:52:05.753239 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:52:05.753200    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c"
Jul 22 16:52:05.753644 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:52:05.753250    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c"
Jul 22 16:52:05.765527 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:52:05.765498    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c"
Jul 22 16:52:06.242691 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:52:06.242661    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c"
Jul 22 16:52:19.572486 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:52:19.572401    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:52:51.774359 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:52:51.774321    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:52:51.879913 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:52:51.879883    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:52:53.523402 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:52:53.523370    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:52:59.054283 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:52:59.054248    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c"]
Jul 22 16:52:59.081003 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:52:59.080961    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:52:59.950167 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:52:59.950134    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:53:03.251324 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:03.251290    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf"]
Jul 22 16:53:03.254526 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:03.254507    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf"
Jul 22 16:53:03.257906 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:03.257886    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ff2a05\"/\"rate-test-model-ff2a05-kserve-self-signed-certs\""
Jul 22 16:53:03.274066 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:03.274041    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf"]
Jul 22 16:53:03.313961 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:03.313933    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95af09cb-11ac-47f1-96d1-193ae8f6c750-dshm\") pod \"rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf\" (UID: \"95af09cb-11ac-47f1-96d1-193ae8f6c750\") " pod="ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf"
Jul 22 16:53:03.314123 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:03.313970    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95af09cb-11ac-47f1-96d1-193ae8f6c750-model-cache\") pod \"rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf\" (UID: \"95af09cb-11ac-47f1-96d1-193ae8f6c750\") " pod="ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf"
Jul 22 16:53:03.314123 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:03.313993    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95af09cb-11ac-47f1-96d1-193ae8f6c750-home\") pod \"rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf\" (UID: \"95af09cb-11ac-47f1-96d1-193ae8f6c750\") " pod="ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf"
Jul 22 16:53:03.314123 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:03.314086    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lsqc\" (UniqueName: \"kubernetes.io/projected/95af09cb-11ac-47f1-96d1-193ae8f6c750-kube-api-access-7lsqc\") pod \"rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf\" (UID: \"95af09cb-11ac-47f1-96d1-193ae8f6c750\") " pod="ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf"
Jul 22 16:53:03.314230 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:03.314139    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95af09cb-11ac-47f1-96d1-193ae8f6c750-tmp-dir\") pod \"rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf\" (UID: \"95af09cb-11ac-47f1-96d1-193ae8f6c750\") " pod="ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf"
Jul 22 16:53:03.314230 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:03.314167    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95af09cb-11ac-47f1-96d1-193ae8f6c750-tls-certs\") pod \"rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf\" (UID: \"95af09cb-11ac-47f1-96d1-193ae8f6c750\") " pod="ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf"
Jul 22 16:53:03.353556 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:03.353515    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:53:03.415202 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:03.415164    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95af09cb-11ac-47f1-96d1-193ae8f6c750-tls-certs\") pod \"rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf\" (UID: \"95af09cb-11ac-47f1-96d1-193ae8f6c750\") " pod="ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf"
Jul 22 16:53:03.415407 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:03.415223    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95af09cb-11ac-47f1-96d1-193ae8f6c750-dshm\") pod \"rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf\" (UID: \"95af09cb-11ac-47f1-96d1-193ae8f6c750\") " pod="ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf"
Jul 22 16:53:03.415407 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:03.415349    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95af09cb-11ac-47f1-96d1-193ae8f6c750-model-cache\") pod \"rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf\" (UID: \"95af09cb-11ac-47f1-96d1-193ae8f6c750\") " pod="ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf"
Jul 22 16:53:03.415407 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:03.415399    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95af09cb-11ac-47f1-96d1-193ae8f6c750-home\") pod \"rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf\" (UID: \"95af09cb-11ac-47f1-96d1-193ae8f6c750\") " pod="ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf"
Jul 22 16:53:03.415567 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:03.415474    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lsqc\" (UniqueName: \"kubernetes.io/projected/95af09cb-11ac-47f1-96d1-193ae8f6c750-kube-api-access-7lsqc\") pod \"rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf\" (UID: \"95af09cb-11ac-47f1-96d1-193ae8f6c750\") " pod="ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf"
Jul 22 16:53:03.415692 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:03.415665    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95af09cb-11ac-47f1-96d1-193ae8f6c750-tmp-dir\") pod \"rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf\" (UID: \"95af09cb-11ac-47f1-96d1-193ae8f6c750\") " pod="ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf"
Jul 22 16:53:03.415841 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:03.415819    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95af09cb-11ac-47f1-96d1-193ae8f6c750-model-cache\") pod \"rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf\" (UID: \"95af09cb-11ac-47f1-96d1-193ae8f6c750\") " pod="ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf"
Jul 22 16:53:03.415944 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:03.415839    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95af09cb-11ac-47f1-96d1-193ae8f6c750-home\") pod \"rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf\" (UID: \"95af09cb-11ac-47f1-96d1-193ae8f6c750\") " pod="ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf"
Jul 22 16:53:03.416074 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:03.416041    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95af09cb-11ac-47f1-96d1-193ae8f6c750-tmp-dir\") pod \"rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf\" (UID: \"95af09cb-11ac-47f1-96d1-193ae8f6c750\") " pod="ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf"
Jul 22 16:53:03.417667 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:03.417644    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95af09cb-11ac-47f1-96d1-193ae8f6c750-dshm\") pod \"rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf\" (UID: \"95af09cb-11ac-47f1-96d1-193ae8f6c750\") " pod="ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf"
Jul 22 16:53:03.417843 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:03.417815    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95af09cb-11ac-47f1-96d1-193ae8f6c750-tls-certs\") pod \"rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf\" (UID: \"95af09cb-11ac-47f1-96d1-193ae8f6c750\") " pod="ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf"
Jul 22 16:53:03.436159 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:03.436134    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lsqc\" (UniqueName: \"kubernetes.io/projected/95af09cb-11ac-47f1-96d1-193ae8f6c750-kube-api-access-7lsqc\") pod \"rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf\" (UID: \"95af09cb-11ac-47f1-96d1-193ae8f6c750\") " pod="ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf"
Jul 22 16:53:03.563748 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:03.563661    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf"
Jul 22 16:53:03.712165 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:03.712124    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf"]
Jul 22 16:53:03.717151 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:53:03.717122    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod95af09cb_11ac_47f1_96d1_193ae8f6c750.slice/crio-e8f4ac3fe23e3d0e00032e8f46eb8f333e67f70977fb2bb5e77123f0cdaff5d3 WatchSource:0}: Error finding container e8f4ac3fe23e3d0e00032e8f46eb8f333e67f70977fb2bb5e77123f0cdaff5d3: Status 404 returned error can't find the container with id e8f4ac3fe23e3d0e00032e8f46eb8f333e67f70977fb2bb5e77123f0cdaff5d3
Jul 22 16:53:04.502803 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:04.502764    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf" event={"ID":"95af09cb-11ac-47f1-96d1-193ae8f6c750","Type":"ContainerStarted","Data":"0725165bdc36eb4f4f9640922d8022cf3679f4d4b983f9060267377221139a3d"}
Jul 22 16:53:04.503243 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:04.502815    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf" event={"ID":"95af09cb-11ac-47f1-96d1-193ae8f6c750","Type":"ContainerStarted","Data":"e8f4ac3fe23e3d0e00032e8f46eb8f333e67f70977fb2bb5e77123f0cdaff5d3"}
Jul 22 16:53:04.524116 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:04.524029    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf" podStartSLOduration=1.5240132819999999 podStartE2EDuration="1.524013282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:53:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:53:04.522374923 +0000 UTC m=+2744.652844926" watchObservedRunningTime="2026-07-22 16:53:04.524013282 +0000 UTC m=+2744.654483283"
Jul 22 16:53:13.564013 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:13.563979    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf"
Jul 22 16:53:13.564410 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:13.564041    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf"
Jul 22 16:53:13.576461 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:13.576437    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf"
Jul 22 16:53:14.074435 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:14.074399    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c" podUID="587cd3ba-463b-4c3f-9b5c-d8ef96422dca" containerName="main" containerID="cri-o://8b378a2483e7bf257940dd2f8c2a610d38522da74ddb531ba3d0856568d114ac" gracePeriod=45
Jul 22 16:53:14.326618 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:14.326551    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c"
Jul 22 16:53:14.411470 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:14.411429    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-home\") pod \"587cd3ba-463b-4c3f-9b5c-d8ef96422dca\" (UID: \"587cd3ba-463b-4c3f-9b5c-d8ef96422dca\") "
Jul 22 16:53:14.411470 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:14.411471    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jblcf\" (UniqueName: \"kubernetes.io/projected/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-kube-api-access-jblcf\") pod \"587cd3ba-463b-4c3f-9b5c-d8ef96422dca\" (UID: \"587cd3ba-463b-4c3f-9b5c-d8ef96422dca\") "
Jul 22 16:53:14.411742 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:14.411493    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-dshm\") pod \"587cd3ba-463b-4c3f-9b5c-d8ef96422dca\" (UID: \"587cd3ba-463b-4c3f-9b5c-d8ef96422dca\") "
Jul 22 16:53:14.411742 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:14.411519    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-tls-certs\") pod \"587cd3ba-463b-4c3f-9b5c-d8ef96422dca\" (UID: \"587cd3ba-463b-4c3f-9b5c-d8ef96422dca\") "
Jul 22 16:53:14.411742 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:14.411571    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-model-cache\") pod \"587cd3ba-463b-4c3f-9b5c-d8ef96422dca\" (UID: \"587cd3ba-463b-4c3f-9b5c-d8ef96422dca\") "
Jul 22 16:53:14.411742 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:14.411593    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-tmp-dir\") pod \"587cd3ba-463b-4c3f-9b5c-d8ef96422dca\" (UID: \"587cd3ba-463b-4c3f-9b5c-d8ef96422dca\") "
Jul 22 16:53:14.411977 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:14.411792    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-home" (OuterVolumeSpecName: "home") pod "587cd3ba-463b-4c3f-9b5c-d8ef96422dca" (UID: "587cd3ba-463b-4c3f-9b5c-d8ef96422dca"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:53:14.411977 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:14.411840    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-model-cache" (OuterVolumeSpecName: "model-cache") pod "587cd3ba-463b-4c3f-9b5c-d8ef96422dca" (UID: "587cd3ba-463b-4c3f-9b5c-d8ef96422dca"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:53:14.411977 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:14.411958    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "587cd3ba-463b-4c3f-9b5c-d8ef96422dca" (UID: "587cd3ba-463b-4c3f-9b5c-d8ef96422dca"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:53:14.412107 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:14.412020    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-home\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:53:14.412107 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:14.412039    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-model-cache\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:53:14.413766 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:14.413740    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "587cd3ba-463b-4c3f-9b5c-d8ef96422dca" (UID: "587cd3ba-463b-4c3f-9b5c-d8ef96422dca"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:53:14.413766 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:14.413753    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-dshm" (OuterVolumeSpecName: "dshm") pod "587cd3ba-463b-4c3f-9b5c-d8ef96422dca" (UID: "587cd3ba-463b-4c3f-9b5c-d8ef96422dca"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:53:14.413949 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:14.413752    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-kube-api-access-jblcf" (OuterVolumeSpecName: "kube-api-access-jblcf") pod "587cd3ba-463b-4c3f-9b5c-d8ef96422dca" (UID: "587cd3ba-463b-4c3f-9b5c-d8ef96422dca"). InnerVolumeSpecName "kube-api-access-jblcf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:53:14.512795 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:14.512764    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jblcf\" (UniqueName: \"kubernetes.io/projected/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-kube-api-access-jblcf\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:53:14.512795 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:14.512794    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-dshm\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:53:14.513022 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:14.512806    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-tls-certs\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:53:14.513022 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:14.512819    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/587cd3ba-463b-4c3f-9b5c-d8ef96422dca-tmp-dir\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:53:14.547624 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:14.547587    2563 generic.go:356] "Generic (PLEG): container finished" podID="587cd3ba-463b-4c3f-9b5c-d8ef96422dca" containerID="8b378a2483e7bf257940dd2f8c2a610d38522da74ddb531ba3d0856568d114ac" exitCode=0
Jul 22 16:53:14.547769 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:14.547635    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c"
Jul 22 16:53:14.547769 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:14.547660    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c" event={"ID":"587cd3ba-463b-4c3f-9b5c-d8ef96422dca","Type":"ContainerDied","Data":"8b378a2483e7bf257940dd2f8c2a610d38522da74ddb531ba3d0856568d114ac"}
Jul 22 16:53:14.547769 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:14.547705    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c" event={"ID":"587cd3ba-463b-4c3f-9b5c-d8ef96422dca","Type":"ContainerDied","Data":"b2db6504fe4403307858cf99532d4cb66672d8a40d9c098fbc2a86d2ed9f9e4f"}
Jul 22 16:53:14.547769 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:14.547723    2563 scope.go:117] "RemoveContainer" containerID="8b378a2483e7bf257940dd2f8c2a610d38522da74ddb531ba3d0856568d114ac"
Jul 22 16:53:14.556310 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:14.556291    2563 scope.go:117] "RemoveContainer" containerID="8b378a2483e7bf257940dd2f8c2a610d38522da74ddb531ba3d0856568d114ac"
Jul 22 16:53:14.556588 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:53:14.556570    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b378a2483e7bf257940dd2f8c2a610d38522da74ddb531ba3d0856568d114ac\": container with ID starting with 8b378a2483e7bf257940dd2f8c2a610d38522da74ddb531ba3d0856568d114ac not found: ID does not exist" containerID="8b378a2483e7bf257940dd2f8c2a610d38522da74ddb531ba3d0856568d114ac"
Jul 22 16:53:14.556639 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:14.556596    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b378a2483e7bf257940dd2f8c2a610d38522da74ddb531ba3d0856568d114ac"} err="failed to get container status \"8b378a2483e7bf257940dd2f8c2a610d38522da74ddb531ba3d0856568d114ac\": rpc error: code = NotFound desc = could not find container \"8b378a2483e7bf257940dd2f8c2a610d38522da74ddb531ba3d0856568d114ac\": container with ID starting with 8b378a2483e7bf257940dd2f8c2a610d38522da74ddb531ba3d0856568d114ac not found: ID does not exist"
Jul 22 16:53:14.560033 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:14.560015    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf"
Jul 22 16:53:14.567051 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:14.567028    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c"]
Jul 22 16:53:14.572019 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:14.571988    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-ff2a05/sub-test-model-ff2a05-kserve-5dcb886c5f-4qb6c"]
Jul 22 16:53:16.447540 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:16.447506    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="587cd3ba-463b-4c3f-9b5c-d8ef96422dca" path="/var/lib/kubelet/pods/587cd3ba-463b-4c3f-9b5c-d8ef96422dca/volumes"
Jul 22 16:53:31.801919 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:31.801875    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj"]
Jul 22 16:53:31.802460 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:31.802215    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="587cd3ba-463b-4c3f-9b5c-d8ef96422dca" containerName="main"
Jul 22 16:53:31.802460 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:31.802227    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="587cd3ba-463b-4c3f-9b5c-d8ef96422dca" containerName="main"
Jul 22 16:53:31.802460 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:31.802292    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="587cd3ba-463b-4c3f-9b5c-d8ef96422dca" containerName="main"
Jul 22 16:53:31.805133 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:31.805105    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj"
Jul 22 16:53:31.812248 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:31.812221    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-afe7a0\"/\"default-dockercfg-zlhgh\""
Jul 22 16:53:31.813471 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:31.813447    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-afe7a0\"/\"rate-test-model-afe7a0-kserve-self-signed-certs\""
Jul 22 16:53:31.813585 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:31.813467    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-afe7a0\"/\"openshift-service-ca.crt\""
Jul 22 16:53:31.813776 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:31.813759    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-afe7a0\"/\"kube-root-ca.crt\""
Jul 22 16:53:31.822070 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:31.822048    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj"]
Jul 22 16:53:31.890465 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:31.890436    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:53:31.961732 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:31.961696    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07579647-d5b2-4608-8259-52aeb33dcd8c-tmp-dir\") pod \"rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj\" (UID: \"07579647-d5b2-4608-8259-52aeb33dcd8c\") " pod="ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj"
Jul 22 16:53:31.961966 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:31.961743    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07579647-d5b2-4608-8259-52aeb33dcd8c-home\") pod \"rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj\" (UID: \"07579647-d5b2-4608-8259-52aeb33dcd8c\") " pod="ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj"
Jul 22 16:53:31.961966 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:31.961821    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07579647-d5b2-4608-8259-52aeb33dcd8c-tls-certs\") pod \"rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj\" (UID: \"07579647-d5b2-4608-8259-52aeb33dcd8c\") " pod="ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj"
Jul 22 16:53:31.961966 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:31.961929    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07579647-d5b2-4608-8259-52aeb33dcd8c-model-cache\") pod \"rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj\" (UID: \"07579647-d5b2-4608-8259-52aeb33dcd8c\") " pod="ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj"
Jul 22 16:53:31.962137 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:31.961975    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhtkt\" (UniqueName: \"kubernetes.io/projected/07579647-d5b2-4608-8259-52aeb33dcd8c-kube-api-access-vhtkt\") pod \"rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj\" (UID: \"07579647-d5b2-4608-8259-52aeb33dcd8c\") " pod="ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj"
Jul 22 16:53:31.962137 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:31.962015    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07579647-d5b2-4608-8259-52aeb33dcd8c-dshm\") pod \"rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj\" (UID: \"07579647-d5b2-4608-8259-52aeb33dcd8c\") " pod="ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj"
Jul 22 16:53:32.062927 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:32.062826    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07579647-d5b2-4608-8259-52aeb33dcd8c-dshm\") pod \"rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj\" (UID: \"07579647-d5b2-4608-8259-52aeb33dcd8c\") " pod="ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj"
Jul 22 16:53:32.063087 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:32.062946    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07579647-d5b2-4608-8259-52aeb33dcd8c-tmp-dir\") pod \"rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj\" (UID: \"07579647-d5b2-4608-8259-52aeb33dcd8c\") " pod="ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj"
Jul 22 16:53:32.063087 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:32.062995    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07579647-d5b2-4608-8259-52aeb33dcd8c-home\") pod \"rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj\" (UID: \"07579647-d5b2-4608-8259-52aeb33dcd8c\") " pod="ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj"
Jul 22 16:53:32.063087 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:32.063044    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07579647-d5b2-4608-8259-52aeb33dcd8c-tls-certs\") pod \"rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj\" (UID: \"07579647-d5b2-4608-8259-52aeb33dcd8c\") " pod="ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj"
Jul 22 16:53:32.063256 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:32.063091    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07579647-d5b2-4608-8259-52aeb33dcd8c-model-cache\") pod \"rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj\" (UID: \"07579647-d5b2-4608-8259-52aeb33dcd8c\") " pod="ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj"
Jul 22 16:53:32.063256 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:32.063126    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhtkt\" (UniqueName: \"kubernetes.io/projected/07579647-d5b2-4608-8259-52aeb33dcd8c-kube-api-access-vhtkt\") pod \"rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj\" (UID: \"07579647-d5b2-4608-8259-52aeb33dcd8c\") " pod="ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj"
Jul 22 16:53:32.063400 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:32.063374    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07579647-d5b2-4608-8259-52aeb33dcd8c-tmp-dir\") pod \"rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj\" (UID: \"07579647-d5b2-4608-8259-52aeb33dcd8c\") " pod="ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj"
Jul 22 16:53:32.063468 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:32.063423    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07579647-d5b2-4608-8259-52aeb33dcd8c-home\") pod \"rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj\" (UID: \"07579647-d5b2-4608-8259-52aeb33dcd8c\") " pod="ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj"
Jul 22 16:53:32.063511 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:32.063459    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07579647-d5b2-4608-8259-52aeb33dcd8c-model-cache\") pod \"rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj\" (UID: \"07579647-d5b2-4608-8259-52aeb33dcd8c\") " pod="ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj"
Jul 22 16:53:32.065273 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:32.065250    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07579647-d5b2-4608-8259-52aeb33dcd8c-dshm\") pod \"rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj\" (UID: \"07579647-d5b2-4608-8259-52aeb33dcd8c\") " pod="ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj"
Jul 22 16:53:32.065521 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:32.065504    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07579647-d5b2-4608-8259-52aeb33dcd8c-tls-certs\") pod \"rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj\" (UID: \"07579647-d5b2-4608-8259-52aeb33dcd8c\") " pod="ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj"
Jul 22 16:53:32.077758 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:32.077731    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhtkt\" (UniqueName: \"kubernetes.io/projected/07579647-d5b2-4608-8259-52aeb33dcd8c-kube-api-access-vhtkt\") pod \"rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj\" (UID: \"07579647-d5b2-4608-8259-52aeb33dcd8c\") " pod="ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj"
Jul 22 16:53:32.116080 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:32.116042    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj"
Jul 22 16:53:32.250315 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:32.250284    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj"]
Jul 22 16:53:32.253720 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:53:32.253693    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07579647_d5b2_4608_8259_52aeb33dcd8c.slice/crio-0af6192501d5c2dba9e586a4647fa7094958bf4922a723a2ae7ace6e57054397 WatchSource:0}: Error finding container 0af6192501d5c2dba9e586a4647fa7094958bf4922a723a2ae7ace6e57054397: Status 404 returned error can't find the container with id 0af6192501d5c2dba9e586a4647fa7094958bf4922a723a2ae7ace6e57054397
Jul 22 16:53:32.628845 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:32.628808    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj" event={"ID":"07579647-d5b2-4608-8259-52aeb33dcd8c","Type":"ContainerStarted","Data":"d812e5696a5fff89ef7ddb9a82e9c236a7e1e825764313ffd7208f9c49ed1d63"}
Jul 22 16:53:32.628845 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:32.628865    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj" event={"ID":"07579647-d5b2-4608-8259-52aeb33dcd8c","Type":"ContainerStarted","Data":"0af6192501d5c2dba9e586a4647fa7094958bf4922a723a2ae7ace6e57054397"}
Jul 22 16:53:32.662585 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:32.662531    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj" podStartSLOduration=1.662512919 podStartE2EDuration="1.662512919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:53:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:53:32.661540857 +0000 UTC m=+2772.792010869" watchObservedRunningTime="2026-07-22 16:53:32.662512919 +0000 UTC m=+2772.792982920"
Jul 22 16:53:42.117037 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:42.116948    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj"
Jul 22 16:53:42.117037 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:42.117008    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj"
Jul 22 16:53:42.130476 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:42.130445    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj"
Jul 22 16:53:42.602517 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:42.602489    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:53:42.685066 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:42.685031    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj"
Jul 22 16:53:42.710169 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:42.710127    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:53:49.828543 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:49.828509    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:53:49.929101 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:53:49.929068    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:54:17.341540 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:17.341500    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf"]
Jul 22 16:54:17.372423 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:17.372379    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:54:18.392741 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:18.392707    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj"]
Jul 22 16:54:18.447674 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:18.447631    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:54:32.361863 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:32.361806    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf" podUID="95af09cb-11ac-47f1-96d1-193ae8f6c750" containerName="main" containerID="cri-o://0725165bdc36eb4f4f9640922d8022cf3679f4d4b983f9060267377221139a3d" gracePeriod=45
Jul 22 16:54:32.608918 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:32.608891    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf"
Jul 22 16:54:32.803655 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:32.803622    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95af09cb-11ac-47f1-96d1-193ae8f6c750-tls-certs\") pod \"95af09cb-11ac-47f1-96d1-193ae8f6c750\" (UID: \"95af09cb-11ac-47f1-96d1-193ae8f6c750\") "
Jul 22 16:54:32.803655 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:32.803668    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95af09cb-11ac-47f1-96d1-193ae8f6c750-dshm\") pod \"95af09cb-11ac-47f1-96d1-193ae8f6c750\" (UID: \"95af09cb-11ac-47f1-96d1-193ae8f6c750\") "
Jul 22 16:54:32.803971 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:32.803715    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95af09cb-11ac-47f1-96d1-193ae8f6c750-tmp-dir\") pod \"95af09cb-11ac-47f1-96d1-193ae8f6c750\" (UID: \"95af09cb-11ac-47f1-96d1-193ae8f6c750\") "
Jul 22 16:54:32.803971 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:32.803734    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95af09cb-11ac-47f1-96d1-193ae8f6c750-model-cache\") pod \"95af09cb-11ac-47f1-96d1-193ae8f6c750\" (UID: \"95af09cb-11ac-47f1-96d1-193ae8f6c750\") "
Jul 22 16:54:32.803971 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:32.803764    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7lsqc\" (UniqueName: \"kubernetes.io/projected/95af09cb-11ac-47f1-96d1-193ae8f6c750-kube-api-access-7lsqc\") pod \"95af09cb-11ac-47f1-96d1-193ae8f6c750\" (UID: \"95af09cb-11ac-47f1-96d1-193ae8f6c750\") "
Jul 22 16:54:32.803971 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:32.803826    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95af09cb-11ac-47f1-96d1-193ae8f6c750-home\") pod \"95af09cb-11ac-47f1-96d1-193ae8f6c750\" (UID: \"95af09cb-11ac-47f1-96d1-193ae8f6c750\") "
Jul 22 16:54:32.804259 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:32.804064    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95af09cb-11ac-47f1-96d1-193ae8f6c750-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "95af09cb-11ac-47f1-96d1-193ae8f6c750" (UID: "95af09cb-11ac-47f1-96d1-193ae8f6c750"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:54:32.804259 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:32.804077    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95af09cb-11ac-47f1-96d1-193ae8f6c750-model-cache" (OuterVolumeSpecName: "model-cache") pod "95af09cb-11ac-47f1-96d1-193ae8f6c750" (UID: "95af09cb-11ac-47f1-96d1-193ae8f6c750"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:54:32.804259 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:32.804145    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95af09cb-11ac-47f1-96d1-193ae8f6c750-home" (OuterVolumeSpecName: "home") pod "95af09cb-11ac-47f1-96d1-193ae8f6c750" (UID: "95af09cb-11ac-47f1-96d1-193ae8f6c750"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:54:32.806004 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:32.805977    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95af09cb-11ac-47f1-96d1-193ae8f6c750-dshm" (OuterVolumeSpecName: "dshm") pod "95af09cb-11ac-47f1-96d1-193ae8f6c750" (UID: "95af09cb-11ac-47f1-96d1-193ae8f6c750"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:54:32.806098 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:32.806046    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95af09cb-11ac-47f1-96d1-193ae8f6c750-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "95af09cb-11ac-47f1-96d1-193ae8f6c750" (UID: "95af09cb-11ac-47f1-96d1-193ae8f6c750"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:54:32.806098 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:32.806057    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95af09cb-11ac-47f1-96d1-193ae8f6c750-kube-api-access-7lsqc" (OuterVolumeSpecName: "kube-api-access-7lsqc") pod "95af09cb-11ac-47f1-96d1-193ae8f6c750" (UID: "95af09cb-11ac-47f1-96d1-193ae8f6c750"). InnerVolumeSpecName "kube-api-access-7lsqc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:54:32.899251 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:32.899220    2563 generic.go:356] "Generic (PLEG): container finished" podID="95af09cb-11ac-47f1-96d1-193ae8f6c750" containerID="0725165bdc36eb4f4f9640922d8022cf3679f4d4b983f9060267377221139a3d" exitCode=0
Jul 22 16:54:32.899413 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:32.899265    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf"
Jul 22 16:54:32.899413 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:32.899266    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf" event={"ID":"95af09cb-11ac-47f1-96d1-193ae8f6c750","Type":"ContainerDied","Data":"0725165bdc36eb4f4f9640922d8022cf3679f4d4b983f9060267377221139a3d"}
Jul 22 16:54:32.899413 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:32.899387    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf" event={"ID":"95af09cb-11ac-47f1-96d1-193ae8f6c750","Type":"ContainerDied","Data":"e8f4ac3fe23e3d0e00032e8f46eb8f333e67f70977fb2bb5e77123f0cdaff5d3"}
Jul 22 16:54:32.899522 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:32.899418    2563 scope.go:117] "RemoveContainer" containerID="0725165bdc36eb4f4f9640922d8022cf3679f4d4b983f9060267377221139a3d"
Jul 22 16:54:32.904737 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:32.904712    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lsqc\" (UniqueName: \"kubernetes.io/projected/95af09cb-11ac-47f1-96d1-193ae8f6c750-kube-api-access-7lsqc\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:54:32.904894 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:32.904739    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95af09cb-11ac-47f1-96d1-193ae8f6c750-home\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:54:32.904894 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:32.904755    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95af09cb-11ac-47f1-96d1-193ae8f6c750-tls-certs\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:54:32.904894 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:32.904767    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95af09cb-11ac-47f1-96d1-193ae8f6c750-dshm\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:54:32.904894 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:32.904777    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95af09cb-11ac-47f1-96d1-193ae8f6c750-tmp-dir\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:54:32.904894 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:32.904787    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95af09cb-11ac-47f1-96d1-193ae8f6c750-model-cache\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:54:32.908332 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:32.908313    2563 scope.go:117] "RemoveContainer" containerID="0725165bdc36eb4f4f9640922d8022cf3679f4d4b983f9060267377221139a3d"
Jul 22 16:54:32.908609 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:54:32.908590    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0725165bdc36eb4f4f9640922d8022cf3679f4d4b983f9060267377221139a3d\": container with ID starting with 0725165bdc36eb4f4f9640922d8022cf3679f4d4b983f9060267377221139a3d not found: ID does not exist" containerID="0725165bdc36eb4f4f9640922d8022cf3679f4d4b983f9060267377221139a3d"
Jul 22 16:54:32.908655 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:32.908619    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0725165bdc36eb4f4f9640922d8022cf3679f4d4b983f9060267377221139a3d"} err="failed to get container status \"0725165bdc36eb4f4f9640922d8022cf3679f4d4b983f9060267377221139a3d\": rpc error: code = NotFound desc = could not find container \"0725165bdc36eb4f4f9640922d8022cf3679f4d4b983f9060267377221139a3d\": container with ID starting with 0725165bdc36eb4f4f9640922d8022cf3679f4d4b983f9060267377221139a3d not found: ID does not exist"
Jul 22 16:54:32.932201 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:32.932164    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf"]
Jul 22 16:54:32.936804 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:32.936771    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-ff2a05/rate-test-model-ff2a05-kserve-567f4cbb67-x9jzf"]
Jul 22 16:54:33.412656 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:33.412617    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj" podUID="07579647-d5b2-4608-8259-52aeb33dcd8c" containerName="main" containerID="cri-o://d812e5696a5fff89ef7ddb9a82e9c236a7e1e825764313ffd7208f9c49ed1d63" gracePeriod=45
Jul 22 16:54:33.664209 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:33.664145    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj"
Jul 22 16:54:33.712040 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:33.712005    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07579647-d5b2-4608-8259-52aeb33dcd8c-home\") pod \"07579647-d5b2-4608-8259-52aeb33dcd8c\" (UID: \"07579647-d5b2-4608-8259-52aeb33dcd8c\") "
Jul 22 16:54:33.712040 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:33.712045    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07579647-d5b2-4608-8259-52aeb33dcd8c-tls-certs\") pod \"07579647-d5b2-4608-8259-52aeb33dcd8c\" (UID: \"07579647-d5b2-4608-8259-52aeb33dcd8c\") "
Jul 22 16:54:33.712310 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:33.712096    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07579647-d5b2-4608-8259-52aeb33dcd8c-tmp-dir\") pod \"07579647-d5b2-4608-8259-52aeb33dcd8c\" (UID: \"07579647-d5b2-4608-8259-52aeb33dcd8c\") "
Jul 22 16:54:33.712310 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:33.712117    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07579647-d5b2-4608-8259-52aeb33dcd8c-model-cache\") pod \"07579647-d5b2-4608-8259-52aeb33dcd8c\" (UID: \"07579647-d5b2-4608-8259-52aeb33dcd8c\") "
Jul 22 16:54:33.712310 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:33.712133    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhtkt\" (UniqueName: \"kubernetes.io/projected/07579647-d5b2-4608-8259-52aeb33dcd8c-kube-api-access-vhtkt\") pod \"07579647-d5b2-4608-8259-52aeb33dcd8c\" (UID: \"07579647-d5b2-4608-8259-52aeb33dcd8c\") "
Jul 22 16:54:33.712310 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:33.712193    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07579647-d5b2-4608-8259-52aeb33dcd8c-dshm\") pod \"07579647-d5b2-4608-8259-52aeb33dcd8c\" (UID: \"07579647-d5b2-4608-8259-52aeb33dcd8c\") "
Jul 22 16:54:33.712526 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:33.712328    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07579647-d5b2-4608-8259-52aeb33dcd8c-home" (OuterVolumeSpecName: "home") pod "07579647-d5b2-4608-8259-52aeb33dcd8c" (UID: "07579647-d5b2-4608-8259-52aeb33dcd8c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:54:33.712526 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:33.712503    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07579647-d5b2-4608-8259-52aeb33dcd8c-home\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:54:33.712606 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:33.712574    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07579647-d5b2-4608-8259-52aeb33dcd8c-model-cache" (OuterVolumeSpecName: "model-cache") pod "07579647-d5b2-4608-8259-52aeb33dcd8c" (UID: "07579647-d5b2-4608-8259-52aeb33dcd8c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:54:33.712779 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:33.712748    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07579647-d5b2-4608-8259-52aeb33dcd8c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "07579647-d5b2-4608-8259-52aeb33dcd8c" (UID: "07579647-d5b2-4608-8259-52aeb33dcd8c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:54:33.714397 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:33.714369    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07579647-d5b2-4608-8259-52aeb33dcd8c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "07579647-d5b2-4608-8259-52aeb33dcd8c" (UID: "07579647-d5b2-4608-8259-52aeb33dcd8c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:54:33.714529 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:33.714436    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07579647-d5b2-4608-8259-52aeb33dcd8c-dshm" (OuterVolumeSpecName: "dshm") pod "07579647-d5b2-4608-8259-52aeb33dcd8c" (UID: "07579647-d5b2-4608-8259-52aeb33dcd8c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:54:33.714529 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:33.714512    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07579647-d5b2-4608-8259-52aeb33dcd8c-kube-api-access-vhtkt" (OuterVolumeSpecName: "kube-api-access-vhtkt") pod "07579647-d5b2-4608-8259-52aeb33dcd8c" (UID: "07579647-d5b2-4608-8259-52aeb33dcd8c"). InnerVolumeSpecName "kube-api-access-vhtkt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:54:33.813449 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:33.813418    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07579647-d5b2-4608-8259-52aeb33dcd8c-dshm\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:54:33.813449 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:33.813447    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07579647-d5b2-4608-8259-52aeb33dcd8c-tls-certs\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:54:33.813449 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:33.813457    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07579647-d5b2-4608-8259-52aeb33dcd8c-tmp-dir\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:54:33.813682 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:33.813469    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07579647-d5b2-4608-8259-52aeb33dcd8c-model-cache\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:54:33.813682 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:33.813477    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhtkt\" (UniqueName: \"kubernetes.io/projected/07579647-d5b2-4608-8259-52aeb33dcd8c-kube-api-access-vhtkt\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:54:33.905993 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:33.905957    2563 generic.go:356] "Generic (PLEG): container finished" podID="07579647-d5b2-4608-8259-52aeb33dcd8c" containerID="d812e5696a5fff89ef7ddb9a82e9c236a7e1e825764313ffd7208f9c49ed1d63" exitCode=0
Jul 22 16:54:33.906158 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:33.905999    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj"
Jul 22 16:54:33.906158 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:33.906026    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj" event={"ID":"07579647-d5b2-4608-8259-52aeb33dcd8c","Type":"ContainerDied","Data":"d812e5696a5fff89ef7ddb9a82e9c236a7e1e825764313ffd7208f9c49ed1d63"}
Jul 22 16:54:33.906158 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:33.906061    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj" event={"ID":"07579647-d5b2-4608-8259-52aeb33dcd8c","Type":"ContainerDied","Data":"0af6192501d5c2dba9e586a4647fa7094958bf4922a723a2ae7ace6e57054397"}
Jul 22 16:54:33.906158 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:33.906079    2563 scope.go:117] "RemoveContainer" containerID="d812e5696a5fff89ef7ddb9a82e9c236a7e1e825764313ffd7208f9c49ed1d63"
Jul 22 16:54:33.918731 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:33.918712    2563 scope.go:117] "RemoveContainer" containerID="d812e5696a5fff89ef7ddb9a82e9c236a7e1e825764313ffd7208f9c49ed1d63"
Jul 22 16:54:33.919041 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:54:33.919021    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d812e5696a5fff89ef7ddb9a82e9c236a7e1e825764313ffd7208f9c49ed1d63\": container with ID starting with d812e5696a5fff89ef7ddb9a82e9c236a7e1e825764313ffd7208f9c49ed1d63 not found: ID does not exist" containerID="d812e5696a5fff89ef7ddb9a82e9c236a7e1e825764313ffd7208f9c49ed1d63"
Jul 22 16:54:33.919117 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:33.919053    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d812e5696a5fff89ef7ddb9a82e9c236a7e1e825764313ffd7208f9c49ed1d63"} err="failed to get container status \"d812e5696a5fff89ef7ddb9a82e9c236a7e1e825764313ffd7208f9c49ed1d63\": rpc error: code = NotFound desc = could not find container \"d812e5696a5fff89ef7ddb9a82e9c236a7e1e825764313ffd7208f9c49ed1d63\": container with ID starting with d812e5696a5fff89ef7ddb9a82e9c236a7e1e825764313ffd7208f9c49ed1d63 not found: ID does not exist"
Jul 22 16:54:33.930727 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:33.930695    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj"]
Jul 22 16:54:33.936669 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:33.936643    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-afe7a0/rate-test-model-afe7a0-kserve-5f94687bf9-zp6sj"]
Jul 22 16:54:34.389721 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:34.389682    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7694-140398118664208"]
Jul 22 16:54:34.390074 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:34.390061    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95af09cb-11ac-47f1-96d1-193ae8f6c750" containerName="main"
Jul 22 16:54:34.390129 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:34.390075    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="95af09cb-11ac-47f1-96d1-193ae8f6c750" containerName="main"
Jul 22 16:54:34.390129 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:34.390096    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07579647-d5b2-4608-8259-52aeb33dcd8c" containerName="main"
Jul 22 16:54:34.390129 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:34.390101    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="07579647-d5b2-4608-8259-52aeb33dcd8c" containerName="main"
Jul 22 16:54:34.390228 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:34.390147    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="95af09cb-11ac-47f1-96d1-193ae8f6c750" containerName="main"
Jul 22 16:54:34.390228 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:34.390159    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="07579647-d5b2-4608-8259-52aeb33dcd8c" containerName="main"
Jul 22 16:54:34.394933 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:34.394911    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-140398118664208"
Jul 22 16:54:34.399766 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:34.399741    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tjvgd\""
Jul 22 16:54:34.408769 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:34.408733    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7694-140398118664208"]
Jul 22 16:54:34.416743 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:34.416707    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mc2j\" (UniqueName: \"kubernetes.io/projected/22c2cabb-c680-4e17-8cfe-0a07f05fecb1-kube-api-access-8mc2j\") pod \"test-curl-7694-140398118664208\" (UID: \"22c2cabb-c680-4e17-8cfe-0a07f05fecb1\") " pod="opendatahub/test-curl-7694-140398118664208"
Jul 22 16:54:34.448674 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:34.448640    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07579647-d5b2-4608-8259-52aeb33dcd8c" path="/var/lib/kubelet/pods/07579647-d5b2-4608-8259-52aeb33dcd8c/volumes"
Jul 22 16:54:34.449056 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:34.449043    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95af09cb-11ac-47f1-96d1-193ae8f6c750" path="/var/lib/kubelet/pods/95af09cb-11ac-47f1-96d1-193ae8f6c750/volumes"
Jul 22 16:54:34.517498 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:34.517462    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mc2j\" (UniqueName: \"kubernetes.io/projected/22c2cabb-c680-4e17-8cfe-0a07f05fecb1-kube-api-access-8mc2j\") pod \"test-curl-7694-140398118664208\" (UID: \"22c2cabb-c680-4e17-8cfe-0a07f05fecb1\") " pod="opendatahub/test-curl-7694-140398118664208"
Jul 22 16:54:34.527568 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:34.527535    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mc2j\" (UniqueName: \"kubernetes.io/projected/22c2cabb-c680-4e17-8cfe-0a07f05fecb1-kube-api-access-8mc2j\") pod \"test-curl-7694-140398118664208\" (UID: \"22c2cabb-c680-4e17-8cfe-0a07f05fecb1\") " pod="opendatahub/test-curl-7694-140398118664208"
Jul 22 16:54:34.705739 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:34.705705    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-140398118664208"
Jul 22 16:54:34.841141 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:34.841103    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7694-140398118664208"]
Jul 22 16:54:34.844010 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:54:34.843985    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod22c2cabb_c680_4e17_8cfe_0a07f05fecb1.slice/crio-6fb7011994d6528534289f8e23974d0b396a242ecd2513c62beac7ef65f10217 WatchSource:0}: Error finding container 6fb7011994d6528534289f8e23974d0b396a242ecd2513c62beac7ef65f10217: Status 404 returned error can't find the container with id 6fb7011994d6528534289f8e23974d0b396a242ecd2513c62beac7ef65f10217
Jul 22 16:54:34.845632 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:34.845607    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 16:54:34.911901 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:34.911842    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7694-140398118664208" event={"ID":"22c2cabb-c680-4e17-8cfe-0a07f05fecb1","Type":"ContainerStarted","Data":"6fb7011994d6528534289f8e23974d0b396a242ecd2513c62beac7ef65f10217"}
Jul 22 16:54:35.919204 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:35.919096    2563 generic.go:356] "Generic (PLEG): container finished" podID="22c2cabb-c680-4e17-8cfe-0a07f05fecb1" containerID="30c028c54c0267338297eadddd2f7405c49f07f70960ec94b8851b0648517371" exitCode=0
Jul 22 16:54:35.919204 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:35.919192    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7694-140398118664208" event={"ID":"22c2cabb-c680-4e17-8cfe-0a07f05fecb1","Type":"ContainerDied","Data":"30c028c54c0267338297eadddd2f7405c49f07f70960ec94b8851b0648517371"}
Jul 22 16:54:37.055556 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:37.055532    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-140398118664208"
Jul 22 16:54:37.078349 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:37.078320    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7694-140398118664208_22c2cabb-c680-4e17-8cfe-0a07f05fecb1/test-curl-7694-140398118664208/0.log"
Jul 22 16:54:37.104916 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:37.104879    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7694-140398118664208"]
Jul 22 16:54:37.113474 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:37.113440    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7694-140398118664208"]
Jul 22 16:54:37.137461 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:37.137434    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mc2j\" (UniqueName: \"kubernetes.io/projected/22c2cabb-c680-4e17-8cfe-0a07f05fecb1-kube-api-access-8mc2j\") pod \"22c2cabb-c680-4e17-8cfe-0a07f05fecb1\" (UID: \"22c2cabb-c680-4e17-8cfe-0a07f05fecb1\") "
Jul 22 16:54:37.139511 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:37.139487    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22c2cabb-c680-4e17-8cfe-0a07f05fecb1-kube-api-access-8mc2j" (OuterVolumeSpecName: "kube-api-access-8mc2j") pod "22c2cabb-c680-4e17-8cfe-0a07f05fecb1" (UID: "22c2cabb-c680-4e17-8cfe-0a07f05fecb1"). InnerVolumeSpecName "kube-api-access-8mc2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:54:37.238571 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:37.238532    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mc2j\" (UniqueName: \"kubernetes.io/projected/22c2cabb-c680-4e17-8cfe-0a07f05fecb1-kube-api-access-8mc2j\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:54:37.930299 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:37.930263    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-140398118664208"
Jul 22 16:54:37.930299 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:37.930295    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6fb7011994d6528534289f8e23974d0b396a242ecd2513c62beac7ef65f10217"
Jul 22 16:54:37.940299 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:37.940273    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:54:37.940727 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:54:37.940697    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7694-140398118664208\" is forbidden: User \"system:node:ip-10-0-142-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-224.ec2.internal' and this object" podUID="22c2cabb-c680-4e17-8cfe-0a07f05fecb1" pod="opendatahub/test-curl-7694-140398118664208"
Jul 22 16:54:38.037947 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:38.037914    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:54:38.448019 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:38.447985    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22c2cabb-c680-4e17-8cfe-0a07f05fecb1" path="/var/lib/kubelet/pods/22c2cabb-c680-4e17-8cfe-0a07f05fecb1/volumes"
Jul 22 16:54:47.780049 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:47.780006    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc"]
Jul 22 16:54:47.780518 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:47.780287    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc" podUID="7a187da1-a115-4464-9e71-f49a19c1e112" containerName="maas-api" containerID="cri-o://ec0d446043992b926b677c5e3644f279af96a68b8f6514190af185254b8f516a" gracePeriod=30
Jul 22 16:54:47.806033 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:47.806005    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:54:47.979543 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:47.979512    2563 generic.go:356] "Generic (PLEG): container finished" podID="7a187da1-a115-4464-9e71-f49a19c1e112" containerID="ec0d446043992b926b677c5e3644f279af96a68b8f6514190af185254b8f516a" exitCode=0
Jul 22 16:54:47.979703 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:47.979584    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc" event={"ID":"7a187da1-a115-4464-9e71-f49a19c1e112","Type":"ContainerDied","Data":"ec0d446043992b926b677c5e3644f279af96a68b8f6514190af185254b8f516a"}
Jul 22 16:54:48.030846 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:48.030783    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc"
Jul 22 16:54:48.135418 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:48.135376    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a187da1-a115-4464-9e71-f49a19c1e112-openshift-service-ca\") pod \"7a187da1-a115-4464-9e71-f49a19c1e112\" (UID: \"7a187da1-a115-4464-9e71-f49a19c1e112\") "
Jul 22 16:54:48.135596 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:48.135433    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a187da1-a115-4464-9e71-f49a19c1e112-maas-api-tls\") pod \"7a187da1-a115-4464-9e71-f49a19c1e112\" (UID: \"7a187da1-a115-4464-9e71-f49a19c1e112\") "
Jul 22 16:54:48.135659 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:48.135638    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ljdp4\" (UniqueName: \"kubernetes.io/projected/7a187da1-a115-4464-9e71-f49a19c1e112-kube-api-access-ljdp4\") pod \"7a187da1-a115-4464-9e71-f49a19c1e112\" (UID: \"7a187da1-a115-4464-9e71-f49a19c1e112\") "
Jul 22 16:54:48.142929 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:48.135741    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a187da1-a115-4464-9e71-f49a19c1e112-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7a187da1-a115-4464-9e71-f49a19c1e112" (UID: "7a187da1-a115-4464-9e71-f49a19c1e112"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 16:54:48.142929 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:48.136391    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a187da1-a115-4464-9e71-f49a19c1e112-openshift-service-ca\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:54:48.143480 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:48.143308    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a187da1-a115-4464-9e71-f49a19c1e112-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7a187da1-a115-4464-9e71-f49a19c1e112" (UID: "7a187da1-a115-4464-9e71-f49a19c1e112"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:54:48.143834 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:48.143808    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a187da1-a115-4464-9e71-f49a19c1e112-kube-api-access-ljdp4" (OuterVolumeSpecName: "kube-api-access-ljdp4") pod "7a187da1-a115-4464-9e71-f49a19c1e112" (UID: "7a187da1-a115-4464-9e71-f49a19c1e112"). InnerVolumeSpecName "kube-api-access-ljdp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:54:48.237786 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:48.237750    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ljdp4\" (UniqueName: \"kubernetes.io/projected/7a187da1-a115-4464-9e71-f49a19c1e112-kube-api-access-ljdp4\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:54:48.237786 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:48.237782    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a187da1-a115-4464-9e71-f49a19c1e112-maas-api-tls\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:54:48.984891 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:48.984830    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc" event={"ID":"7a187da1-a115-4464-9e71-f49a19c1e112","Type":"ContainerDied","Data":"85c85ef14a4b99ca051b0f8405170a36a74046b0b6a272881ec2a5faac02c8fd"}
Jul 22 16:54:48.984891 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:48.984871    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc"
Jul 22 16:54:48.985406 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:48.984922    2563 scope.go:117] "RemoveContainer" containerID="ec0d446043992b926b677c5e3644f279af96a68b8f6514190af185254b8f516a"
Jul 22 16:54:49.004530 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:49.004502    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc"]
Jul 22 16:54:49.010211 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:49.010183    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ff2a05-69d469546c-v6kqc"]
Jul 22 16:54:50.448296 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:50.448265    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a187da1-a115-4464-9e71-f49a19c1e112" path="/var/lib/kubelet/pods/7a187da1-a115-4464-9e71-f49a19c1e112/volumes"
Jul 22 16:54:52.374256 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:52.374226    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:54:52.783100 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:52.783065    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:54:59.008024 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:59.007988    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:54:59.083108 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:59.083073    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:54:59.350512 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:59.350415    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"]
Jul 22 16:54:59.350672 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:59.350646    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj" podUID="482e44f8-dec4-4797-89f0-6804ec7b3738" containerName="istio-proxy" containerID="cri-o://130f21c8f31412f47461f094c9e5362e2831871cea9f8aec2f07fbce30aba019" gracePeriod=30
Jul 22 16:54:59.355138 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:59.355109    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:54:59.561141 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:54:59.561102    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:55:00.194080 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:00.194048    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:55:00.298900 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:00.298844    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:55:04.586885 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:04.586845    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:55:04.679483 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:04.679452    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/482e44f8-dec4-4797-89f0-6804ec7b3738-workload-certs\") pod \"482e44f8-dec4-4797-89f0-6804ec7b3738\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") "
Jul 22 16:55:04.679673 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:04.679500    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/482e44f8-dec4-4797-89f0-6804ec7b3738-istiod-ca-cert\") pod \"482e44f8-dec4-4797-89f0-6804ec7b3738\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") "
Jul 22 16:55:04.679673 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:04.679551    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/482e44f8-dec4-4797-89f0-6804ec7b3738-istio-token\") pod \"482e44f8-dec4-4797-89f0-6804ec7b3738\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") "
Jul 22 16:55:04.679673 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:04.679586    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/482e44f8-dec4-4797-89f0-6804ec7b3738-credential-socket\") pod \"482e44f8-dec4-4797-89f0-6804ec7b3738\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") "
Jul 22 16:55:04.679673 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:04.679619    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/482e44f8-dec4-4797-89f0-6804ec7b3738-workload-socket\") pod \"482e44f8-dec4-4797-89f0-6804ec7b3738\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") "
Jul 22 16:55:04.679673 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:04.679643    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/482e44f8-dec4-4797-89f0-6804ec7b3738-istio-podinfo\") pod \"482e44f8-dec4-4797-89f0-6804ec7b3738\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") "
Jul 22 16:55:04.679673 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:04.679668    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/482e44f8-dec4-4797-89f0-6804ec7b3738-istio-data\") pod \"482e44f8-dec4-4797-89f0-6804ec7b3738\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") "
Jul 22 16:55:04.680020 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:04.679694    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zs6hd\" (UniqueName: \"kubernetes.io/projected/482e44f8-dec4-4797-89f0-6804ec7b3738-kube-api-access-zs6hd\") pod \"482e44f8-dec4-4797-89f0-6804ec7b3738\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") "
Jul 22 16:55:04.680020 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:04.679759    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/482e44f8-dec4-4797-89f0-6804ec7b3738-istio-envoy\") pod \"482e44f8-dec4-4797-89f0-6804ec7b3738\" (UID: \"482e44f8-dec4-4797-89f0-6804ec7b3738\") "
Jul 22 16:55:04.680020 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:04.679968    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/482e44f8-dec4-4797-89f0-6804ec7b3738-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "482e44f8-dec4-4797-89f0-6804ec7b3738" (UID: "482e44f8-dec4-4797-89f0-6804ec7b3738"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:55:04.680178 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:04.680071    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/482e44f8-dec4-4797-89f0-6804ec7b3738-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "482e44f8-dec4-4797-89f0-6804ec7b3738" (UID: "482e44f8-dec4-4797-89f0-6804ec7b3738"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 16:55:04.680178 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:04.680144    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/482e44f8-dec4-4797-89f0-6804ec7b3738-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "482e44f8-dec4-4797-89f0-6804ec7b3738" (UID: "482e44f8-dec4-4797-89f0-6804ec7b3738"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:55:04.680296 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:04.680179    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/482e44f8-dec4-4797-89f0-6804ec7b3738-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "482e44f8-dec4-4797-89f0-6804ec7b3738" (UID: "482e44f8-dec4-4797-89f0-6804ec7b3738"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:55:04.680679 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:04.680652    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/482e44f8-dec4-4797-89f0-6804ec7b3738-istio-data" (OuterVolumeSpecName: "istio-data") pod "482e44f8-dec4-4797-89f0-6804ec7b3738" (UID: "482e44f8-dec4-4797-89f0-6804ec7b3738"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:55:04.681921 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:04.681892    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/482e44f8-dec4-4797-89f0-6804ec7b3738-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "482e44f8-dec4-4797-89f0-6804ec7b3738" (UID: "482e44f8-dec4-4797-89f0-6804ec7b3738"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 16:55:04.682056 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:04.682038    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/482e44f8-dec4-4797-89f0-6804ec7b3738-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "482e44f8-dec4-4797-89f0-6804ec7b3738" (UID: "482e44f8-dec4-4797-89f0-6804ec7b3738"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 16:55:04.682297 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:04.682273    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/482e44f8-dec4-4797-89f0-6804ec7b3738-kube-api-access-zs6hd" (OuterVolumeSpecName: "kube-api-access-zs6hd") pod "482e44f8-dec4-4797-89f0-6804ec7b3738" (UID: "482e44f8-dec4-4797-89f0-6804ec7b3738"). InnerVolumeSpecName "kube-api-access-zs6hd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:55:04.682358 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:04.682330    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/482e44f8-dec4-4797-89f0-6804ec7b3738-istio-token" (OuterVolumeSpecName: "istio-token") pod "482e44f8-dec4-4797-89f0-6804ec7b3738" (UID: "482e44f8-dec4-4797-89f0-6804ec7b3738"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:55:04.781353 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:04.781313    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/482e44f8-dec4-4797-89f0-6804ec7b3738-workload-certs\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:55:04.781353 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:04.781348    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/482e44f8-dec4-4797-89f0-6804ec7b3738-istiod-ca-cert\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:55:04.781353 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:04.781358    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/482e44f8-dec4-4797-89f0-6804ec7b3738-istio-token\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:55:04.781353 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:04.781368    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/482e44f8-dec4-4797-89f0-6804ec7b3738-credential-socket\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:55:04.781657 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:04.781377    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/482e44f8-dec4-4797-89f0-6804ec7b3738-workload-socket\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:55:04.781657 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:04.781385    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/482e44f8-dec4-4797-89f0-6804ec7b3738-istio-podinfo\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:55:04.781657 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:04.781393    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/482e44f8-dec4-4797-89f0-6804ec7b3738-istio-data\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:55:04.781657 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:04.781402    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zs6hd\" (UniqueName: \"kubernetes.io/projected/482e44f8-dec4-4797-89f0-6804ec7b3738-kube-api-access-zs6hd\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:55:04.781657 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:04.781409    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/482e44f8-dec4-4797-89f0-6804ec7b3738-istio-envoy\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:55:05.056741 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:05.056653    2563 generic.go:356] "Generic (PLEG): container finished" podID="482e44f8-dec4-4797-89f0-6804ec7b3738" containerID="130f21c8f31412f47461f094c9e5362e2831871cea9f8aec2f07fbce30aba019" exitCode=0
Jul 22 16:55:05.056741 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:05.056708    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"
Jul 22 16:55:05.056939 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:05.056752    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj" event={"ID":"482e44f8-dec4-4797-89f0-6804ec7b3738","Type":"ContainerDied","Data":"130f21c8f31412f47461f094c9e5362e2831871cea9f8aec2f07fbce30aba019"}
Jul 22 16:55:05.056939 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:05.056812    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj" event={"ID":"482e44f8-dec4-4797-89f0-6804ec7b3738","Type":"ContainerDied","Data":"273bdc1e8d5b9c02f16d77c62ec8da1ce3957e4bf29ddecef78710490fa22c03"}
Jul 22 16:55:05.056939 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:05.056836    2563 scope.go:117] "RemoveContainer" containerID="130f21c8f31412f47461f094c9e5362e2831871cea9f8aec2f07fbce30aba019"
Jul 22 16:55:05.066036 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:05.066016    2563 scope.go:117] "RemoveContainer" containerID="791533f07cf02379fa7501a826b1b2088de091ecb01c0d7b01d58da84184cd7f"
Jul 22 16:55:05.074072 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:05.074051    2563 scope.go:117] "RemoveContainer" containerID="130f21c8f31412f47461f094c9e5362e2831871cea9f8aec2f07fbce30aba019"
Jul 22 16:55:05.074377 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:55:05.074359    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"130f21c8f31412f47461f094c9e5362e2831871cea9f8aec2f07fbce30aba019\": container with ID starting with 130f21c8f31412f47461f094c9e5362e2831871cea9f8aec2f07fbce30aba019 not found: ID does not exist" containerID="130f21c8f31412f47461f094c9e5362e2831871cea9f8aec2f07fbce30aba019"
Jul 22 16:55:05.074423 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:05.074388    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"130f21c8f31412f47461f094c9e5362e2831871cea9f8aec2f07fbce30aba019"} err="failed to get container status \"130f21c8f31412f47461f094c9e5362e2831871cea9f8aec2f07fbce30aba019\": rpc error: code = NotFound desc = could not find container \"130f21c8f31412f47461f094c9e5362e2831871cea9f8aec2f07fbce30aba019\": container with ID starting with 130f21c8f31412f47461f094c9e5362e2831871cea9f8aec2f07fbce30aba019 not found: ID does not exist"
Jul 22 16:55:05.074423 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:05.074409    2563 scope.go:117] "RemoveContainer" containerID="791533f07cf02379fa7501a826b1b2088de091ecb01c0d7b01d58da84184cd7f"
Jul 22 16:55:05.074673 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:55:05.074653    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"791533f07cf02379fa7501a826b1b2088de091ecb01c0d7b01d58da84184cd7f\": container with ID starting with 791533f07cf02379fa7501a826b1b2088de091ecb01c0d7b01d58da84184cd7f not found: ID does not exist" containerID="791533f07cf02379fa7501a826b1b2088de091ecb01c0d7b01d58da84184cd7f"
Jul 22 16:55:05.074729 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:05.074679    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"791533f07cf02379fa7501a826b1b2088de091ecb01c0d7b01d58da84184cd7f"} err="failed to get container status \"791533f07cf02379fa7501a826b1b2088de091ecb01c0d7b01d58da84184cd7f\": rpc error: code = NotFound desc = could not find container \"791533f07cf02379fa7501a826b1b2088de091ecb01c0d7b01d58da84184cd7f\": container with ID starting with 791533f07cf02379fa7501a826b1b2088de091ecb01c0d7b01d58da84184cd7f not found: ID does not exist"
Jul 22 16:55:05.079291 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:05.079266    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"]
Jul 22 16:55:05.083557 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:05.083536    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-ff2a05-openshift-default-8d4cf6867-7swsj"]
Jul 22 16:55:06.447787 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:06.447751    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="482e44f8-dec4-4797-89f0-6804ec7b3738" path="/var/lib/kubelet/pods/482e44f8-dec4-4797-89f0-6804ec7b3738/volumes"
Jul 22 16:55:10.111183 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:10.111143    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr"]
Jul 22 16:55:10.111667 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:10.111343    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr" podUID="7c251985-c1ec-4c42-b249-518d75a4870a" containerName="maas-api" containerID="cri-o://bea3905a2a003d6597522f51df1790993faee84f0dccdd16d018e3db19ca92fe" gracePeriod=30
Jul 22 16:55:10.133845 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:10.133816    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:55:10.355462 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:10.355439    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr"
Jul 22 16:55:10.428305 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:10.428276    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c251985-c1ec-4c42-b249-518d75a4870a-openshift-service-ca\") pod \"7c251985-c1ec-4c42-b249-518d75a4870a\" (UID: \"7c251985-c1ec-4c42-b249-518d75a4870a\") "
Jul 22 16:55:10.428470 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:10.428387    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c251985-c1ec-4c42-b249-518d75a4870a-maas-api-tls\") pod \"7c251985-c1ec-4c42-b249-518d75a4870a\" (UID: \"7c251985-c1ec-4c42-b249-518d75a4870a\") "
Jul 22 16:55:10.428470 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:10.428416    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kzx2f\" (UniqueName: \"kubernetes.io/projected/7c251985-c1ec-4c42-b249-518d75a4870a-kube-api-access-kzx2f\") pod \"7c251985-c1ec-4c42-b249-518d75a4870a\" (UID: \"7c251985-c1ec-4c42-b249-518d75a4870a\") "
Jul 22 16:55:10.428698 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:10.428677    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c251985-c1ec-4c42-b249-518d75a4870a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7c251985-c1ec-4c42-b249-518d75a4870a" (UID: "7c251985-c1ec-4c42-b249-518d75a4870a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 16:55:10.430553 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:10.430524    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c251985-c1ec-4c42-b249-518d75a4870a-kube-api-access-kzx2f" (OuterVolumeSpecName: "kube-api-access-kzx2f") pod "7c251985-c1ec-4c42-b249-518d75a4870a" (UID: "7c251985-c1ec-4c42-b249-518d75a4870a"). InnerVolumeSpecName "kube-api-access-kzx2f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 16:55:10.430677 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:10.430573    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c251985-c1ec-4c42-b249-518d75a4870a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7c251985-c1ec-4c42-b249-518d75a4870a" (UID: "7c251985-c1ec-4c42-b249-518d75a4870a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 16:55:10.529104 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:10.529069    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c251985-c1ec-4c42-b249-518d75a4870a-maas-api-tls\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:55:10.529104 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:10.529098    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzx2f\" (UniqueName: \"kubernetes.io/projected/7c251985-c1ec-4c42-b249-518d75a4870a-kube-api-access-kzx2f\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:55:10.529104 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:10.529110    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c251985-c1ec-4c42-b249-518d75a4870a-openshift-service-ca\") on node \"ip-10-0-142-224.ec2.internal\" DevicePath \"\""
Jul 22 16:55:11.086767 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:11.086732    2563 generic.go:356] "Generic (PLEG): container finished" podID="7c251985-c1ec-4c42-b249-518d75a4870a" containerID="bea3905a2a003d6597522f51df1790993faee84f0dccdd16d018e3db19ca92fe" exitCode=0
Jul 22 16:55:11.086994 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:11.086784    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr"
Jul 22 16:55:11.086994 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:11.086785    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr" event={"ID":"7c251985-c1ec-4c42-b249-518d75a4870a","Type":"ContainerDied","Data":"bea3905a2a003d6597522f51df1790993faee84f0dccdd16d018e3db19ca92fe"}
Jul 22 16:55:11.086994 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:11.086914    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr" event={"ID":"7c251985-c1ec-4c42-b249-518d75a4870a","Type":"ContainerDied","Data":"bcc1c6c4da79130290823d799d445f31964844e694e7aed56357be076dd42c81"}
Jul 22 16:55:11.086994 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:11.086945    2563 scope.go:117] "RemoveContainer" containerID="bea3905a2a003d6597522f51df1790993faee84f0dccdd16d018e3db19ca92fe"
Jul 22 16:55:11.095169 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:11.095151    2563 scope.go:117] "RemoveContainer" containerID="bea3905a2a003d6597522f51df1790993faee84f0dccdd16d018e3db19ca92fe"
Jul 22 16:55:11.095460 ip-10-0-142-224 kubenswrapper[2563]: E0722 16:55:11.095441    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bea3905a2a003d6597522f51df1790993faee84f0dccdd16d018e3db19ca92fe\": container with ID starting with bea3905a2a003d6597522f51df1790993faee84f0dccdd16d018e3db19ca92fe not found: ID does not exist" containerID="bea3905a2a003d6597522f51df1790993faee84f0dccdd16d018e3db19ca92fe"
Jul 22 16:55:11.095511 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:11.095469    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bea3905a2a003d6597522f51df1790993faee84f0dccdd16d018e3db19ca92fe"} err="failed to get container status \"bea3905a2a003d6597522f51df1790993faee84f0dccdd16d018e3db19ca92fe\": rpc error: code = NotFound desc = could not find container \"bea3905a2a003d6597522f51df1790993faee84f0dccdd16d018e3db19ca92fe\": container with ID starting with bea3905a2a003d6597522f51df1790993faee84f0dccdd16d018e3db19ca92fe not found: ID does not exist"
Jul 22 16:55:11.107446 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:11.107412    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr"]
Jul 22 16:55:11.110004 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:11.109978    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-afe7a0-8449b9d584-8sftr"]
Jul 22 16:55:12.448804 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:12.448771    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c251985-c1ec-4c42-b249-518d75a4870a" path="/var/lib/kubelet/pods/7c251985-c1ec-4c42-b249-518d75a4870a/volumes"
Jul 22 16:55:14.755056 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:14.755021    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:55:15.157095 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:15.157015    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:55:21.148105 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:21.148007    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:55:21.249013 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:21.248972    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:55:21.410290 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:21.410198    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:55:21.654842 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:21.654809    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:55:27.667470 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:27.667439    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6848f74968-64kv4_9d14f63a-49aa-44a4-b1a2-eece5bb92883/manager/0.log"
Jul 22 16:55:28.025861 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:28.025832    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-587bf8b54-52jlh_15b90659-90dd-4fd8-aaf5-1e97d6aad202/copy-manifests/0.log"
Jul 22 16:55:28.033139 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:28.033108    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-587bf8b54-52jlh_15b90659-90dd-4fd8-aaf5-1e97d6aad202/manager/0.log"
Jul 22 16:55:28.767301 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:28.767273    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29745630-v5cfc_9179d684-6a21-4aaa-9f9f-d8a3f55360c2/cleanup/0.log"
Jul 22 16:55:31.170373 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:31.170344    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-87f9g_c0a0cd66-4510-442e-8717-b9be93deafac/manager/0.log"
Jul 22 16:55:31.900758 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:31.900727    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9b87bf6fb-g6x9x_79491bd3-166f-4502-b60d-ec1eb758d07a/kube-auth-proxy/0.log"
Jul 22 16:55:32.989147 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:32.989099    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-9d4rx_d76b9940-884e-43c6-b562-7829ace2b94c/main/0.log"
Jul 22 16:55:33.111527 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:33.111497    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6zvf5_f690408c-96a3-4268-9596-4a0b12d05ac1/main/0.log"
Jul 22 16:55:38.596137 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:38.596106    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bzfkk/must-gather-dnqvl"]
Jul 22 16:55:38.596558 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:38.596407    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="482e44f8-dec4-4797-89f0-6804ec7b3738" containerName="istio-proxy"
Jul 22 16:55:38.596558 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:38.596418    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="482e44f8-dec4-4797-89f0-6804ec7b3738" containerName="istio-proxy"
Jul 22 16:55:38.596558 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:38.596434    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="482e44f8-dec4-4797-89f0-6804ec7b3738" containerName="istio-proxy"
Jul 22 16:55:38.596558 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:38.596439    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="482e44f8-dec4-4797-89f0-6804ec7b3738" containerName="istio-proxy"
Jul 22 16:55:38.596558 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:38.596446    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22c2cabb-c680-4e17-8cfe-0a07f05fecb1" containerName="test-curl-7694-140398118664208"
Jul 22 16:55:38.596558 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:38.596452    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="22c2cabb-c680-4e17-8cfe-0a07f05fecb1" containerName="test-curl-7694-140398118664208"
Jul 22 16:55:38.596558 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:38.596461    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c251985-c1ec-4c42-b249-518d75a4870a" containerName="maas-api"
Jul 22 16:55:38.596558 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:38.596466    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c251985-c1ec-4c42-b249-518d75a4870a" containerName="maas-api"
Jul 22 16:55:38.596558 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:38.596480    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a187da1-a115-4464-9e71-f49a19c1e112" containerName="maas-api"
Jul 22 16:55:38.596558 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:38.596485    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a187da1-a115-4464-9e71-f49a19c1e112" containerName="maas-api"
Jul 22 16:55:38.596558 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:38.596531    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="482e44f8-dec4-4797-89f0-6804ec7b3738" containerName="istio-proxy"
Jul 22 16:55:38.596558 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:38.596537    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a187da1-a115-4464-9e71-f49a19c1e112" containerName="maas-api"
Jul 22 16:55:38.596558 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:38.596546    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="22c2cabb-c680-4e17-8cfe-0a07f05fecb1" containerName="test-curl-7694-140398118664208"
Jul 22 16:55:38.596558 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:38.596555    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c251985-c1ec-4c42-b249-518d75a4870a" containerName="maas-api"
Jul 22 16:55:38.597073 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:38.596643    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="482e44f8-dec4-4797-89f0-6804ec7b3738" containerName="istio-proxy"
Jul 22 16:55:38.599716 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:38.599697    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bzfkk/must-gather-dnqvl"
Jul 22 16:55:38.603161 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:38.603136    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bzfkk\"/\"default-dockercfg-s9whn\""
Jul 22 16:55:38.603161 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:38.603145    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bzfkk\"/\"kube-root-ca.crt\""
Jul 22 16:55:38.603343 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:38.603234    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bzfkk\"/\"openshift-service-ca.crt\""
Jul 22 16:55:38.616625 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:38.616602    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bzfkk/must-gather-dnqvl"]
Jul 22 16:55:38.623977 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:38.623954    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:55:38.658747 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:38.658712    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8c08059d-ec72-47d6-ac29-56ac3e27acec-must-gather-output\") pod \"must-gather-dnqvl\" (UID: \"8c08059d-ec72-47d6-ac29-56ac3e27acec\") " pod="openshift-must-gather-bzfkk/must-gather-dnqvl"
Jul 22 16:55:38.658966 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:38.658813    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8krbt\" (UniqueName: \"kubernetes.io/projected/8c08059d-ec72-47d6-ac29-56ac3e27acec-kube-api-access-8krbt\") pod \"must-gather-dnqvl\" (UID: \"8c08059d-ec72-47d6-ac29-56ac3e27acec\") " pod="openshift-must-gather-bzfkk/must-gather-dnqvl"
Jul 22 16:55:38.725123 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:38.725091    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:55:38.760147 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:38.760116    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8c08059d-ec72-47d6-ac29-56ac3e27acec-must-gather-output\") pod \"must-gather-dnqvl\" (UID: \"8c08059d-ec72-47d6-ac29-56ac3e27acec\") " pod="openshift-must-gather-bzfkk/must-gather-dnqvl"
Jul 22 16:55:38.760319 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:38.760260    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8krbt\" (UniqueName: \"kubernetes.io/projected/8c08059d-ec72-47d6-ac29-56ac3e27acec-kube-api-access-8krbt\") pod \"must-gather-dnqvl\" (UID: \"8c08059d-ec72-47d6-ac29-56ac3e27acec\") " pod="openshift-must-gather-bzfkk/must-gather-dnqvl"
Jul 22 16:55:38.760470 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:38.760450    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8c08059d-ec72-47d6-ac29-56ac3e27acec-must-gather-output\") pod \"must-gather-dnqvl\" (UID: \"8c08059d-ec72-47d6-ac29-56ac3e27acec\") " pod="openshift-must-gather-bzfkk/must-gather-dnqvl"
Jul 22 16:55:38.772675 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:38.772650    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8krbt\" (UniqueName: \"kubernetes.io/projected/8c08059d-ec72-47d6-ac29-56ac3e27acec-kube-api-access-8krbt\") pod \"must-gather-dnqvl\" (UID: \"8c08059d-ec72-47d6-ac29-56ac3e27acec\") " pod="openshift-must-gather-bzfkk/must-gather-dnqvl"
Jul 22 16:55:38.909548 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:38.909466    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bzfkk/must-gather-dnqvl"
Jul 22 16:55:39.043242 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:39.043214    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bzfkk/must-gather-dnqvl"]
Jul 22 16:55:39.045448 ip-10-0-142-224 kubenswrapper[2563]: W0722 16:55:39.045423    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8c08059d_ec72_47d6_ac29_56ac3e27acec.slice/crio-570e81ef1e5a4b6dcda5ac73111650cc87a6ec324b24d9cdfa3d8538d8c7c18d WatchSource:0}: Error finding container 570e81ef1e5a4b6dcda5ac73111650cc87a6ec324b24d9cdfa3d8538d8c7c18d: Status 404 returned error can't find the container with id 570e81ef1e5a4b6dcda5ac73111650cc87a6ec324b24d9cdfa3d8538d8c7c18d
Jul 22 16:55:39.218408 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:39.218376    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bzfkk/must-gather-dnqvl" event={"ID":"8c08059d-ec72-47d6-ac29-56ac3e27acec","Type":"ContainerStarted","Data":"570e81ef1e5a4b6dcda5ac73111650cc87a6ec324b24d9cdfa3d8538d8c7c18d"}
Jul 22 16:55:40.227909 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:40.227837    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bzfkk/must-gather-dnqvl" event={"ID":"8c08059d-ec72-47d6-ac29-56ac3e27acec","Type":"ContainerStarted","Data":"c04646c840c74fdba39aa856a9fbeb84ff9b7593fcd90fb8f2901b179b35c3fe"}
Jul 22 16:55:40.228426 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:40.227930    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bzfkk/must-gather-dnqvl" event={"ID":"8c08059d-ec72-47d6-ac29-56ac3e27acec","Type":"ContainerStarted","Data":"84a295c26d4e376345976f64d85999ebf5145453fcc64e61227835c1d57e91de"}
Jul 22 16:55:41.543674 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:41.543640    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-5dhg6_1cd85e6d-9e7c-4c21-bc75-98cc25975bfc/global-pull-secret-syncer/0.log"
Jul 22 16:55:41.691505 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:41.691457    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log"
Jul 22 16:55:41.808163 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:41.808074    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-224.ec2.internal_7b422f2f9cb0c4f90f53104ef61aedd3/haproxy/0.log"
Jul 22 16:55:42.342929 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:42.342896    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2b2_b6d03e93-ac5d-4b1a-87a2-2c3b2358dcac/konnectivity-agent/0.log" limit=1024
Jul 22 16:55:47.499280 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:47.499222    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-87f9g_c0a0cd66-4510-442e-8717-b9be93deafac/manager/0.log"
Jul 22 16:55:49.618218 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:49.618186    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-dsckk_62ca114b-81f3-470f-8f20-65212dfc953e/monitoring-plugin/0.log"
Jul 22 16:55:49.839826 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:49.839769    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h6kg9_c7280975-6841-4d25-82f2-18702e8cf8ef/node-exporter/0.log"
Jul 22 16:55:49.906727 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:49.906642    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h6kg9_c7280975-6841-4d25-82f2-18702e8cf8ef/kube-rbac-proxy/0.log"
Jul 22 16:55:49.936013 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:49.935972    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h6kg9_c7280975-6841-4d25-82f2-18702e8cf8ef/init-textfile/0.log"
Jul 22 16:55:49.967984 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:49.967940    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-pcfcd_a1d3f28a-7b56-4185-b5af-6bad89156abc/kube-rbac-proxy-main/0.log"
Jul 22 16:55:49.992409 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:49.992379    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-pcfcd_a1d3f28a-7b56-4185-b5af-6bad89156abc/kube-rbac-proxy-self/0.log"
Jul 22 16:55:50.014643 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:50.014612    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-pcfcd_a1d3f28a-7b56-4185-b5af-6bad89156abc/openshift-state-metrics/0.log"
Jul 22 16:55:50.059013 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:50.058983    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c800f5b3-a21b-4128-baf5-09b8819c1a16/prometheus/0.log"
Jul 22 16:55:50.081485 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:50.081454    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c800f5b3-a21b-4128-baf5-09b8819c1a16/config-reloader/0.log"
Jul 22 16:55:50.105386 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:50.105354    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c800f5b3-a21b-4128-baf5-09b8819c1a16/thanos-sidecar/0.log"
Jul 22 16:55:50.126129 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:50.126072    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c800f5b3-a21b-4128-baf5-09b8819c1a16/kube-rbac-proxy-web/0.log"
Jul 22 16:55:50.152501 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:50.152443    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c800f5b3-a21b-4128-baf5-09b8819c1a16/kube-rbac-proxy/0.log"
Jul 22 16:55:50.175043 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:50.174946    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c800f5b3-a21b-4128-baf5-09b8819c1a16/kube-rbac-proxy-thanos/0.log"
Jul 22 16:55:50.197773 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:50.197730    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c800f5b3-a21b-4128-baf5-09b8819c1a16/init-config-reloader/0.log"
Jul 22 16:55:51.957821 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:51.957759    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bzfkk/must-gather-dnqvl" podStartSLOduration=13.110788716 podStartE2EDuration="13.957740358s" totalImagesPullingTime="846.951642ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:55:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:55:40.245906363 +0000 UTC m=+2900.376376365" watchObservedRunningTime="2026-07-22 16:55:51.957740358 +0000 UTC m=+2912.088210359"
Jul 22 16:55:51.958663 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:51.958640    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bzfkk/perf-node-gather-daemonset-nppzg"]
Jul 22 16:55:51.964735 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:51.964701    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-nppzg"
Jul 22 16:55:51.975666 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:51.975626    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bzfkk/perf-node-gather-daemonset-nppzg"]
Jul 22 16:55:52.092777 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:52.092732    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn5lg\" (UniqueName: \"kubernetes.io/projected/bd36563a-fb46-4bb1-878d-c50e9cfcee62-kube-api-access-xn5lg\") pod \"perf-node-gather-daemonset-nppzg\" (UID: \"bd36563a-fb46-4bb1-878d-c50e9cfcee62\") " pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-nppzg"
Jul 22 16:55:52.092995 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:52.092815    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bd36563a-fb46-4bb1-878d-c50e9cfcee62-sys\") pod \"perf-node-gather-daemonset-nppzg\" (UID: \"bd36563a-fb46-4bb1-878d-c50e9cfcee62\") " pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-nppzg"
Jul 22 16:55:52.092995 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:52.092911    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bd36563a-fb46-4bb1-878d-c50e9cfcee62-proc\") pod \"perf-node-gather-daemonset-nppzg\" (UID: \"bd36563a-fb46-4bb1-878d-c50e9cfcee62\") " pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-nppzg"
Jul 22 16:55:52.092995 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:52.092940    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bd36563a-fb46-4bb1-878d-c50e9cfcee62-lib-modules\") pod \"perf-node-gather-daemonset-nppzg\" (UID: \"bd36563a-fb46-4bb1-878d-c50e9cfcee62\") " pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-nppzg"
Jul 22 16:55:52.092995 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:52.092966    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bd36563a-fb46-4bb1-878d-c50e9cfcee62-podres\") pod \"perf-node-gather-daemonset-nppzg\" (UID: \"bd36563a-fb46-4bb1-878d-c50e9cfcee62\") " pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-nppzg"
Jul 22 16:55:52.194276 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:52.194233    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bd36563a-fb46-4bb1-878d-c50e9cfcee62-sys\") pod \"perf-node-gather-daemonset-nppzg\" (UID: \"bd36563a-fb46-4bb1-878d-c50e9cfcee62\") " pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-nppzg"
Jul 22 16:55:52.194515 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:52.194417    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bd36563a-fb46-4bb1-878d-c50e9cfcee62-sys\") pod \"perf-node-gather-daemonset-nppzg\" (UID: \"bd36563a-fb46-4bb1-878d-c50e9cfcee62\") " pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-nppzg"
Jul 22 16:55:52.195131 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:52.194695    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bd36563a-fb46-4bb1-878d-c50e9cfcee62-proc\") pod \"perf-node-gather-daemonset-nppzg\" (UID: \"bd36563a-fb46-4bb1-878d-c50e9cfcee62\") " pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-nppzg"
Jul 22 16:55:52.195386 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:52.195367    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bd36563a-fb46-4bb1-878d-c50e9cfcee62-lib-modules\") pod \"perf-node-gather-daemonset-nppzg\" (UID: \"bd36563a-fb46-4bb1-878d-c50e9cfcee62\") " pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-nppzg"
Jul 22 16:55:52.195553 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:52.195521    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bd36563a-fb46-4bb1-878d-c50e9cfcee62-podres\") pod \"perf-node-gather-daemonset-nppzg\" (UID: \"bd36563a-fb46-4bb1-878d-c50e9cfcee62\") " pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-nppzg"
Jul 22 16:55:52.195705 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:52.195667    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bd36563a-fb46-4bb1-878d-c50e9cfcee62-podres\") pod \"perf-node-gather-daemonset-nppzg\" (UID: \"bd36563a-fb46-4bb1-878d-c50e9cfcee62\") " pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-nppzg"
Jul 22 16:55:52.195705 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:52.194751    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bd36563a-fb46-4bb1-878d-c50e9cfcee62-proc\") pod \"perf-node-gather-daemonset-nppzg\" (UID: \"bd36563a-fb46-4bb1-878d-c50e9cfcee62\") " pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-nppzg"
Jul 22 16:55:52.195908 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:52.195590    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bd36563a-fb46-4bb1-878d-c50e9cfcee62-lib-modules\") pod \"perf-node-gather-daemonset-nppzg\" (UID: \"bd36563a-fb46-4bb1-878d-c50e9cfcee62\") " pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-nppzg"
Jul 22 16:55:52.195908 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:52.195733    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xn5lg\" (UniqueName: \"kubernetes.io/projected/bd36563a-fb46-4bb1-878d-c50e9cfcee62-kube-api-access-xn5lg\") pod \"perf-node-gather-daemonset-nppzg\" (UID: \"bd36563a-fb46-4bb1-878d-c50e9cfcee62\") " pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-nppzg"
Jul 22 16:55:52.205082 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:52.205054    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn5lg\" (UniqueName: \"kubernetes.io/projected/bd36563a-fb46-4bb1-878d-c50e9cfcee62-kube-api-access-xn5lg\") pod \"perf-node-gather-daemonset-nppzg\" (UID: \"bd36563a-fb46-4bb1-878d-c50e9cfcee62\") " pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-nppzg"
Jul 22 16:55:52.284301 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:52.284213    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-nppzg"
Jul 22 16:55:52.473716 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:52.471140    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bzfkk/perf-node-gather-daemonset-nppzg"]
Jul 22 16:55:53.313722 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:53.313681    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-nppzg" event={"ID":"bd36563a-fb46-4bb1-878d-c50e9cfcee62","Type":"ContainerStarted","Data":"bc5c31c0eef33d7fdb27693b1caa99f77f453663c4642cce4cf23d5c0a211c6a"}
Jul 22 16:55:53.314270 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:53.313788    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-nppzg" event={"ID":"bd36563a-fb46-4bb1-878d-c50e9cfcee62","Type":"ContainerStarted","Data":"dd69166c17241beca73d56488468ec5cbe754e307025f8ceecf1f05d80dc3485"}
Jul 22 16:55:53.314270 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:53.314018    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-nppzg"
Jul 22 16:55:53.332446 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:53.332397    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-nppzg" podStartSLOduration=2.332383803 podStartE2EDuration="2.332383803s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:55:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:55:53.33113278 +0000 UTC m=+2913.461602785" watchObservedRunningTime="2026-07-22 16:55:53.332383803 +0000 UTC m=+2913.462853805"
Jul 22 16:55:54.618713 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:54.618680    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xdxv8_08a6a730-eb43-4bde-9079-47c469793f1d/dns/0.log"
Jul 22 16:55:54.640988 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:54.640961    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xdxv8_08a6a730-eb43-4bde-9079-47c469793f1d/kube-rbac-proxy/0.log"
Jul 22 16:55:54.721161 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:54.721134    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xr6pt_2193e465-f388-4298-80aa-c4fd96ec3f2f/dns-node-resolver/0.log"
Jul 22 16:55:55.293288 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:55.293257    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bzc75_5e6f48d6-b0c6-4827-a1de-17ec418ccaf2/node-ca/0.log"
Jul 22 16:55:56.433210 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:56.433187    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9b87bf6fb-g6x9x_79491bd3-166f-4502-b60d-ec1eb758d07a/kube-auth-proxy/0.log"
Jul 22 16:55:57.287873 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:57.287814    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vmgf6_196426fb-47a2-47d8-8a89-094572fbe64e/serve-healthcheck-canary/0.log"
Jul 22 16:55:57.965464 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:57.965432    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ntb4r_9c8214fa-bb50-454f-a84b-558b13bd799e/kube-rbac-proxy/0.log"
Jul 22 16:55:57.985814 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:57.985782    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ntb4r_9c8214fa-bb50-454f-a84b-558b13bd799e/exporter/0.log"
Jul 22 16:55:58.007178 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:58.007152    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ntb4r_9c8214fa-bb50-454f-a84b-558b13bd799e/extractor/0.log"
Jul 22 16:55:59.333907 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:55:59.333819    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-nppzg"
Jul 22 16:56:00.593447 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:56:00.593396    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6848f74968-64kv4_9d14f63a-49aa-44a4-b1a2-eece5bb92883/manager/0.log"
Jul 22 16:56:00.755599 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:56:00.755557    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-587bf8b54-52jlh_15b90659-90dd-4fd8-aaf5-1e97d6aad202/manager/0.log"
Jul 22 16:56:00.783964 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:56:00.783926    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-587bf8b54-52jlh_15b90659-90dd-4fd8-aaf5-1e97d6aad202/copy-manifests/0.log"
Jul 22 16:56:02.229216 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:56:02.229187    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9b87bf6fb-g6x9x_79491bd3-166f-4502-b60d-ec1eb758d07a/kube-auth-proxy/0.log"
Jul 22 16:56:02.372028 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:56:02.371990    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-b9d69d95c-hjlf7_c829e2d9-e402-4105-a098-64dc65ca6574/manager/0.log"
Jul 22 16:56:03.647049 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:56:03.646977    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9b87bf6fb-g6x9x_79491bd3-166f-4502-b60d-ec1eb758d07a/kube-auth-proxy/0.log"
Jul 22 16:56:07.356288 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:56:07.356257    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-hnxj2_cec619e3-7150-417b-ab93-67b2161ca87c/migrator/0.log"
Jul 22 16:56:07.377157 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:56:07.377127    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-hnxj2_cec619e3-7150-417b-ab93-67b2161ca87c/graceful-termination/0.log"
Jul 22 16:56:09.054673 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:56:09.054646    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xtqkt_64b5ff5e-8328-4e1b-9a8d-341aa0cebba5/kube-multus-additional-cni-plugins/0.log"
Jul 22 16:56:09.076085 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:56:09.076059    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xtqkt_64b5ff5e-8328-4e1b-9a8d-341aa0cebba5/egress-router-binary-copy/0.log"
Jul 22 16:56:09.099137 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:56:09.099114    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xtqkt_64b5ff5e-8328-4e1b-9a8d-341aa0cebba5/cni-plugins/0.log"
Jul 22 16:56:09.120777 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:56:09.120752    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xtqkt_64b5ff5e-8328-4e1b-9a8d-341aa0cebba5/bond-cni-plugin/0.log"
Jul 22 16:56:09.140870 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:56:09.140820    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xtqkt_64b5ff5e-8328-4e1b-9a8d-341aa0cebba5/routeoverride-cni/0.log"
Jul 22 16:56:09.168281 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:56:09.168252    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xtqkt_64b5ff5e-8328-4e1b-9a8d-341aa0cebba5/whereabouts-cni-bincopy/0.log"
Jul 22 16:56:09.190513 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:56:09.190485    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xtqkt_64b5ff5e-8328-4e1b-9a8d-341aa0cebba5/whereabouts-cni/0.log"
Jul 22 16:56:09.262490 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:56:09.262400    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qj9nm_861b97be-45d1-4aee-90eb-b2387ede2142/kube-multus/0.log"
Jul 22 16:56:09.335534 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:56:09.335488    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-87ltq_6440de1a-aff3-4b9d-a374-4e029ece2bfe/network-metrics-daemon/0.log"
Jul 22 16:56:09.357799 ip-10-0-142-224 kubenswrapper[2563]: I0722 16:56:09.357770    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-87ltq_6440de1a-aff3-4b9d-a374-4e029ece2bfe/kube-rbac-proxy/0.log"