Jul 20 16:29:36.642566 ip-10-0-130-85 systemd[1]: Starting Kubernetes Kubelet...
Jul 20 16:29:37.146898 ip-10-0-130-85 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 20 16:29:37.146898 ip-10-0-130-85 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 20 16:29:37.146898 ip-10-0-130-85 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 20 16:29:37.146898 ip-10-0-130-85 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 20 16:29:37.146898 ip-10-0-130-85 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 20 16:29:37.146898 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.986137    2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 20 16:29:37.146898 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994158    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 16:29:37.146898 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994172    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 16:29:37.146898 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994177    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 16:29:37.146898 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994183    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 16:29:37.146898 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994188    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 16:29:37.146898 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994194    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 16:29:37.146898 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994198    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 16:29:37.146898 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994202    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 16:29:37.058141 ip-10-0-130-85 systemd[1]: Started Kubernetes Kubelet.
Jul 20 16:29:37.162760 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994206    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 16:29:37.162760 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994210    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 16:29:37.162760 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994214    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 16:29:37.162760 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994218    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 16:29:37.162760 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994222    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 16:29:37.162760 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994226    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 16:29:37.162760 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994230    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 16:29:37.162760 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994234    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 16:29:37.162760 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994238    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 16:29:37.162760 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994242    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 16:29:37.162760 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994246    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 16:29:37.162760 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994250    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 16:29:37.162760 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994254    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 16:29:37.162760 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994261    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 16:29:37.162760 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994265    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 16:29:37.162760 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994269    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 16:29:37.162760 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994280    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 16:29:37.162760 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994284    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 16:29:37.162760 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994288    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 16:29:37.162760 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994292    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 16:29:37.163569 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994313    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 16:29:37.163569 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994317    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 16:29:37.163569 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994321    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 16:29:37.163569 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994326    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 16:29:37.163569 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994330    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 16:29:37.163569 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994333    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 16:29:37.163569 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994337    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 16:29:37.163569 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994342    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 16:29:37.163569 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994345    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 16:29:37.163569 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994350    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 16:29:37.163569 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994354    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 16:29:37.163569 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994358    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 16:29:37.163569 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994363    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 16:29:37.163569 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994368    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 16:29:37.163569 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994371    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 16:29:37.163569 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994376    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 16:29:37.163569 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994379    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 16:29:37.163569 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994383    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 16:29:37.163569 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994387    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 16:29:37.164341 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994398    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 16:29:37.164341 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994403    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 16:29:37.164341 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994406    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 16:29:37.164341 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994410    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 16:29:37.164341 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994414    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 16:29:37.164341 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994418    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 16:29:37.164341 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994423    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 16:29:37.164341 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994427    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 16:29:37.164341 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994431    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 16:29:37.164341 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994435    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 16:29:37.164341 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994440    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 16:29:37.164341 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994443    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 16:29:37.164341 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994447    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 16:29:37.164341 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994452    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 16:29:37.164341 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994456    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 16:29:37.164341 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994460    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 16:29:37.164341 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994464    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 16:29:37.164341 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994469    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 16:29:37.164341 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994474    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 16:29:37.164934 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994480    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 16:29:37.164934 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994487    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 16:29:37.164934 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994492    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 16:29:37.164934 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994496    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 16:29:37.164934 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994500    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 16:29:37.164934 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994505    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 16:29:37.164934 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994508    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 16:29:37.164934 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994513    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 16:29:37.164934 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994521    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 16:29:37.164934 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994524    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 16:29:37.164934 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994528    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 16:29:37.164934 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994534    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 16:29:37.164934 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994538    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 16:29:37.164934 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994542    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 16:29:37.164934 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994546    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 16:29:37.164934 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994550    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 16:29:37.164934 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994554    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 16:29:37.164934 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994558    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 16:29:37.164934 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994561    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 16:29:37.165727 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994565    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 16:29:37.165727 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994571    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 16:29:37.165727 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994576    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 16:29:37.165727 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994580    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 16:29:37.165727 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994585    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 16:29:37.165727 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994589    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 16:29:37.165727 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994593    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 16:29:37.165727 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994597    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 16:29:37.165727 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994601    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 16:29:37.165727 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994605    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 16:29:37.165727 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994609    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 16:29:37.165727 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994612    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 16:29:37.165727 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994616    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 16:29:37.165727 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.994620    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 16:29:37.165727 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997359    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 16:29:37.165727 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997375    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 16:29:37.165727 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997382    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 16:29:37.165727 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997388    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 16:29:37.165727 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997391    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 16:29:37.166271 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997396    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 16:29:37.166271 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997401    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 16:29:37.166271 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997409    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 16:29:37.166271 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997413    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 16:29:37.166271 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997417    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 16:29:37.166271 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997421    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 16:29:37.166271 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997424    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 16:29:37.166271 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997429    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 16:29:37.166271 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997434    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 16:29:37.166271 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997438    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 16:29:37.166271 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997442    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 16:29:37.166271 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997446    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 16:29:37.166271 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997452    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 16:29:37.166271 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997457    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 16:29:37.166271 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997461    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 16:29:37.166271 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997466    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 16:29:37.166271 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997470    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 16:29:37.166271 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997474    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 16:29:37.166271 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997478    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 16:29:37.166952 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997482    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 16:29:37.166952 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997487    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 16:29:37.166952 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997491    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 16:29:37.166952 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997495    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 16:29:37.166952 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997500    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 16:29:37.166952 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997504    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 16:29:37.166952 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997523    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 16:29:37.166952 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997527    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 16:29:37.166952 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997531    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 16:29:37.166952 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997537    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 16:29:37.166952 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997541    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 16:29:37.166952 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997544    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 16:29:37.166952 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997548    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 16:29:37.166952 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997552    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 16:29:37.166952 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997557    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 16:29:37.166952 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997562    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 16:29:37.166952 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997565    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 16:29:37.166952 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997570    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 16:29:37.166952 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997574    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 16:29:37.166952 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997578    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 16:29:37.167543 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997582    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 16:29:37.167543 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997585    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 16:29:37.167543 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997589    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 16:29:37.167543 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997593    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 16:29:37.167543 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997597    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 16:29:37.167543 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997601    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 16:29:37.167543 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997605    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 16:29:37.167543 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997609    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 16:29:37.167543 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997613    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 16:29:37.167543 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997617    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 16:29:37.167543 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997622    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 16:29:37.167543 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997626    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 16:29:37.167543 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997630    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 16:29:37.167543 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997634    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 16:29:37.167543 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997638    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 16:29:37.167543 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997642    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 16:29:37.167543 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997648    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 16:29:37.167543 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997653    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 16:29:37.167543 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997657    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 16:29:37.167543 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997662    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 16:29:37.168157 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997666    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 16:29:37.168157 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997671    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 16:29:37.168157 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997677    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 16:29:37.168157 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997682    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 16:29:37.168157 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997686    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 16:29:37.168157 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997690    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 16:29:37.168157 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997695    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 16:29:37.168157 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997700    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 16:29:37.168157 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997704    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 16:29:37.168157 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997708    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 16:29:37.168157 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997712    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 16:29:37.168157 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997716    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 16:29:37.168157 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997719    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 16:29:37.168157 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997723    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 16:29:37.168157 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997727    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 16:29:37.168157 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997731    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 16:29:37.168157 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997735    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 16:29:37.168157 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997739    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 16:29:37.168157 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997743    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 16:29:37.168735 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997749    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 16:29:37.168735 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997753    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 16:29:37.168735 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997757    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 16:29:37.168735 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997761    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 16:29:37.168735 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997764    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 16:29:37.168735 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997768    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 16:29:37.168735 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997772    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 16:29:37.168735 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997776    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 16:29:37.168735 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997780    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 16:29:37.168735 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997783    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 16:29:37.168735 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997787    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 16:29:37.168735 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997793    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 16:29:37.168735 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997797    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 16:29:37.168735 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997801    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 16:29:37.168735 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997804    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 16:29:37.168735 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.997808    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 16:29:37.168735 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998434    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 16:29:37.168735 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998443    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 16:29:37.168735 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998448    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 16:29:37.168735 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998452    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 16:29:37.169361 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998459    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 16:29:37.169361 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998464    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 16:29:37.169361 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998469    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 16:29:37.169361 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998474    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 16:29:37.169361 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998478    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 16:29:37.169361 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998482    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 16:29:37.169361 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998486    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 16:29:37.169361 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998490    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 16:29:37.169361 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998494    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 16:29:37.169361 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998498    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 16:29:37.169361 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998502    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 16:29:37.169361 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998507    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 16:29:37.169361 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998511    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 16:29:37.169361 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998518    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 16:29:37.169361 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998522    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 16:29:37.169361 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998526    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 16:29:37.169361 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998530    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 16:29:37.169361 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998535    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 16:29:37.169361 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998538    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 16:29:37.169361 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998542    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 16:29:37.169923 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998546    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 16:29:37.169923 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998550    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 16:29:37.169923 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998554    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 16:29:37.169923 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998559    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 16:29:37.169923 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998563    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 16:29:37.169923 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998567    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 16:29:37.169923 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998571    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 16:29:37.169923 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998575    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 16:29:37.169923 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998580    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 16:29:37.169923 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998584    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 16:29:37.169923 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998588    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 16:29:37.169923 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998594    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 16:29:37.169923 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998601    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 16:29:37.169923 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998606    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 16:29:37.169923 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998610    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 16:29:37.169923 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998613    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 16:29:37.169923 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998617    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 16:29:37.169923 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998622    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 16:29:37.169923 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998626    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 16:29:37.169923 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998629    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 16:29:37.170605 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998634    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 16:29:37.170605 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998638    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 16:29:37.170605 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998642    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 16:29:37.170605 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998645    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 16:29:37.170605 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998650    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 16:29:37.170605 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998655    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 16:29:37.170605 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998659    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 16:29:37.170605 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998663    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 16:29:37.170605 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998667    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 16:29:37.170605 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998671    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 16:29:37.170605 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998675    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 16:29:37.170605 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998679    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 16:29:37.170605 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998684    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 16:29:37.170605 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998688    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 16:29:37.170605 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998692    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 16:29:37.170605 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998698    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 16:29:37.170605 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998704    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 16:29:37.170605 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998708    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 16:29:37.170605 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998712    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 16:29:37.170605 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998716    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 16:29:37.171141 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998720    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 16:29:37.171141 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998724    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 16:29:37.171141 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998728    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 16:29:37.171141 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998732    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 16:29:37.171141 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998737    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 16:29:37.171141 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998741    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 16:29:37.171141 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998745    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 16:29:37.171141 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998749    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 16:29:37.171141 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998752    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 16:29:37.171141 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998756    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 16:29:37.171141 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998760    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 16:29:37.171141 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998764    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 16:29:37.171141 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998768    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 16:29:37.171141 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998772    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 16:29:37.171141 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998776    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 16:29:37.171141 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998779    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 16:29:37.171141 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998783    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 16:29:37.171141 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998787    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 16:29:37.171141 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998792    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 16:29:37.171141 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998795    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 16:29:37.171709 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998799    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 16:29:37.171709 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998803    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 16:29:37.171709 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998806    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 16:29:37.171709 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998810    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 16:29:37.171709 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998814    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 16:29:37.171709 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998817    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 16:29:37.171709 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998821    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 16:29:37.171709 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998827    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 16:29:37.171709 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998831    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 16:29:37.171709 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998835    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 16:29:37.171709 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998839    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 16:29:37.171709 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998843    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 16:29:37.171709 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998846    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 16:29:37.171709 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998850    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 16:29:37.171709 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.998854    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 16:29:37.171709 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.998940    2577 flags.go:64] FLAG: --address="0.0.0.0"
Jul 20 16:29:37.171709 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.998952    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 20 16:29:37.171709 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.998970    2577 flags.go:64] FLAG: --anonymous-auth="true"
Jul 20 16:29:37.171709 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.998977    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 20 16:29:37.171709 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.998983    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 20 16:29:37.172230 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.998988    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 20 16:29:37.172230 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.998995    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 20 16:29:37.172230 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999001    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 20 16:29:37.172230 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999006    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 20 16:29:37.172230 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999010    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 20 16:29:37.172230 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999015    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 20 16:29:37.172230 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999020    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 20 16:29:37.172230 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999025    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 20 16:29:37.172230 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999029    2577 flags.go:64] FLAG: --cgroup-root=""
Jul 20 16:29:37.172230 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999033    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 20 16:29:37.172230 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999037    2577 flags.go:64] FLAG: --client-ca-file=""
Jul 20 16:29:37.172230 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999044    2577 flags.go:64] FLAG: --cloud-provider="external"
Jul 20 16:29:37.172230 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999048    2577 flags.go:64] FLAG: --cluster-dns="[]"
Jul 20 16:29:37.172230 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999055    2577 flags.go:64] FLAG: --cluster-domain=""
Jul 20 16:29:37.172230 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999059    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 20 16:29:37.172230 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999064    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 20 16:29:37.172230 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999068    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 20 16:29:37.172230 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999074    2577 flags.go:64] FLAG: --container-log-max-files="5"
Jul 20 16:29:37.172230 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999080    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 20 16:29:37.172230 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999086    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 20 16:29:37.172230 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999092    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 20 16:29:37.172230 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999097    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 20 16:29:37.172230 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999101    2577 flags.go:64] FLAG: --contention-profiling="false"
Jul 20 16:29:37.172230 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999106    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 20 16:29:37.172914 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999110    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 20 16:29:37.172914 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999115    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 20 16:29:37.172914 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999119    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 20 16:29:37.172914 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999126    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 20 16:29:37.172914 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999131    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 20 16:29:37.172914 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999135    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 20 16:29:37.172914 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999140    2577 flags.go:64] FLAG: --enable-load-reader="false"
Jul 20 16:29:37.172914 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999144    2577 flags.go:64] FLAG: --enable-server="true"
Jul 20 16:29:37.172914 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999165    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 20 16:29:37.172914 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999172    2577 flags.go:64] FLAG: --event-burst="100"
Jul 20 16:29:37.172914 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999177    2577 flags.go:64] FLAG: --event-qps="50"
Jul 20 16:29:37.172914 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999182    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 20 16:29:37.172914 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999187    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 20 16:29:37.172914 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999191    2577 flags.go:64] FLAG: --eviction-hard=""
Jul 20 16:29:37.172914 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999197    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 20 16:29:37.172914 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999201    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 20 16:29:37.172914 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999206    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 20 16:29:37.172914 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999210    2577 flags.go:64] FLAG: --eviction-soft=""
Jul 20 16:29:37.172914 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999215    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 20 16:29:37.172914 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999219    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 20 16:29:37.172914 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999224    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 20 16:29:37.172914 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999229    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 20 16:29:37.172914 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999233    2577 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 20 16:29:37.172914 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999237    2577 flags.go:64] FLAG: --fail-swap-on="true"
Jul 20 16:29:37.172914 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999242    2577 flags.go:64] FLAG: --feature-gates=""
Jul 20 16:29:37.173643 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999247    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 20 16:29:37.173643 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999252    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 20 16:29:37.173643 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999258    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 20 16:29:37.173643 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999263    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 20 16:29:37.173643 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999268    2577 flags.go:64] FLAG: --healthz-port="10248"
Jul 20 16:29:37.173643 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999272    2577 flags.go:64] FLAG: --help="false"
Jul 20 16:29:37.173643 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999276    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-130-85.ec2.internal"
Jul 20 16:29:37.173643 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999281    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 20 16:29:37.173643 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999286    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 20 16:29:37.173643 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999291    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 20 16:29:37.173643 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999316    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 20 16:29:37.173643 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999325    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 20 16:29:37.173643 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999329    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 20 16:29:37.173643 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999333    2577 flags.go:64] FLAG: --image-service-endpoint=""
Jul 20 16:29:37.173643 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999338    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 20 16:29:37.173643 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999342    2577 flags.go:64] FLAG: --kube-api-burst="100"
Jul 20 16:29:37.173643 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999346    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 20 16:29:37.173643 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999351    2577 flags.go:64] FLAG: --kube-api-qps="50"
Jul 20 16:29:37.173643 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999356    2577 flags.go:64] FLAG: --kube-reserved=""
Jul 20 16:29:37.173643 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999361    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 20 16:29:37.173643 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999365    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 20 16:29:37.173643 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999370    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 20 16:29:37.173643 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999373    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 20 16:29:37.173643 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999378    2577 flags.go:64] FLAG: --lock-file=""
Jul 20 16:29:37.174340 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999383    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 20 16:29:37.174340 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999387    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 20 16:29:37.174340 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999392    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 20 16:29:37.174340 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999400    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 20 16:29:37.174340 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999406    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 20 16:29:37.174340 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999410    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 20 16:29:37.174340 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999415    2577 flags.go:64] FLAG: --logging-format="text"
Jul 20 16:29:37.174340 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999419    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 20 16:29:37.174340 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999423    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 20 16:29:37.174340 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999427    2577 flags.go:64] FLAG: --manifest-url=""
Jul 20 16:29:37.174340 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999433    2577 flags.go:64] FLAG: --manifest-url-header=""
Jul 20 16:29:37.174340 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999440    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 20 16:29:37.174340 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999444    2577 flags.go:64] FLAG: --max-open-files="1000000"
Jul 20 16:29:37.174340 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999450    2577 flags.go:64] FLAG: --max-pods="110"
Jul 20 16:29:37.174340 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999454    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 20 16:29:37.174340 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999459    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 20 16:29:37.174340 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999463    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 20 16:29:37.174340 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999468    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 20 16:29:37.174340 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999472    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 20 16:29:37.174340 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999478    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 20 16:29:37.174340 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999483    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 20 16:29:37.174340 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999494    2577 flags.go:64] FLAG: --node-status-max-images="50"
Jul 20 16:29:37.174340 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999498    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 20 16:29:37.174340 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999504    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 20 16:29:37.174340 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999508    2577 flags.go:64] FLAG: --pod-cidr=""
Jul 20 16:29:37.175309 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999512    2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 20 16:29:37.175309 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999520    2577 flags.go:64] FLAG: --pod-manifest-path=""
Jul 20 16:29:37.175309 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999525    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 20 16:29:37.175309 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999529    2577 flags.go:64] FLAG: --pods-per-core="0"
Jul 20 16:29:37.175309 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999534    2577 flags.go:64] FLAG: --port="10250"
Jul 20 16:29:37.175309 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999538    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 20 16:29:37.175309 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999544    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-056daa9da9ab0be58"
Jul 20 16:29:37.175309 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999549    2577 flags.go:64] FLAG: --qos-reserved=""
Jul 20 16:29:37.175309 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999553    2577 flags.go:64] FLAG: --read-only-port="10255"
Jul 20 16:29:37.175309 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999558    2577 flags.go:64] FLAG: --register-node="true"
Jul 20 16:29:37.175309 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999562    2577 flags.go:64] FLAG: --register-with-taints=""
Jul 20 16:29:37.175309 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999567    2577 flags.go:64] FLAG: --registry-burst="10"
Jul 20 16:29:37.175309 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999572    2577 flags.go:64] FLAG: --registry-qps="5"
Jul 20 16:29:37.175309 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999576    2577 flags.go:64] FLAG: --reserved-cpus=""
Jul 20 16:29:37.175309 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999581    2577 flags.go:64] FLAG: --reserved-memory=""
Jul 20 16:29:37.175309 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999586    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 20 16:29:37.175309 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999590    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 20 16:29:37.175309 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999597    2577 flags.go:64] FLAG: --rotate-certificates="false"
Jul 20 16:29:37.175309 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999601    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 20 16:29:37.175309 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999606    2577 flags.go:64] FLAG: --runonce="false"
Jul 20 16:29:37.175309 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999610    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 20 16:29:37.175309 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999615    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 20 16:29:37.175309 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999619    2577 flags.go:64] FLAG: --seccomp-default="false"
Jul 20 16:29:37.175309 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999624    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 20 16:29:37.175309 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999628    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 20 16:29:37.176013 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999633    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 20 16:29:37.176013 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999637    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 20 16:29:37.176013 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999643    2577 flags.go:64] FLAG: --storage-driver-password="root"
Jul 20 16:29:37.176013 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999648    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 20 16:29:37.176013 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999652    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 20 16:29:37.176013 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999656    2577 flags.go:64] FLAG: --storage-driver-user="root"
Jul 20 16:29:37.176013 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999661    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 20 16:29:37.176013 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999665    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 20 16:29:37.176013 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999669    2577 flags.go:64] FLAG: --system-cgroups=""
Jul 20 16:29:37.176013 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999674    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 20 16:29:37.176013 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999682    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 20 16:29:37.176013 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999686    2577 flags.go:64] FLAG: --tls-cert-file=""
Jul 20 16:29:37.176013 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999690    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 20 16:29:37.176013 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999697    2577 flags.go:64] FLAG: --tls-min-version=""
Jul 20 16:29:37.176013 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999701    2577 flags.go:64] FLAG: --tls-private-key-file=""
Jul 20 16:29:37.176013 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999705    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 20 16:29:37.176013 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999709    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 20 16:29:37.176013 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999714    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 20 16:29:37.176013 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999718    2577 flags.go:64] FLAG: --v="2"
Jul 20 16:29:37.176013 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999723    2577 flags.go:64] FLAG: --version="false"
Jul 20 16:29:37.176013 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999730    2577 flags.go:64] FLAG: --vmodule=""
Jul 20 16:29:37.176013 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999735    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 20 16:29:37.176013 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:36.999740    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 20 16:29:37.176013 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.999904    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 16:29:37.176013 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.999911    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 16:29:37.370377 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.999916    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 16:29:37.370377 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.999920    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 16:29:37.370377 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.999924    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 16:29:37.370377 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.999928    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 16:29:37.370377 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.999932    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 16:29:37.370377 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.999937    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 16:29:37.370377 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.999943    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 16:29:37.370377 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.999948    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 16:29:37.370377 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.999952    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 16:29:37.370377 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.999959    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 16:29:37.370377 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.999962    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 16:29:37.370377 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.999967    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 16:29:37.370377 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.999971    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 16:29:37.370377 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.999975    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 16:29:37.370377 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.999979    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 16:29:37.370377 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.999983    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 16:29:37.370377 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.999986    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 16:29:37.370377 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.999990    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 16:29:37.370377 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.999994    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 16:29:37.370377 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:36.999998    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 16:29:37.371168 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000002    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 16:29:37.371168 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000006    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 16:29:37.371168 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000010    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 16:29:37.371168 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000015    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 16:29:37.371168 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000018    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 16:29:37.371168 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000022    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 16:29:37.371168 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000028    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 16:29:37.371168 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000033    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 16:29:37.371168 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000041    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 16:29:37.371168 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000045    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 16:29:37.371168 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000049    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 16:29:37.371168 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000054    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 16:29:37.371168 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000058    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 16:29:37.371168 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000062    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 16:29:37.371168 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000065    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 16:29:37.371168 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000069    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 16:29:37.371168 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000073    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 16:29:37.371168 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000077    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 16:29:37.371168 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000081    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 16:29:37.371168 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000085    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 16:29:37.371865 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000088    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 16:29:37.371865 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000094    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 16:29:37.371865 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000098    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 16:29:37.371865 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000102    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 16:29:37.371865 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000106    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 16:29:37.371865 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000110    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 16:29:37.371865 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000115    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 16:29:37.371865 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000118    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 16:29:37.371865 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000122    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 16:29:37.371865 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000126    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 16:29:37.371865 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000130    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 16:29:37.371865 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000134    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 16:29:37.371865 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000138    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 16:29:37.371865 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000142    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 16:29:37.371865 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000146    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 16:29:37.371865 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000149    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 16:29:37.371865 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000153    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 16:29:37.371865 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000157    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 16:29:37.371865 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000161    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 16:29:37.371865 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000165    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 16:29:37.372442 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000168    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 16:29:37.372442 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000173    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 16:29:37.372442 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000179    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 16:29:37.372442 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000184    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 16:29:37.372442 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000189    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 16:29:37.372442 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000193    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 16:29:37.372442 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000198    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 16:29:37.372442 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000201    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 16:29:37.372442 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000205    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 16:29:37.372442 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000209    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 16:29:37.372442 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000213    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 16:29:37.372442 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000217    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 16:29:37.372442 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000221    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 16:29:37.372442 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000227    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 16:29:37.372442 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000231    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 16:29:37.372442 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000235    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 16:29:37.372442 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000239    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 16:29:37.372442 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000243    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 16:29:37.372442 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000247    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 16:29:37.373041 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000250    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 16:29:37.373041 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000254    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 16:29:37.373041 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000258    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 16:29:37.373041 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000262    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 16:29:37.373041 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000266    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 16:29:37.373041 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000270    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 16:29:37.373041 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000274    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 16:29:37.373041 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000277    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 16:29:37.373041 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000281    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 16:29:37.373041 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000285    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 16:29:37.373041 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000290    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 16:29:37.373041 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000307    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 16:29:37.373041 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000312    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 16:29:37.373041 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000316    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 16:29:37.373041 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000320    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 16:29:37.373041 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000324    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 16:29:37.373041 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000330    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 16:29:37.373041 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.000334    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 16:29:37.373598 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.000342    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 20 16:29:37.373598 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.006911    2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 20 16:29:37.373598 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.006925    2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 20 16:29:37.373598 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.006945    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 20 16:29:37.373598 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.006951    2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 20 16:29:37.373598 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007005    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 16:29:37.373598 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007010    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 16:29:37.373598 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007013    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 16:29:37.373598 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007016    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 16:29:37.373598 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007019    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 16:29:37.373598 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007022    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 16:29:37.373598 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007026    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 16:29:37.373598 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007029    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 16:29:37.373598 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007032    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 16:29:37.373598 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007036    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 16:29:37.374015 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007040    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 16:29:37.374015 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007043    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 16:29:37.374015 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007047    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 16:29:37.374015 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007051    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 16:29:37.374015 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007054    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 16:29:37.374015 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007057    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 16:29:37.374015 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007060    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 16:29:37.374015 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007064    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 16:29:37.374015 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007067    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 16:29:37.374015 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007071    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 16:29:37.374015 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007075    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 16:29:37.374015 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007079    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 16:29:37.374015 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007083    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 16:29:37.374015 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007086    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 16:29:37.374015 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007090    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 16:29:37.374015 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007092    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 16:29:37.374015 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007095    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 16:29:37.374015 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007098    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 16:29:37.374015 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007101    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 16:29:37.374594 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007107    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 16:29:37.374594 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007110    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 16:29:37.374594 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007112    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 16:29:37.374594 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007115    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 16:29:37.374594 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007118    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 16:29:37.374594 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007121    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 16:29:37.374594 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007124    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 16:29:37.374594 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007127    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 16:29:37.374594 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007129    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 16:29:37.374594 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007132    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 16:29:37.374594 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007135    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 16:29:37.374594 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007137    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 16:29:37.374594 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007140    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 16:29:37.374594 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007143    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 16:29:37.374594 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007145    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 16:29:37.374594 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007148    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 16:29:37.374594 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007151    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 16:29:37.374594 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007154    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 16:29:37.374594 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007156    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 16:29:37.375087 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007159    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 16:29:37.375087 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007162    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 16:29:37.375087 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007164    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 16:29:37.375087 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007167    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 16:29:37.375087 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007170    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 16:29:37.375087 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007172    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 16:29:37.375087 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007175    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 16:29:37.375087 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007178    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 16:29:37.375087 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007180    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 16:29:37.375087 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007183    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 16:29:37.375087 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007186    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 16:29:37.375087 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007188    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 16:29:37.375087 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007191    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 16:29:37.375087 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007200    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 16:29:37.375087 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007203    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 16:29:37.375087 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007205    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 16:29:37.375087 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007208    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 16:29:37.375087 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007211    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 16:29:37.375087 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007214    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 16:29:37.375087 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007217    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 16:29:37.375697 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007219    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 16:29:37.375697 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007222    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 16:29:37.375697 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007225    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 16:29:37.375697 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007227    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 16:29:37.375697 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007230    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 16:29:37.375697 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007233    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 16:29:37.375697 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007235    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 16:29:37.375697 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007238    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 16:29:37.375697 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007241    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 16:29:37.375697 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007243    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 16:29:37.375697 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007246    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 16:29:37.375697 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007248    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 16:29:37.375697 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007251    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 16:29:37.375697 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007254    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 16:29:37.375697 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007256    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 16:29:37.375697 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007259    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 16:29:37.375697 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007262    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 16:29:37.375697 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007264    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 16:29:37.375697 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007267    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 16:29:37.376202 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007270    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 16:29:37.376202 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007272    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 16:29:37.376202 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007275    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 16:29:37.376202 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007279    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 16:29:37.376202 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007281    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 16:29:37.376202 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007284    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 16:29:37.376202 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007289    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 16:29:37.376202 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007292    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 16:29:37.376202 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007314    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 16:29:37.376202 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007317    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 16:29:37.376202 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007320    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 16:29:37.376202 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007323    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 16:29:37.376202 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.007329    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 20 16:29:37.376202 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007793    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 16:29:37.376202 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007798    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 16:29:37.376668 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007801    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 16:29:37.376668 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007804    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 16:29:37.376668 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007809    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 16:29:37.376668 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007813    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 16:29:37.376668 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007816    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 16:29:37.376668 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007819    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 16:29:37.376668 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007822    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 16:29:37.376668 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007825    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 16:29:37.376668 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007830    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 16:29:37.376668 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007834    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 16:29:37.376668 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007837    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 16:29:37.376668 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007840    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 16:29:37.376668 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007843    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 16:29:37.376668 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007845    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 16:29:37.376668 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007848    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 16:29:37.376668 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007851    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 16:29:37.376668 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007854    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 16:29:37.376668 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007856    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 16:29:37.376668 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007859    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 16:29:37.377206 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007862    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 16:29:37.377206 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007864    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 16:29:37.377206 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007868    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 16:29:37.377206 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007870    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 16:29:37.377206 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007874    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 16:29:37.377206 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007878    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 16:29:37.377206 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007881    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 16:29:37.377206 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007884    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 16:29:37.377206 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007887    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 16:29:37.377206 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007889    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 16:29:37.377206 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007892    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 16:29:37.377206 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007894    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 16:29:37.377206 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007897    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 16:29:37.377206 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007900    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 16:29:37.377206 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007902    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 16:29:37.377206 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007905    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 16:29:37.377206 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007908    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 16:29:37.377206 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007910    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 16:29:37.377206 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007913    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 16:29:37.377206 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007916    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 16:29:37.377809 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007918    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 16:29:37.377809 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007921    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 16:29:37.377809 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007924    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 16:29:37.377809 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007926    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 16:29:37.377809 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007929    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 16:29:37.377809 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007932    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 16:29:37.377809 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007934    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 16:29:37.377809 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007937    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 16:29:37.377809 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007939    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 16:29:37.377809 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007942    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 16:29:37.377809 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007944    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 16:29:37.377809 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007947    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 16:29:37.377809 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007950    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 16:29:37.377809 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007952    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 16:29:37.377809 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007955    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 16:29:37.377809 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007958    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 16:29:37.377809 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007962    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 16:29:37.377809 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007966    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 16:29:37.377809 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007969    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 16:29:37.377809 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007986    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 16:29:37.378352 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007990    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 16:29:37.378352 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007993    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 16:29:37.378352 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007996    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 16:29:37.378352 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.007999    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 16:29:37.378352 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.008002    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 16:29:37.378352 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.008005    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 16:29:37.378352 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.008008    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 16:29:37.378352 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.008011    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 16:29:37.378352 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.008014    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 16:29:37.378352 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.008017    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 16:29:37.378352 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.008020    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 16:29:37.378352 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.008023    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 16:29:37.378352 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.008025    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 16:29:37.378352 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.008028    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 16:29:37.378352 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.008031    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 16:29:37.378352 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.008033    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 16:29:37.378352 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.008036    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 16:29:37.378352 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.008039    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 16:29:37.378352 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.008041    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 16:29:37.378352 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.008044    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 16:29:37.378922 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.008047    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 16:29:37.378922 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.008049    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 16:29:37.378922 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.008052    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 16:29:37.378922 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.008054    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 16:29:37.378922 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.008057    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 16:29:37.378922 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.008060    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 16:29:37.378922 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.008062    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 16:29:37.378922 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.008066    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 16:29:37.378922 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.008073    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 16:29:37.378922 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.008077    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 16:29:37.378922 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.008080    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 16:29:37.378922 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.008082    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 16:29:37.378922 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.008085    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 16:29:37.378922 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.008088    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 16:29:37.378922 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.008091    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 16:29:37.378922 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.008094    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 16:29:37.378922 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.008096    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 16:29:37.378922 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:37.008099    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 16:29:37.379403 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.008103    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 20 16:29:37.379403 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.008207    2577 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 20 16:29:37.379403 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.010282    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 20 16:29:37.379403 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.011482    2577 server.go:1019] "Starting client certificate rotation"
Jul 20 16:29:37.379403 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.011580    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 20 16:29:37.379403 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.011610    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 20 16:29:37.379403 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.013619    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 20 16:29:37.379403 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.015321    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 20 16:29:37.379403 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.021956    2577 log.go:25] "Validated CRI v1 runtime API"
Jul 20 16:29:37.379403 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.028274    2577 log.go:25] "Validated CRI v1 image API"
Jul 20 16:29:37.379403 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.029775    2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 20 16:29:37.379403 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.031870    2577 fs.go:135] Filesystem UUIDs: map[099934b5-4086-4312-9e64-62a1c6e0687b:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 89883cdc-dc8d-476e-876e-c10cd11b5d19:/dev/nvme0n1p3]
Jul 20 16:29:37.379792 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.031885    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 20 16:29:37.379862 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.036780    2577 manager.go:216] Machine: {Timestamp:2026-07-20 16:29:37.035626015 +0000 UTC m=+0.285120025 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3106064 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2914859d68e6833fff4172ec9e5436 SystemUUID:ec291485-9d68-e683-3fff-4172ec9e5436 BootID:44939170-73fd-411f-86f8-a6b07eacc6f5 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:90:ff:f3:83:4b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:90:ff:f3:83:4b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7e:9d:c0:ab:bb:2e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 20 16:29:37.379862 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.036880    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 20 16:29:37.379862 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.036975    2577 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 20 16:29:37.379862 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.038304    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 20 16:29:37.379862 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.039933    2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 20 16:29:37.379862 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.039958    2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-85.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 20 16:29:37.379862 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.040668    2577 topology_manager.go:138] "Creating topology manager with none policy"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.040682    2577 container_manager_linux.go:309] "Creating device plugin manager"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.040691    2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.041275    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.043264    2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.047143    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.047282    2577 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.047333    2577 kubelet.go:486] "Attempting to sync node with API server"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.047347    2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.047366    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.047376    2577 kubelet.go:398] "Adding apiserver pod source"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.047388    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.047781    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.047800    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.049856    2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.051912    2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.051945    2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.052974    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.052993    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.053001    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.053008    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.053018    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.053027    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.053037    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.053043    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.053076    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.053093    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.053107    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.053113    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.053689    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.053697    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.057112    2577 server.go:1268] "Started kubelet"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.057320    2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.057343    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.057463    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.057749    2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-85.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:37.057893    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-85.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:37.057956    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.060147    2577 server.go:310] "Adding debug handlers to kubelet server"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.061742    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-559ff"
Jul 20 16:29:37.380140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.062445    2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:37.063740    2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:37.062608    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-85.ec2.internal.18c40c644013f368  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-85.ec2.internal,UID:ip-10-0-130-85.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-85.ec2.internal,},FirstTimestamp:2026-07-20 16:29:37.056871272 +0000 UTC m=+0.306365284,LastTimestamp:2026-07-20 16:29:37.056871272 +0000 UTC m=+0.306365284,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-85.ec2.internal,}"
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.068671    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-559ff"
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.069427    2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.069445    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:37.069554    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-85.ec2.internal\" not found"
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.070091    2577 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.070097    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.070108    2577 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.070245    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.070259    2577 factory.go:55] Registering systemd factory
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.070267    2577 factory.go:223] Registration of the systemd container factory successfully
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.070279    2577 reconstruct.go:97] "Volume reconstruction finished"
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.070289    2577 reconciler.go:29] "Reconciler: start to sync state"
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.070499    2577 factory.go:153] Registering CRI-O factory
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.070514    2577 factory.go:223] Registration of the crio container factory successfully
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.070538    2577 factory.go:103] Registering Raw factory
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.070552    2577 manager.go:1196] Started watching for new ooms in manager
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.070994    2577 manager.go:349] Starting recovery of all containers
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.077675    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:37.081115    2577 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-130-85.ec2.internal\" not found" node="ip-10-0-130-85.ec2.internal"
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.082658    2577 manager.go:354] Recovery completed
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.086887    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.088969    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-85.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.088995    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-85.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.089005    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-85.ec2.internal" event="NodeHasSufficientPID"
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.089492    2577 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.089500    2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.089519    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.089549    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.090215    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-85.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.090241    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-85.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.090251    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-85.ec2.internal" event="NodeHasSufficientPID"
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.091537    2577 policy_none.go:49] "None policy: Start"
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.091551    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 20 16:29:37.570183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.091564    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.093476    2577 policy_none.go:47] "Start"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.131834    2577 manager.go:337] "Starting Device Plugin manager"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:37.131924    2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.131941    2577 server.go:87] "Starting device plugin registration server"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.132185    2577 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.132194    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.132309    2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.132396    2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.132406    2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:37.132850    2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:37.132903    2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-85.ec2.internal\" not found"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.204220    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.205612    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.205651    2577 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.205681    2577 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:37.205721    2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.207934    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.232797    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.233677    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-85.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.233708    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-85.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.233722    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-85.ec2.internal" event="NodeHasSufficientPID"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.233747    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-85.ec2.internal"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.241856    2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-85.ec2.internal"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:37.241875    2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-85.ec2.internal\": node \"ip-10-0-130-85.ec2.internal\" not found"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:37.255778    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-85.ec2.internal\" not found"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.306006    2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-85.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-85.ec2.internal"]
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.306071    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.307610    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-85.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.307641    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-85.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.307651    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-85.ec2.internal" event="NodeHasSufficientPID"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.309897    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.310010    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-85.ec2.internal"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.310033    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.310589    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-85.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.310600    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-85.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.310615    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-85.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 16:29:37.571282 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.310621    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-85.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 16:29:37.573085 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.310626    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-85.ec2.internal" event="NodeHasSufficientPID"
Jul 20 16:29:37.573085 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.310635    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-85.ec2.internal" event="NodeHasSufficientPID"
Jul 20 16:29:37.573085 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.312859    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-85.ec2.internal"
Jul 20 16:29:37.573085 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.312891    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 16:29:37.573085 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.313477    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-85.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 16:29:37.573085 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.313502    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-85.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 16:29:37.573085 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.313514    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-85.ec2.internal" event="NodeHasSufficientPID"
Jul 20 16:29:37.573085 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:37.343892    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-85.ec2.internal\" not found" node="ip-10-0-130-85.ec2.internal"
Jul 20 16:29:37.573085 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:37.348264    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-85.ec2.internal\" not found" node="ip-10-0-130-85.ec2.internal"
Jul 20 16:29:37.573085 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:37.356670    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-85.ec2.internal\" not found"
Jul 20 16:29:37.573085 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.371616    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b5c2cca67b804c078f4c05792f7283b5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-85.ec2.internal\" (UID: \"b5c2cca67b804c078f4c05792f7283b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-85.ec2.internal"
Jul 20 16:29:37.573085 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.371636    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5c2cca67b804c078f4c05792f7283b5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-85.ec2.internal\" (UID: \"b5c2cca67b804c078f4c05792f7283b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-85.ec2.internal"
Jul 20 16:29:37.573085 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.371657    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/786ae2d571f496bce3302e9386a601e9-config\") pod \"kube-apiserver-proxy-ip-10-0-130-85.ec2.internal\" (UID: \"786ae2d571f496bce3302e9386a601e9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-85.ec2.internal"
Jul 20 16:29:37.573085 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:37.457618    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-85.ec2.internal\" not found"
Jul 20 16:29:37.573085 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.471896    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b5c2cca67b804c078f4c05792f7283b5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-85.ec2.internal\" (UID: \"b5c2cca67b804c078f4c05792f7283b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-85.ec2.internal"
Jul 20 16:29:37.573085 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.471922    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5c2cca67b804c078f4c05792f7283b5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-85.ec2.internal\" (UID: \"b5c2cca67b804c078f4c05792f7283b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-85.ec2.internal"
Jul 20 16:29:37.573085 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.471939    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/786ae2d571f496bce3302e9386a601e9-config\") pod \"kube-apiserver-proxy-ip-10-0-130-85.ec2.internal\" (UID: \"786ae2d571f496bce3302e9386a601e9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-85.ec2.internal"
Jul 20 16:29:37.573085 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.471978    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/786ae2d571f496bce3302e9386a601e9-config\") pod \"kube-apiserver-proxy-ip-10-0-130-85.ec2.internal\" (UID: \"786ae2d571f496bce3302e9386a601e9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-85.ec2.internal"
Jul 20 16:29:37.573085 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.471998    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5c2cca67b804c078f4c05792f7283b5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-85.ec2.internal\" (UID: \"b5c2cca67b804c078f4c05792f7283b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-85.ec2.internal"
Jul 20 16:29:37.573662 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.471998    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b5c2cca67b804c078f4c05792f7283b5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-85.ec2.internal\" (UID: \"b5c2cca67b804c078f4c05792f7283b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-85.ec2.internal"
Jul 20 16:29:37.573662 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:37.558287    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-85.ec2.internal\" not found"
Jul 20 16:29:37.786105 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.651294    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-85.ec2.internal"
Jul 20 16:29:37.786105 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.651298    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-85.ec2.internal"
Jul 20 16:29:37.786105 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:37.659254    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-85.ec2.internal\" not found"
Jul 20 16:29:37.786105 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:37.759771    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-85.ec2.internal\" not found"
Jul 20 16:29:37.998678 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:37.861211    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-85.ec2.internal\" not found"
Jul 20 16:29:37.998678 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.910915    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 16:29:37.998678 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.970570    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-85.ec2.internal"
Jul 20 16:29:37.998678 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.980460    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 20 16:29:37.998678 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.981982    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-85.ec2.internal"
Jul 20 16:29:37.998678 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:37.992234    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 20 16:29:38.151491 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.011602    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 20 16:29:38.151491 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.011750    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 20 16:29:38.151491 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.011749    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 20 16:29:38.151491 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.011755    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 20 16:29:38.151491 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.048356    2577 apiserver.go:52] "Watching apiserver"
Jul 20 16:29:38.151491 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.064647    2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 20 16:29:38.151491 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.064999    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-5bsgv","openshift-ovn-kubernetes/ovnkube-node-gd4p4","kube-system/global-pull-secret-syncer-4957k","kube-system/konnectivity-agent-82fz4","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgsds","openshift-dns/node-resolver-h568x","openshift-image-registry/node-ca-qq95v","openshift-multus/multus-72dfr","openshift-multus/multus-additional-cni-plugins-gsdf2","openshift-network-diagnostics/network-check-target-ss6qh","kube-system/kube-apiserver-proxy-ip-10-0-130-85.ec2.internal","openshift-cluster-node-tuning-operator/tuned-tr8vn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-85.ec2.internal","openshift-multus/network-metrics-daemon-2hx9p"]
Jul 20 16:29:38.151491 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.067721    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgsds"
Jul 20 16:29:38.151491 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.070131    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 20 16:29:38.151491 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.070468    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 20 16:29:38.151491 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.070609    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-6xlqj\""
Jul 20 16:29:38.151491 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.070750    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 20 16:29:38.151491 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.070868    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.151491 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.071441    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 20 16:29:38.151491 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.072252    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qq95v"
Jul 20 16:29:38.151491 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.073004    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 20 16:29:38.151491 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.073620    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 20 16:29:38.151491 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.073686    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-922wt\""
Jul 20 16:29:38.151491 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.073908    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-19 16:24:37 +0000 UTC" deadline="2027-12-27 18:22:18.351440335 +0000 UTC"
Jul 20 16:29:38.151491 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.073946    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12601h52m40.277496528s"
Jul 20 16:29:38.151491 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.074904    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 20 16:29:38.151491 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.075151    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 20 16:29:38.151491 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.075263    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 20 16:29:38.151491 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.075432    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4x88t\""
Jul 20 16:29:38.153660 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.075498    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d8418755-37c8-4c02-bdcd-1ac4c903c6ab-serviceca\") pod \"node-ca-qq95v\" (UID: \"d8418755-37c8-4c02-bdcd-1ac4c903c6ab\") " pod="openshift-image-registry/node-ca-qq95v"
Jul 20 16:29:38.153660 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.075539    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgg4w\" (UniqueName: \"kubernetes.io/projected/d8418755-37c8-4c02-bdcd-1ac4c903c6ab-kube-api-access-tgg4w\") pod \"node-ca-qq95v\" (UID: \"d8418755-37c8-4c02-bdcd-1ac4c903c6ab\") " pod="openshift-image-registry/node-ca-qq95v"
Jul 20 16:29:38.153660 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.075569    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f2e27d15-4856-4628-bd38-eb55b3ae21c3-etc-sysconfig\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.153660 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.075613    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f2e27d15-4856-4628-bd38-eb55b3ae21c3-etc-sysctl-d\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.153660 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.075701    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ed50e3c6-a333-426a-89d8-56f99d082d59-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fgsds\" (UID: \"ed50e3c6-a333-426a-89d8-56f99d082d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgsds"
Jul 20 16:29:38.153660 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.075741    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ed50e3c6-a333-426a-89d8-56f99d082d59-device-dir\") pod \"aws-ebs-csi-driver-node-fgsds\" (UID: \"ed50e3c6-a333-426a-89d8-56f99d082d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgsds"
Jul 20 16:29:38.153660 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.075764    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-82fz4"
Jul 20 16:29:38.153660 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.075769    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ed50e3c6-a333-426a-89d8-56f99d082d59-sys-fs\") pod \"aws-ebs-csi-driver-node-fgsds\" (UID: \"ed50e3c6-a333-426a-89d8-56f99d082d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgsds"
Jul 20 16:29:38.153660 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.075806    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f2e27d15-4856-4628-bd38-eb55b3ae21c3-etc-sysctl-conf\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.153660 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.075840    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f2e27d15-4856-4628-bd38-eb55b3ae21c3-lib-modules\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.153660 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.075912    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2e27d15-4856-4628-bd38-eb55b3ae21c3-var-lib-kubelet\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.153660 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.075956    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czm4w\" (UniqueName: \"kubernetes.io/projected/f2e27d15-4856-4628-bd38-eb55b3ae21c3-kube-api-access-czm4w\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.153660 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.076017    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8418755-37c8-4c02-bdcd-1ac4c903c6ab-host\") pod \"node-ca-qq95v\" (UID: \"d8418755-37c8-4c02-bdcd-1ac4c903c6ab\") " pod="openshift-image-registry/node-ca-qq95v"
Jul 20 16:29:38.153660 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.076051    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ed50e3c6-a333-426a-89d8-56f99d082d59-socket-dir\") pod \"aws-ebs-csi-driver-node-fgsds\" (UID: \"ed50e3c6-a333-426a-89d8-56f99d082d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgsds"
Jul 20 16:29:38.154412 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.076083    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ed50e3c6-a333-426a-89d8-56f99d082d59-registration-dir\") pod \"aws-ebs-csi-driver-node-fgsds\" (UID: \"ed50e3c6-a333-426a-89d8-56f99d082d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgsds"
Jul 20 16:29:38.154412 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.076123    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2rxg\" (UniqueName: \"kubernetes.io/projected/ed50e3c6-a333-426a-89d8-56f99d082d59-kube-api-access-l2rxg\") pod \"aws-ebs-csi-driver-node-fgsds\" (UID: \"ed50e3c6-a333-426a-89d8-56f99d082d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgsds"
Jul 20 16:29:38.154412 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.076171    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f2e27d15-4856-4628-bd38-eb55b3ae21c3-etc-modprobe-d\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.154412 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.076231    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2e27d15-4856-4628-bd38-eb55b3ae21c3-sys\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.154412 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.076261    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f2e27d15-4856-4628-bd38-eb55b3ae21c3-host\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.154412 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.076289    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f2e27d15-4856-4628-bd38-eb55b3ae21c3-etc-tuned\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.154412 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.076316    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2e27d15-4856-4628-bd38-eb55b3ae21c3-tmp\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.154412 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.076355    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ed50e3c6-a333-426a-89d8-56f99d082d59-etc-selinux\") pod \"aws-ebs-csi-driver-node-fgsds\" (UID: \"ed50e3c6-a333-426a-89d8-56f99d082d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgsds"
Jul 20 16:29:38.154412 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.076402    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2e27d15-4856-4628-bd38-eb55b3ae21c3-etc-kubernetes\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.154412 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.076436    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f2e27d15-4856-4628-bd38-eb55b3ae21c3-etc-systemd\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.154412 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.076475    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f2e27d15-4856-4628-bd38-eb55b3ae21c3-run\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.154412 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.077824    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:29:38.154412 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:38.078037    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4957k" podUID="1230abfc-e6e1-432a-b99e-2d700df05d26"
Jul 20 16:29:38.154412 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.078316    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 20 16:29:38.154412 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.078347    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-clxcx\""
Jul 20 16:29:38.154412 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.078630    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 20 16:29:38.154412 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.080040    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-h568x"
Jul 20 16:29:38.155347 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.082394    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5bsgv"
Jul 20 16:29:38.155347 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.082830    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 20 16:29:38.155347 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.082887    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6lrf6\""
Jul 20 16:29:38.155347 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.082953    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 20 16:29:38.155347 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.082980    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 20 16:29:38.155347 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.084746    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 20 16:29:38.155347 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.085422    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-27h4q\""
Jul 20 16:29:38.155347 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.085466    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 20 16:29:38.155347 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.085564    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 20 16:29:38.155347 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.090544    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.155347 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.093035    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 20 16:29:38.155347 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.093121    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gsdf2"
Jul 20 16:29:38.155347 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.093364    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 20 16:29:38.155347 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.093384    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-vfjp6\""
Jul 20 16:29:38.155347 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.093554    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 20 16:29:38.155347 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.093593    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 20 16:29:38.155347 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.095363    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pwqdw\""
Jul 20 16:29:38.155347 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.095675    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ss6qh"
Jul 20 16:29:38.155347 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.095713    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 20 16:29:38.155347 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:38.095740    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ss6qh" podUID="3cb425f5-e1bd-49d1-ab10-38f4431ebc91"
Jul 20 16:29:38.155347 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.095680    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 20 16:29:38.155347 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.098053    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.155347 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.100352    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 20 16:29:38.155347 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.100556    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 20 16:29:38.155347 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.100608    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 20 16:29:38.155347 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.100686    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 20 16:29:38.155347 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.100824    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 20 16:29:38.155347 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.100891    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-thc65\""
Jul 20 16:29:38.155347 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.100953    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hx9p"
Jul 20 16:29:38.155347 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.101021    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 20 16:29:38.156931 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:38.101019    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hx9p" podUID="d1c26b2e-68c2-4421-82c7-7f9e73836624"
Jul 20 16:29:38.156931 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.111881    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-85qgp"
Jul 20 16:29:38.156931 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.120309    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-85qgp"
Jul 20 16:29:38.294521 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.171639    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 20 16:29:38.294521 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.177605    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/650ecc59-8e80-424e-b8fd-1e285a0cda1a-tmp-dir\") pod \"node-resolver-h568x\" (UID: \"650ecc59-8e80-424e-b8fd-1e285a0cda1a\") " pod="openshift-dns/node-resolver-h568x"
Jul 20 16:29:38.294521 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.177629    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b43abd8a-afc1-4551-a69c-94b38eaa6f9b-os-release\") pod \"multus-additional-cni-plugins-gsdf2\" (UID: \"b43abd8a-afc1-4551-a69c-94b38eaa6f9b\") " pod="openshift-multus/multus-additional-cni-plugins-gsdf2"
Jul 20 16:29:38.294521 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.177648    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-etc-openvswitch\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.294521 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.177663    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-log-socket\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.294521 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.177679    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-os-release\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.294521 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.177724    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ed50e3c6-a333-426a-89d8-56f99d082d59-socket-dir\") pod \"aws-ebs-csi-driver-node-fgsds\" (UID: \"ed50e3c6-a333-426a-89d8-56f99d082d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgsds"
Jul 20 16:29:38.294521 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.177753    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ed50e3c6-a333-426a-89d8-56f99d082d59-registration-dir\") pod \"aws-ebs-csi-driver-node-fgsds\" (UID: \"ed50e3c6-a333-426a-89d8-56f99d082d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgsds"
Jul 20 16:29:38.294521 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.177788    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2e27d15-4856-4628-bd38-eb55b3ae21c3-tmp\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.294521 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.177816    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-var-lib-openvswitch\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.294521 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.177817    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ed50e3c6-a333-426a-89d8-56f99d082d59-registration-dir\") pod \"aws-ebs-csi-driver-node-fgsds\" (UID: \"ed50e3c6-a333-426a-89d8-56f99d082d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgsds"
Jul 20 16:29:38.294521 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.177845    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ed50e3c6-a333-426a-89d8-56f99d082d59-socket-dir\") pod \"aws-ebs-csi-driver-node-fgsds\" (UID: \"ed50e3c6-a333-426a-89d8-56f99d082d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgsds"
Jul 20 16:29:38.294521 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.177847    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-host-run-ovn-kubernetes\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.294521 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.177914    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6b7e918b-2d0a-40ca-8b4e-802815b05bc8-iptables-alerter-script\") pod \"iptables-alerter-5bsgv\" (UID: \"6b7e918b-2d0a-40ca-8b4e-802815b05bc8\") " pod="openshift-network-operator/iptables-alerter-5bsgv"
Jul 20 16:29:38.295424 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.177933    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-cnibin\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.295424 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.177953    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ed50e3c6-a333-426a-89d8-56f99d082d59-etc-selinux\") pod \"aws-ebs-csi-driver-node-fgsds\" (UID: \"ed50e3c6-a333-426a-89d8-56f99d082d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgsds"
Jul 20 16:29:38.295424 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.177969    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b43abd8a-afc1-4551-a69c-94b38eaa6f9b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gsdf2\" (UID: \"b43abd8a-afc1-4551-a69c-94b38eaa6f9b\") " pod="openshift-multus/multus-additional-cni-plugins-gsdf2"
Jul 20 16:29:38.295424 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.177986    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-host-slash\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.295424 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178001    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-host-var-lib-cni-multus\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.295424 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178016    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f2e27d15-4856-4628-bd38-eb55b3ae21c3-etc-sysctl-d\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.295424 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178053    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b43abd8a-afc1-4551-a69c-94b38eaa6f9b-cnibin\") pod \"multus-additional-cni-plugins-gsdf2\" (UID: \"b43abd8a-afc1-4551-a69c-94b38eaa6f9b\") " pod="openshift-multus/multus-additional-cni-plugins-gsdf2"
Jul 20 16:29:38.295424 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178089    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-run-systemd\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.295424 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178094    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ed50e3c6-a333-426a-89d8-56f99d082d59-etc-selinux\") pod \"aws-ebs-csi-driver-node-fgsds\" (UID: \"ed50e3c6-a333-426a-89d8-56f99d082d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgsds"
Jul 20 16:29:38.295424 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178110    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f2e27d15-4856-4628-bd38-eb55b3ae21c3-etc-sysctl-d\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.295424 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178118    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-node-log\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.295424 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178128    2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 20 16:29:38.295424 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178148    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-run-openvswitch\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.295424 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178176    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-run-ovn\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.296280 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178203    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45q5c\" (UniqueName: \"kubernetes.io/projected/6b7e918b-2d0a-40ca-8b4e-802815b05bc8-kube-api-access-45q5c\") pod \"iptables-alerter-5bsgv\" (UID: \"6b7e918b-2d0a-40ca-8b4e-802815b05bc8\") " pod="openshift-network-operator/iptables-alerter-5bsgv"
Jul 20 16:29:38.296280 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178237    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1230abfc-e6e1-432a-b99e-2d700df05d26-dbus\") pod \"global-pull-secret-syncer-4957k\" (UID: \"1230abfc-e6e1-432a-b99e-2d700df05d26\") " pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:29:38.296280 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178264    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2e27d15-4856-4628-bd38-eb55b3ae21c3-sys\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.296280 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178287    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f2e27d15-4856-4628-bd38-eb55b3ae21c3-host\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.296280 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178325    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbtl6\" (UniqueName: \"kubernetes.io/projected/650ecc59-8e80-424e-b8fd-1e285a0cda1a-kube-api-access-rbtl6\") pod \"node-resolver-h568x\" (UID: \"650ecc59-8e80-424e-b8fd-1e285a0cda1a\") " pod="openshift-dns/node-resolver-h568x"
Jul 20 16:29:38.296280 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178357    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2e27d15-4856-4628-bd38-eb55b3ae21c3-sys\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.296280 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178366    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f2e27d15-4856-4628-bd38-eb55b3ae21c3-host\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.296280 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178387    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b43abd8a-afc1-4551-a69c-94b38eaa6f9b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gsdf2\" (UID: \"b43abd8a-afc1-4551-a69c-94b38eaa6f9b\") " pod="openshift-multus/multus-additional-cni-plugins-gsdf2"
Jul 20 16:29:38.296280 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178404    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-env-overrides\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.296280 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178420    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-host-kubelet\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.296280 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178437    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1c26b2e-68c2-4421-82c7-7f9e73836624-metrics-certs\") pod \"network-metrics-daemon-2hx9p\" (UID: \"d1c26b2e-68c2-4421-82c7-7f9e73836624\") " pod="openshift-multus/network-metrics-daemon-2hx9p"
Jul 20 16:29:38.296280 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178452    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dc7c0532-d1ad-4a0d-b63b-5a952a7cae21-agent-certs\") pod \"konnectivity-agent-82fz4\" (UID: \"dc7c0532-d1ad-4a0d-b63b-5a952a7cae21\") " pod="kube-system/konnectivity-agent-82fz4"
Jul 20 16:29:38.296280 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178467    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dc7c0532-d1ad-4a0d-b63b-5a952a7cae21-konnectivity-ca\") pod \"konnectivity-agent-82fz4\" (UID: \"dc7c0532-d1ad-4a0d-b63b-5a952a7cae21\") " pod="kube-system/konnectivity-agent-82fz4"
Jul 20 16:29:38.296280 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178483    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-hostroot\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.296948 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178500    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f2e27d15-4856-4628-bd38-eb55b3ae21c3-etc-sysconfig\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.296948 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178531    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ed50e3c6-a333-426a-89d8-56f99d082d59-device-dir\") pod \"aws-ebs-csi-driver-node-fgsds\" (UID: \"ed50e3c6-a333-426a-89d8-56f99d082d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgsds"
Jul 20 16:29:38.296948 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178563    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ed50e3c6-a333-426a-89d8-56f99d082d59-device-dir\") pod \"aws-ebs-csi-driver-node-fgsds\" (UID: \"ed50e3c6-a333-426a-89d8-56f99d082d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgsds"
Jul 20 16:29:38.296948 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178536    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f2e27d15-4856-4628-bd38-eb55b3ae21c3-etc-sysconfig\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.296948 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178563    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99t5v\" (UniqueName: \"kubernetes.io/projected/3cb425f5-e1bd-49d1-ab10-38f4431ebc91-kube-api-access-99t5v\") pod \"network-check-target-ss6qh\" (UID: \"3cb425f5-e1bd-49d1-ab10-38f4431ebc91\") " pod="openshift-network-diagnostics/network-check-target-ss6qh"
Jul 20 16:29:38.296948 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178603    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-host-run-netns\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.296948 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178620    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-host-cni-netd\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.296948 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178635    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-ovnkube-config\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.296948 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178651    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n87nx\" (UniqueName: \"kubernetes.io/projected/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-kube-api-access-n87nx\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.296948 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178667    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqczt\" (UniqueName: \"kubernetes.io/projected/d1c26b2e-68c2-4421-82c7-7f9e73836624-kube-api-access-qqczt\") pod \"network-metrics-daemon-2hx9p\" (UID: \"d1c26b2e-68c2-4421-82c7-7f9e73836624\") " pod="openshift-multus/network-metrics-daemon-2hx9p"
Jul 20 16:29:38.296948 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178685    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8418755-37c8-4c02-bdcd-1ac4c903c6ab-host\") pod \"node-ca-qq95v\" (UID: \"d8418755-37c8-4c02-bdcd-1ac4c903c6ab\") " pod="openshift-image-registry/node-ca-qq95v"
Jul 20 16:29:38.296948 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178715    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8418755-37c8-4c02-bdcd-1ac4c903c6ab-host\") pod \"node-ca-qq95v\" (UID: \"d8418755-37c8-4c02-bdcd-1ac4c903c6ab\") " pod="openshift-image-registry/node-ca-qq95v"
Jul 20 16:29:38.296948 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178740    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-host-run-k8s-cni-cncf-io\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.296948 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178756    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-multus-conf-dir\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.297621 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178791    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2rxg\" (UniqueName: \"kubernetes.io/projected/ed50e3c6-a333-426a-89d8-56f99d082d59-kube-api-access-l2rxg\") pod \"aws-ebs-csi-driver-node-fgsds\" (UID: \"ed50e3c6-a333-426a-89d8-56f99d082d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgsds"
Jul 20 16:29:38.297621 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178830    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d8418755-37c8-4c02-bdcd-1ac4c903c6ab-serviceca\") pod \"node-ca-qq95v\" (UID: \"d8418755-37c8-4c02-bdcd-1ac4c903c6ab\") " pod="openshift-image-registry/node-ca-qq95v"
Jul 20 16:29:38.297621 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.178846    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgg4w\" (UniqueName: \"kubernetes.io/projected/d8418755-37c8-4c02-bdcd-1ac4c903c6ab-kube-api-access-tgg4w\") pod \"node-ca-qq95v\" (UID: \"d8418755-37c8-4c02-bdcd-1ac4c903c6ab\") " pod="openshift-image-registry/node-ca-qq95v"
Jul 20 16:29:38.297621 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179008    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-ovn-node-metrics-cert\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.297621 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179100    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-system-cni-dir\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.297621 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179133    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2e27d15-4856-4628-bd38-eb55b3ae21c3-etc-kubernetes\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.297621 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179161    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b43abd8a-afc1-4551-a69c-94b38eaa6f9b-cni-binary-copy\") pod \"multus-additional-cni-plugins-gsdf2\" (UID: \"b43abd8a-afc1-4551-a69c-94b38eaa6f9b\") " pod="openshift-multus/multus-additional-cni-plugins-gsdf2"
Jul 20 16:29:38.297621 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179186    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-systemd-units\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.297621 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179211    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6b7e918b-2d0a-40ca-8b4e-802815b05bc8-host-slash\") pod \"iptables-alerter-5bsgv\" (UID: \"6b7e918b-2d0a-40ca-8b4e-802815b05bc8\") " pod="openshift-network-operator/iptables-alerter-5bsgv"
Jul 20 16:29:38.297621 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179216    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2e27d15-4856-4628-bd38-eb55b3ae21c3-etc-kubernetes\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.297621 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179255    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-multus-cni-dir\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.297621 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179282    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-host-run-netns\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.297621 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179306    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1230abfc-e6e1-432a-b99e-2d700df05d26-original-pull-secret\") pod \"global-pull-secret-syncer-4957k\" (UID: \"1230abfc-e6e1-432a-b99e-2d700df05d26\") " pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:29:38.297621 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179357    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-host-cni-bin\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.298169 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179399    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ed50e3c6-a333-426a-89d8-56f99d082d59-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fgsds\" (UID: \"ed50e3c6-a333-426a-89d8-56f99d082d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgsds"
Jul 20 16:29:38.298169 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179435    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ed50e3c6-a333-426a-89d8-56f99d082d59-sys-fs\") pod \"aws-ebs-csi-driver-node-fgsds\" (UID: \"ed50e3c6-a333-426a-89d8-56f99d082d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgsds"
Jul 20 16:29:38.298169 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179435    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d8418755-37c8-4c02-bdcd-1ac4c903c6ab-serviceca\") pod \"node-ca-qq95v\" (UID: \"d8418755-37c8-4c02-bdcd-1ac4c903c6ab\") " pod="openshift-image-registry/node-ca-qq95v"
Jul 20 16:29:38.298169 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179448    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ed50e3c6-a333-426a-89d8-56f99d082d59-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fgsds\" (UID: \"ed50e3c6-a333-426a-89d8-56f99d082d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgsds"
Jul 20 16:29:38.298169 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179490    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ed50e3c6-a333-426a-89d8-56f99d082d59-sys-fs\") pod \"aws-ebs-csi-driver-node-fgsds\" (UID: \"ed50e3c6-a333-426a-89d8-56f99d082d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgsds"
Jul 20 16:29:38.298169 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179525    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f2e27d15-4856-4628-bd38-eb55b3ae21c3-lib-modules\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.298169 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179542    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-host-var-lib-cni-bin\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.298169 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179567    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94v2x\" (UniqueName: \"kubernetes.io/projected/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-kube-api-access-94v2x\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.298169 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179592    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czm4w\" (UniqueName: \"kubernetes.io/projected/f2e27d15-4856-4628-bd38-eb55b3ae21c3-kube-api-access-czm4w\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.298169 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179643    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.298169 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179658    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f2e27d15-4856-4628-bd38-eb55b3ae21c3-lib-modules\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.298169 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179671    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-ovnkube-script-lib\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.298169 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179697    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1230abfc-e6e1-432a-b99e-2d700df05d26-kubelet-config\") pod \"global-pull-secret-syncer-4957k\" (UID: \"1230abfc-e6e1-432a-b99e-2d700df05d26\") " pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:29:38.298169 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179715    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f2e27d15-4856-4628-bd38-eb55b3ae21c3-etc-modprobe-d\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.298712 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179731    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f2e27d15-4856-4628-bd38-eb55b3ae21c3-etc-tuned\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.298712 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179763    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-multus-daemon-config\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.298712 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179793    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-etc-kubernetes\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.298712 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179807    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f2e27d15-4856-4628-bd38-eb55b3ae21c3-etc-modprobe-d\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.298712 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179831    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f2e27d15-4856-4628-bd38-eb55b3ae21c3-etc-systemd\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.298712 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179855    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f2e27d15-4856-4628-bd38-eb55b3ae21c3-run\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.298712 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179877    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b43abd8a-afc1-4551-a69c-94b38eaa6f9b-system-cni-dir\") pod \"multus-additional-cni-plugins-gsdf2\" (UID: \"b43abd8a-afc1-4551-a69c-94b38eaa6f9b\") " pod="openshift-multus/multus-additional-cni-plugins-gsdf2"
Jul 20 16:29:38.298712 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179901    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns7qn\" (UniqueName: \"kubernetes.io/projected/b43abd8a-afc1-4551-a69c-94b38eaa6f9b-kube-api-access-ns7qn\") pod \"multus-additional-cni-plugins-gsdf2\" (UID: \"b43abd8a-afc1-4551-a69c-94b38eaa6f9b\") " pod="openshift-multus/multus-additional-cni-plugins-gsdf2"
Jul 20 16:29:38.298712 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179943    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-cni-binary-copy\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.298712 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179964    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f2e27d15-4856-4628-bd38-eb55b3ae21c3-etc-systemd\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.298712 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179976    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f2e27d15-4856-4628-bd38-eb55b3ae21c3-run\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.298712 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.179980    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2e27d15-4856-4628-bd38-eb55b3ae21c3-var-lib-kubelet\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.298712 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.180014    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/650ecc59-8e80-424e-b8fd-1e285a0cda1a-hosts-file\") pod \"node-resolver-h568x\" (UID: \"650ecc59-8e80-424e-b8fd-1e285a0cda1a\") " pod="openshift-dns/node-resolver-h568x"
Jul 20 16:29:38.298712 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.180028    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2e27d15-4856-4628-bd38-eb55b3ae21c3-var-lib-kubelet\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.299399 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.180040    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f2e27d15-4856-4628-bd38-eb55b3ae21c3-etc-sysctl-conf\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.299399 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.180078    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b43abd8a-afc1-4551-a69c-94b38eaa6f9b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gsdf2\" (UID: \"b43abd8a-afc1-4551-a69c-94b38eaa6f9b\") " pod="openshift-multus/multus-additional-cni-plugins-gsdf2"
Jul 20 16:29:38.299399 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.180102    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-multus-socket-dir-parent\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.299399 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.180126    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-host-var-lib-kubelet\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.299399 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.180141    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f2e27d15-4856-4628-bd38-eb55b3ae21c3-etc-sysctl-conf\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.299399 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.180148    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-host-run-multus-certs\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.299399 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.186271    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2e27d15-4856-4628-bd38-eb55b3ae21c3-tmp\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.299399 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.186379    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f2e27d15-4856-4628-bd38-eb55b3ae21c3-etc-tuned\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.299399 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.188660    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2rxg\" (UniqueName: \"kubernetes.io/projected/ed50e3c6-a333-426a-89d8-56f99d082d59-kube-api-access-l2rxg\") pod \"aws-ebs-csi-driver-node-fgsds\" (UID: \"ed50e3c6-a333-426a-89d8-56f99d082d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgsds"
Jul 20 16:29:38.299399 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.188683    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgg4w\" (UniqueName: \"kubernetes.io/projected/d8418755-37c8-4c02-bdcd-1ac4c903c6ab-kube-api-access-tgg4w\") pod \"node-ca-qq95v\" (UID: \"d8418755-37c8-4c02-bdcd-1ac4c903c6ab\") " pod="openshift-image-registry/node-ca-qq95v"
Jul 20 16:29:38.299399 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.188700    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czm4w\" (UniqueName: \"kubernetes.io/projected/f2e27d15-4856-4628-bd38-eb55b3ae21c3-kube-api-access-czm4w\") pod \"tuned-tr8vn\" (UID: \"f2e27d15-4856-4628-bd38-eb55b3ae21c3\") " pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.299399 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.280927    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b43abd8a-afc1-4551-a69c-94b38eaa6f9b-cnibin\") pod \"multus-additional-cni-plugins-gsdf2\" (UID: \"b43abd8a-afc1-4551-a69c-94b38eaa6f9b\") " pod="openshift-multus/multus-additional-cni-plugins-gsdf2"
Jul 20 16:29:38.299399 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.280955    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-run-systemd\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.299399 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.280972    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-node-log\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.299896 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.280988    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-run-openvswitch\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.299896 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.281031    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-run-openvswitch\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.299896 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.281042    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-node-log\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.299896 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.281066    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-run-systemd\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.299896 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.281045    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b43abd8a-afc1-4551-a69c-94b38eaa6f9b-cnibin\") pod \"multus-additional-cni-plugins-gsdf2\" (UID: \"b43abd8a-afc1-4551-a69c-94b38eaa6f9b\") " pod="openshift-multus/multus-additional-cni-plugins-gsdf2"
Jul 20 16:29:38.299896 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.281091    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-run-ovn\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.299896 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.281115    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45q5c\" (UniqueName: \"kubernetes.io/projected/6b7e918b-2d0a-40ca-8b4e-802815b05bc8-kube-api-access-45q5c\") pod \"iptables-alerter-5bsgv\" (UID: \"6b7e918b-2d0a-40ca-8b4e-802815b05bc8\") " pod="openshift-network-operator/iptables-alerter-5bsgv"
Jul 20 16:29:38.299896 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.281134    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1230abfc-e6e1-432a-b99e-2d700df05d26-dbus\") pod \"global-pull-secret-syncer-4957k\" (UID: \"1230abfc-e6e1-432a-b99e-2d700df05d26\") " pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:29:38.299896 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.281137    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-run-ovn\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.299896 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.281154    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbtl6\" (UniqueName: \"kubernetes.io/projected/650ecc59-8e80-424e-b8fd-1e285a0cda1a-kube-api-access-rbtl6\") pod \"node-resolver-h568x\" (UID: \"650ecc59-8e80-424e-b8fd-1e285a0cda1a\") " pod="openshift-dns/node-resolver-h568x"
Jul 20 16:29:38.299896 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.281232    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b43abd8a-afc1-4551-a69c-94b38eaa6f9b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gsdf2\" (UID: \"b43abd8a-afc1-4551-a69c-94b38eaa6f9b\") " pod="openshift-multus/multus-additional-cni-plugins-gsdf2"
Jul 20 16:29:38.299896 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.281254    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-env-overrides\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.299896 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.281286    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-host-kubelet\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.299896 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.281280    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1230abfc-e6e1-432a-b99e-2d700df05d26-dbus\") pod \"global-pull-secret-syncer-4957k\" (UID: \"1230abfc-e6e1-432a-b99e-2d700df05d26\") " pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:29:38.299896 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.281365    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-host-kubelet\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.300476 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.281392    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1c26b2e-68c2-4421-82c7-7f9e73836624-metrics-certs\") pod \"network-metrics-daemon-2hx9p\" (UID: \"d1c26b2e-68c2-4421-82c7-7f9e73836624\") " pod="openshift-multus/network-metrics-daemon-2hx9p"
Jul 20 16:29:38.300476 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.281421    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dc7c0532-d1ad-4a0d-b63b-5a952a7cae21-agent-certs\") pod \"konnectivity-agent-82fz4\" (UID: \"dc7c0532-d1ad-4a0d-b63b-5a952a7cae21\") " pod="kube-system/konnectivity-agent-82fz4"
Jul 20 16:29:38.300476 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.281496    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dc7c0532-d1ad-4a0d-b63b-5a952a7cae21-konnectivity-ca\") pod \"konnectivity-agent-82fz4\" (UID: \"dc7c0532-d1ad-4a0d-b63b-5a952a7cae21\") " pod="kube-system/konnectivity-agent-82fz4"
Jul 20 16:29:38.300476 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:38.281559    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 16:29:38.300476 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.281610    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-hostroot\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.300476 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.281560    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-hostroot\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.300476 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.282289    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99t5v\" (UniqueName: \"kubernetes.io/projected/3cb425f5-e1bd-49d1-ab10-38f4431ebc91-kube-api-access-99t5v\") pod \"network-check-target-ss6qh\" (UID: \"3cb425f5-e1bd-49d1-ab10-38f4431ebc91\") " pod="openshift-network-diagnostics/network-check-target-ss6qh"
Jul 20 16:29:38.300476 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:38.282345    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1c26b2e-68c2-4421-82c7-7f9e73836624-metrics-certs podName:d1c26b2e-68c2-4421-82c7-7f9e73836624 nodeName:}" failed. No retries permitted until 2026-07-20 16:29:38.782282279 +0000 UTC m=+2.031776300 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d1c26b2e-68c2-4421-82c7-7f9e73836624-metrics-certs") pod "network-metrics-daemon-2hx9p" (UID: "d1c26b2e-68c2-4421-82c7-7f9e73836624") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 16:29:38.300476 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.282452    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-host-run-netns\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.300476 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.282567    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-host-cni-netd\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.300476 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.282630    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-env-overrides\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.300476 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.282631    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-ovnkube-config\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.300476 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.282729    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-host-cni-netd\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.300476 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.282740    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n87nx\" (UniqueName: \"kubernetes.io/projected/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-kube-api-access-n87nx\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.300988 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.282785    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqczt\" (UniqueName: \"kubernetes.io/projected/d1c26b2e-68c2-4421-82c7-7f9e73836624-kube-api-access-qqczt\") pod \"network-metrics-daemon-2hx9p\" (UID: \"d1c26b2e-68c2-4421-82c7-7f9e73836624\") " pod="openshift-multus/network-metrics-daemon-2hx9p"
Jul 20 16:29:38.300988 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.282790    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-host-run-netns\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.300988 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.282803    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b43abd8a-afc1-4551-a69c-94b38eaa6f9b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gsdf2\" (UID: \"b43abd8a-afc1-4551-a69c-94b38eaa6f9b\") " pod="openshift-multus/multus-additional-cni-plugins-gsdf2"
Jul 20 16:29:38.300988 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.282802    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dc7c0532-d1ad-4a0d-b63b-5a952a7cae21-konnectivity-ca\") pod \"konnectivity-agent-82fz4\" (UID: \"dc7c0532-d1ad-4a0d-b63b-5a952a7cae21\") " pod="kube-system/konnectivity-agent-82fz4"
Jul 20 16:29:38.300988 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.282856    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-host-run-k8s-cni-cncf-io\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.300988 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.283115    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-host-run-k8s-cni-cncf-io\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.300988 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.283162    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-multus-conf-dir\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.300988 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.283213    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-multus-conf-dir\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.300988 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.283204    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-ovn-node-metrics-cert\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.300988 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.283269    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-system-cni-dir\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.300988 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.283307    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b43abd8a-afc1-4551-a69c-94b38eaa6f9b-cni-binary-copy\") pod \"multus-additional-cni-plugins-gsdf2\" (UID: \"b43abd8a-afc1-4551-a69c-94b38eaa6f9b\") " pod="openshift-multus/multus-additional-cni-plugins-gsdf2"
Jul 20 16:29:38.300988 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.283403    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-systemd-units\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.300988 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.283439    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6b7e918b-2d0a-40ca-8b4e-802815b05bc8-host-slash\") pod \"iptables-alerter-5bsgv\" (UID: \"6b7e918b-2d0a-40ca-8b4e-802815b05bc8\") " pod="openshift-network-operator/iptables-alerter-5bsgv"
Jul 20 16:29:38.300988 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.283470    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-multus-cni-dir\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.300988 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.283527    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-host-run-netns\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.301541 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.283565    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1230abfc-e6e1-432a-b99e-2d700df05d26-original-pull-secret\") pod \"global-pull-secret-syncer-4957k\" (UID: \"1230abfc-e6e1-432a-b99e-2d700df05d26\") " pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:29:38.301541 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.283644    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-host-cni-bin\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.301541 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.283633    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-systemd-units\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.301541 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.283672    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-multus-cni-dir\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.301541 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.283722    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6b7e918b-2d0a-40ca-8b4e-802815b05bc8-host-slash\") pod \"iptables-alerter-5bsgv\" (UID: \"6b7e918b-2d0a-40ca-8b4e-802815b05bc8\") " pod="openshift-network-operator/iptables-alerter-5bsgv"
Jul 20 16:29:38.301541 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.283736    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-system-cni-dir\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.301541 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.283635    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-ovnkube-config\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.301541 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.283739    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-host-var-lib-cni-bin\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.301541 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.283737    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-host-run-netns\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.301541 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.283794    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-host-var-lib-cni-bin\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.301541 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.283808    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94v2x\" (UniqueName: \"kubernetes.io/projected/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-kube-api-access-94v2x\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.301541 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:38.283827    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 16:29:38.301541 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.283845    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.301541 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:38.283881    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1230abfc-e6e1-432a-b99e-2d700df05d26-original-pull-secret podName:1230abfc-e6e1-432a-b99e-2d700df05d26 nodeName:}" failed. No retries permitted until 2026-07-20 16:29:38.783867072 +0000 UTC m=+2.033361071 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1230abfc-e6e1-432a-b99e-2d700df05d26-original-pull-secret") pod "global-pull-secret-syncer-4957k" (UID: "1230abfc-e6e1-432a-b99e-2d700df05d26") : object "kube-system"/"original-pull-secret" not registered
Jul 20 16:29:38.302003 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.283894    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-ovnkube-script-lib\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.302003 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.283828    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-host-cni-bin\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.302003 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.283976    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.302003 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.284182    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1230abfc-e6e1-432a-b99e-2d700df05d26-kubelet-config\") pod \"global-pull-secret-syncer-4957k\" (UID: \"1230abfc-e6e1-432a-b99e-2d700df05d26\") " pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:29:38.302003 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.284220    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-multus-daemon-config\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.302003 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.284261    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-etc-kubernetes\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.302003 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.284300    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b43abd8a-afc1-4551-a69c-94b38eaa6f9b-system-cni-dir\") pod \"multus-additional-cni-plugins-gsdf2\" (UID: \"b43abd8a-afc1-4551-a69c-94b38eaa6f9b\") " pod="openshift-multus/multus-additional-cni-plugins-gsdf2"
Jul 20 16:29:38.302003 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.284522    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns7qn\" (UniqueName: \"kubernetes.io/projected/b43abd8a-afc1-4551-a69c-94b38eaa6f9b-kube-api-access-ns7qn\") pod \"multus-additional-cni-plugins-gsdf2\" (UID: \"b43abd8a-afc1-4551-a69c-94b38eaa6f9b\") " pod="openshift-multus/multus-additional-cni-plugins-gsdf2"
Jul 20 16:29:38.302003 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.284540    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b43abd8a-afc1-4551-a69c-94b38eaa6f9b-cni-binary-copy\") pod \"multus-additional-cni-plugins-gsdf2\" (UID: \"b43abd8a-afc1-4551-a69c-94b38eaa6f9b\") " pod="openshift-multus/multus-additional-cni-plugins-gsdf2"
Jul 20 16:29:38.302003 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.284562    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-cni-binary-copy\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.302003 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.284627    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1230abfc-e6e1-432a-b99e-2d700df05d26-kubelet-config\") pod \"global-pull-secret-syncer-4957k\" (UID: \"1230abfc-e6e1-432a-b99e-2d700df05d26\") " pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:29:38.302003 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.284671    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-ovnkube-script-lib\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.302003 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.284745    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b43abd8a-afc1-4551-a69c-94b38eaa6f9b-system-cni-dir\") pod \"multus-additional-cni-plugins-gsdf2\" (UID: \"b43abd8a-afc1-4551-a69c-94b38eaa6f9b\") " pod="openshift-multus/multus-additional-cni-plugins-gsdf2"
Jul 20 16:29:38.302003 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.285110    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-etc-kubernetes\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.302562 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.285162    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/650ecc59-8e80-424e-b8fd-1e285a0cda1a-hosts-file\") pod \"node-resolver-h568x\" (UID: \"650ecc59-8e80-424e-b8fd-1e285a0cda1a\") " pod="openshift-dns/node-resolver-h568x"
Jul 20 16:29:38.302562 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.285163    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-multus-daemon-config\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.302562 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.285222    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/650ecc59-8e80-424e-b8fd-1e285a0cda1a-hosts-file\") pod \"node-resolver-h568x\" (UID: \"650ecc59-8e80-424e-b8fd-1e285a0cda1a\") " pod="openshift-dns/node-resolver-h568x"
Jul 20 16:29:38.302562 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.285264    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b43abd8a-afc1-4551-a69c-94b38eaa6f9b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gsdf2\" (UID: \"b43abd8a-afc1-4551-a69c-94b38eaa6f9b\") " pod="openshift-multus/multus-additional-cni-plugins-gsdf2"
Jul 20 16:29:38.302562 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.285308    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-multus-socket-dir-parent\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.302562 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.285336    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-cni-binary-copy\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.302562 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.285346    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-host-var-lib-kubelet\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.302562 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.285398    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-host-run-multus-certs\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.302562 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.285430    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dc7c0532-d1ad-4a0d-b63b-5a952a7cae21-agent-certs\") pod \"konnectivity-agent-82fz4\" (UID: \"dc7c0532-d1ad-4a0d-b63b-5a952a7cae21\") " pod="kube-system/konnectivity-agent-82fz4"
Jul 20 16:29:38.302562 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.285441    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-multus-socket-dir-parent\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.302562 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.285433    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/650ecc59-8e80-424e-b8fd-1e285a0cda1a-tmp-dir\") pod \"node-resolver-h568x\" (UID: \"650ecc59-8e80-424e-b8fd-1e285a0cda1a\") " pod="openshift-dns/node-resolver-h568x"
Jul 20 16:29:38.302562 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.285486    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-host-run-multus-certs\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.302562 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.285496    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b43abd8a-afc1-4551-a69c-94b38eaa6f9b-os-release\") pod \"multus-additional-cni-plugins-gsdf2\" (UID: \"b43abd8a-afc1-4551-a69c-94b38eaa6f9b\") " pod="openshift-multus/multus-additional-cni-plugins-gsdf2"
Jul 20 16:29:38.302562 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.285450    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-host-var-lib-kubelet\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.302562 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.285535    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-etc-openvswitch\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.303329 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.285573    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-log-socket\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.303329 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.285586    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b43abd8a-afc1-4551-a69c-94b38eaa6f9b-os-release\") pod \"multus-additional-cni-plugins-gsdf2\" (UID: \"b43abd8a-afc1-4551-a69c-94b38eaa6f9b\") " pod="openshift-multus/multus-additional-cni-plugins-gsdf2"
Jul 20 16:29:38.303329 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.285606    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-os-release\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.303329 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.285658    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-log-socket\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.303329 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.285681    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-os-release\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.303329 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.285694    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-var-lib-openvswitch\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.303329 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.285771    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-host-run-ovn-kubernetes\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.303329 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.285787    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b43abd8a-afc1-4551-a69c-94b38eaa6f9b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gsdf2\" (UID: \"b43abd8a-afc1-4551-a69c-94b38eaa6f9b\") " pod="openshift-multus/multus-additional-cni-plugins-gsdf2"
Jul 20 16:29:38.303329 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.285853    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6b7e918b-2d0a-40ca-8b4e-802815b05bc8-iptables-alerter-script\") pod \"iptables-alerter-5bsgv\" (UID: \"6b7e918b-2d0a-40ca-8b4e-802815b05bc8\") " pod="openshift-network-operator/iptables-alerter-5bsgv"
Jul 20 16:29:38.303329 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.285863    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/650ecc59-8e80-424e-b8fd-1e285a0cda1a-tmp-dir\") pod \"node-resolver-h568x\" (UID: \"650ecc59-8e80-424e-b8fd-1e285a0cda1a\") " pod="openshift-dns/node-resolver-h568x"
Jul 20 16:29:38.303329 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.285934    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-var-lib-openvswitch\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.303329 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.286012    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-etc-openvswitch\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.303329 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.286110    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-host-run-ovn-kubernetes\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.303329 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.286166    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-cnibin\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.303839 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.286254    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b43abd8a-afc1-4551-a69c-94b38eaa6f9b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gsdf2\" (UID: \"b43abd8a-afc1-4551-a69c-94b38eaa6f9b\") " pod="openshift-multus/multus-additional-cni-plugins-gsdf2"
Jul 20 16:29:38.303839 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.286286    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-host-slash\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.303839 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.286286    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-cnibin\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.303839 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.286349    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-host-slash\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.303839 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.286400    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-host-var-lib-cni-multus\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.303839 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.286437    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-ovn-node-metrics-cert\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.303839 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.286514    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-host-var-lib-cni-multus\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.303839 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.286538    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6b7e918b-2d0a-40ca-8b4e-802815b05bc8-iptables-alerter-script\") pod \"iptables-alerter-5bsgv\" (UID: \"6b7e918b-2d0a-40ca-8b4e-802815b05bc8\") " pod="openshift-network-operator/iptables-alerter-5bsgv"
Jul 20 16:29:38.303839 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.287243    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b43abd8a-afc1-4551-a69c-94b38eaa6f9b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gsdf2\" (UID: \"b43abd8a-afc1-4551-a69c-94b38eaa6f9b\") " pod="openshift-multus/multus-additional-cni-plugins-gsdf2"
Jul 20 16:29:38.303839 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:38.289367    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 16:29:38.303839 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:38.289405    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 16:29:38.303839 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:38.289419    2577 projected.go:196] Error preparing data for projected volume kube-api-access-99t5v for pod openshift-network-diagnostics/network-check-target-ss6qh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 16:29:38.303839 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:38.289480    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3cb425f5-e1bd-49d1-ab10-38f4431ebc91-kube-api-access-99t5v podName:3cb425f5-e1bd-49d1-ab10-38f4431ebc91 nodeName:}" failed. No retries permitted until 2026-07-20 16:29:38.789467762 +0000 UTC m=+2.038961761 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-99t5v" (UniqueName: "kubernetes.io/projected/3cb425f5-e1bd-49d1-ab10-38f4431ebc91-kube-api-access-99t5v") pod "network-check-target-ss6qh" (UID: "3cb425f5-e1bd-49d1-ab10-38f4431ebc91") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 16:29:38.303839 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.291863    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqczt\" (UniqueName: \"kubernetes.io/projected/d1c26b2e-68c2-4421-82c7-7f9e73836624-kube-api-access-qqczt\") pod \"network-metrics-daemon-2hx9p\" (UID: \"d1c26b2e-68c2-4421-82c7-7f9e73836624\") " pod="openshift-multus/network-metrics-daemon-2hx9p"
Jul 20 16:29:38.304460 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.292115    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbtl6\" (UniqueName: \"kubernetes.io/projected/650ecc59-8e80-424e-b8fd-1e285a0cda1a-kube-api-access-rbtl6\") pod \"node-resolver-h568x\" (UID: \"650ecc59-8e80-424e-b8fd-1e285a0cda1a\") " pod="openshift-dns/node-resolver-h568x"
Jul 20 16:29:38.304460 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.292371    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45q5c\" (UniqueName: \"kubernetes.io/projected/6b7e918b-2d0a-40ca-8b4e-802815b05bc8-kube-api-access-45q5c\") pod \"iptables-alerter-5bsgv\" (UID: \"6b7e918b-2d0a-40ca-8b4e-802815b05bc8\") " pod="openshift-network-operator/iptables-alerter-5bsgv"
Jul 20 16:29:38.304460 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.293609    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94v2x\" (UniqueName: \"kubernetes.io/projected/b244ceb8-ea1a-401f-bf8b-5a85ed236b21-kube-api-access-94v2x\") pod \"multus-72dfr\" (UID: \"b244ceb8-ea1a-401f-bf8b-5a85ed236b21\") " pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.304460 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.293799    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n87nx\" (UniqueName: \"kubernetes.io/projected/fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c-kube-api-access-n87nx\") pod \"ovnkube-node-gd4p4\" (UID: \"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c\") " pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.304460 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.294205    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns7qn\" (UniqueName: \"kubernetes.io/projected/b43abd8a-afc1-4551-a69c-94b38eaa6f9b-kube-api-access-ns7qn\") pod \"multus-additional-cni-plugins-gsdf2\" (UID: \"b43abd8a-afc1-4551-a69c-94b38eaa6f9b\") " pod="openshift-multus/multus-additional-cni-plugins-gsdf2"
Jul 20 16:29:38.484488 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.335758    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 16:29:38.484488 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:38.365682    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5c2cca67b804c078f4c05792f7283b5.slice/crio-039f4c7df6942838bdb1b8b7a07e364db1c5386a7d9a9573360607dc86ae4d5a WatchSource:0}: Error finding container 039f4c7df6942838bdb1b8b7a07e364db1c5386a7d9a9573360607dc86ae4d5a: Status 404 returned error can't find the container with id 039f4c7df6942838bdb1b8b7a07e364db1c5386a7d9a9573360607dc86ae4d5a
Jul 20 16:29:38.484488 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:38.369649    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod786ae2d571f496bce3302e9386a601e9.slice/crio-29c11034740013788716eb16d8e45e990f813a431d04c6df7f193f51a0ad1939 WatchSource:0}: Error finding container 29c11034740013788716eb16d8e45e990f813a431d04c6df7f193f51a0ad1939: Status 404 returned error can't find the container with id 29c11034740013788716eb16d8e45e990f813a431d04c6df7f193f51a0ad1939
Jul 20 16:29:38.484488 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.370395    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 16:29:38.484488 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.389434    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgsds"
Jul 20 16:29:38.484488 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:38.398773    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded50e3c6_a333_426a_89d8_56f99d082d59.slice/crio-1951424c0223932b7254646c332a195514718672e48be1d13b5cd2f77eb0ed84 WatchSource:0}: Error finding container 1951424c0223932b7254646c332a195514718672e48be1d13b5cd2f77eb0ed84: Status 404 returned error can't find the container with id 1951424c0223932b7254646c332a195514718672e48be1d13b5cd2f77eb0ed84
Jul 20 16:29:38.484488 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.423133    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tr8vn"
Jul 20 16:29:38.484488 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:38.433255    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2e27d15_4856_4628_bd38_eb55b3ae21c3.slice/crio-83a5c99a8694025596e7a4294e8df4387a93f9567ffb93b5a42fdbf5a802ab9d WatchSource:0}: Error finding container 83a5c99a8694025596e7a4294e8df4387a93f9567ffb93b5a42fdbf5a802ab9d: Status 404 returned error can't find the container with id 83a5c99a8694025596e7a4294e8df4387a93f9567ffb93b5a42fdbf5a802ab9d
Jul 20 16:29:38.484488 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.452764    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qq95v"
Jul 20 16:29:38.484488 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.459857    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-82fz4"
Jul 20 16:29:38.484488 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:38.461033    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8418755_37c8_4c02_bdcd_1ac4c903c6ab.slice/crio-04dff71e10ebcde93b622f5cd349f17e298af1c2e645825458db67d2464ae4c6 WatchSource:0}: Error finding container 04dff71e10ebcde93b622f5cd349f17e298af1c2e645825458db67d2464ae4c6: Status 404 returned error can't find the container with id 04dff71e10ebcde93b622f5cd349f17e298af1c2e645825458db67d2464ae4c6
Jul 20 16:29:38.484488 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:38.472785    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc7c0532_d1ad_4a0d_b63b_5a952a7cae21.slice/crio-19efee976b23cec6ca6b6b1f184e86d204b4ee4b3821772b643c707492136d79 WatchSource:0}: Error finding container 19efee976b23cec6ca6b6b1f184e86d204b4ee4b3821772b643c707492136d79: Status 404 returned error can't find the container with id 19efee976b23cec6ca6b6b1f184e86d204b4ee4b3821772b643c707492136d79
Jul 20 16:29:38.653138 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.499798    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-h568x"
Jul 20 16:29:38.653138 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:38.508318    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod650ecc59_8e80_424e_b8fd_1e285a0cda1a.slice/crio-9935da8ecc09742df03adf1e84434d98730d284f05e1f09ee4a9a9fa36df0a5d WatchSource:0}: Error finding container 9935da8ecc09742df03adf1e84434d98730d284f05e1f09ee4a9a9fa36df0a5d: Status 404 returned error can't find the container with id 9935da8ecc09742df03adf1e84434d98730d284f05e1f09ee4a9a9fa36df0a5d
Jul 20 16:29:38.653138 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.525168    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5bsgv"
Jul 20 16:29:38.653138 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:38.534671    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b7e918b_2d0a_40ca_8b4e_802815b05bc8.slice/crio-ccfa6a2f7c42c9dac62517407b6aeaf80cf704f4bec9c46993e76ee8d00b2e1b WatchSource:0}: Error finding container ccfa6a2f7c42c9dac62517407b6aeaf80cf704f4bec9c46993e76ee8d00b2e1b: Status 404 returned error can't find the container with id ccfa6a2f7c42c9dac62517407b6aeaf80cf704f4bec9c46993e76ee8d00b2e1b
Jul 20 16:29:38.653138 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.543773    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-72dfr"
Jul 20 16:29:38.653138 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.565268    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gsdf2"
Jul 20 16:29:38.653138 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.571479    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:29:38.653138 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:38.577888    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb43abd8a_afc1_4551_a69c_94b38eaa6f9b.slice/crio-ad5b084a1a6d65a43b70decaf6cafea978cda98615e4b4b0b68509c016d2cf85 WatchSource:0}: Error finding container ad5b084a1a6d65a43b70decaf6cafea978cda98615e4b4b0b68509c016d2cf85: Status 404 returned error can't find the container with id ad5b084a1a6d65a43b70decaf6cafea978cda98615e4b4b0b68509c016d2cf85
Jul 20 16:29:38.653138 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:29:38.582080    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc3cc1d5_9e3b_43d8_9f9d_7e80dfc0f12c.slice/crio-d197160f1bac73325602bf8d617bf5008eb3e90a049523699f7a9682a26ca3da WatchSource:0}: Error finding container d197160f1bac73325602bf8d617bf5008eb3e90a049523699f7a9682a26ca3da: Status 404 returned error can't find the container with id d197160f1bac73325602bf8d617bf5008eb3e90a049523699f7a9682a26ca3da
Jul 20 16:29:38.820888 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.793515    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1c26b2e-68c2-4421-82c7-7f9e73836624-metrics-certs\") pod \"network-metrics-daemon-2hx9p\" (UID: \"d1c26b2e-68c2-4421-82c7-7f9e73836624\") " pod="openshift-multus/network-metrics-daemon-2hx9p"
Jul 20 16:29:38.820888 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.793557    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99t5v\" (UniqueName: \"kubernetes.io/projected/3cb425f5-e1bd-49d1-ab10-38f4431ebc91-kube-api-access-99t5v\") pod \"network-check-target-ss6qh\" (UID: \"3cb425f5-e1bd-49d1-ab10-38f4431ebc91\") " pod="openshift-network-diagnostics/network-check-target-ss6qh"
Jul 20 16:29:38.820888 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:38.793594    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1230abfc-e6e1-432a-b99e-2d700df05d26-original-pull-secret\") pod \"global-pull-secret-syncer-4957k\" (UID: \"1230abfc-e6e1-432a-b99e-2d700df05d26\") " pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:29:38.820888 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:38.793690    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 16:29:38.820888 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:38.793750    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 16:29:38.820888 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:38.793778    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 16:29:38.820888 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:38.793774    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1230abfc-e6e1-432a-b99e-2d700df05d26-original-pull-secret podName:1230abfc-e6e1-432a-b99e-2d700df05d26 nodeName:}" failed. No retries permitted until 2026-07-20 16:29:39.793755998 +0000 UTC m=+3.043249998 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1230abfc-e6e1-432a-b99e-2d700df05d26-original-pull-secret") pod "global-pull-secret-syncer-4957k" (UID: "1230abfc-e6e1-432a-b99e-2d700df05d26") : object "kube-system"/"original-pull-secret" not registered
Jul 20 16:29:38.820888 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:38.793691    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 16:29:38.820888 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:38.793851    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1c26b2e-68c2-4421-82c7-7f9e73836624-metrics-certs podName:d1c26b2e-68c2-4421-82c7-7f9e73836624 nodeName:}" failed. No retries permitted until 2026-07-20 16:29:39.793834411 +0000 UTC m=+3.043328413 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d1c26b2e-68c2-4421-82c7-7f9e73836624-metrics-certs") pod "network-metrics-daemon-2hx9p" (UID: "d1c26b2e-68c2-4421-82c7-7f9e73836624") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 16:29:38.820888 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:38.793792    2577 projected.go:196] Error preparing data for projected volume kube-api-access-99t5v for pod openshift-network-diagnostics/network-check-target-ss6qh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 16:29:38.820888 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:38.793904    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3cb425f5-e1bd-49d1-ab10-38f4431ebc91-kube-api-access-99t5v podName:3cb425f5-e1bd-49d1-ab10-38f4431ebc91 nodeName:}" failed. No retries permitted until 2026-07-20 16:29:39.793892732 +0000 UTC m=+3.043386732 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-99t5v" (UniqueName: "kubernetes.io/projected/3cb425f5-e1bd-49d1-ab10-38f4431ebc91-kube-api-access-99t5v") pod "network-check-target-ss6qh" (UID: "3cb425f5-e1bd-49d1-ab10-38f4431ebc91") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 16:29:39.072907 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:39.011529    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 16:29:39.156160 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:39.136450    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 16:24:38 +0000 UTC" deadline="2028-01-04 09:26:12.271428479 +0000 UTC"
Jul 20 16:29:39.156160 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:39.136515    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12784h56m33.134918643s"
Jul 20 16:29:39.274066 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:39.206630    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ss6qh"
Jul 20 16:29:39.274066 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:39.206749    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ss6qh" podUID="3cb425f5-e1bd-49d1-ab10-38f4431ebc91"
Jul 20 16:29:39.274066 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:39.244683    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gsdf2" event={"ID":"b43abd8a-afc1-4551-a69c-94b38eaa6f9b","Type":"ContainerStarted","Data":"ad5b084a1a6d65a43b70decaf6cafea978cda98615e4b4b0b68509c016d2cf85"}
Jul 20 16:29:39.274066 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:39.246612    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 16:29:39.274066 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:39.248396    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-h568x" event={"ID":"650ecc59-8e80-424e-b8fd-1e285a0cda1a","Type":"ContainerStarted","Data":"9935da8ecc09742df03adf1e84434d98730d284f05e1f09ee4a9a9fa36df0a5d"}
Jul 20 16:29:39.274066 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:39.252546    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-82fz4" event={"ID":"dc7c0532-d1ad-4a0d-b63b-5a952a7cae21","Type":"ContainerStarted","Data":"19efee976b23cec6ca6b6b1f184e86d204b4ee4b3821772b643c707492136d79"}
Jul 20 16:29:39.274066 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:39.258507    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qq95v" event={"ID":"d8418755-37c8-4c02-bdcd-1ac4c903c6ab","Type":"ContainerStarted","Data":"04dff71e10ebcde93b622f5cd349f17e298af1c2e645825458db67d2464ae4c6"}
Jul 20 16:29:39.274066 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:39.262206    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tr8vn" event={"ID":"f2e27d15-4856-4628-bd38-eb55b3ae21c3","Type":"ContainerStarted","Data":"83a5c99a8694025596e7a4294e8df4387a93f9567ffb93b5a42fdbf5a802ab9d"}
Jul 20 16:29:39.274066 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:39.272860    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgsds" event={"ID":"ed50e3c6-a333-426a-89d8-56f99d082d59","Type":"ContainerStarted","Data":"1951424c0223932b7254646c332a195514718672e48be1d13b5cd2f77eb0ed84"}
Jul 20 16:29:39.433334 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:39.276391    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-85.ec2.internal" event={"ID":"786ae2d571f496bce3302e9386a601e9","Type":"ContainerStarted","Data":"29c11034740013788716eb16d8e45e990f813a431d04c6df7f193f51a0ad1939"}
Jul 20 16:29:39.433334 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:39.290285    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-85.ec2.internal" event={"ID":"b5c2cca67b804c078f4c05792f7283b5","Type":"ContainerStarted","Data":"039f4c7df6942838bdb1b8b7a07e364db1c5386a7d9a9573360607dc86ae4d5a"}
Jul 20 16:29:39.433334 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:39.296138    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4" event={"ID":"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c","Type":"ContainerStarted","Data":"d197160f1bac73325602bf8d617bf5008eb3e90a049523699f7a9682a26ca3da"}
Jul 20 16:29:39.433334 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:39.300747    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-72dfr" event={"ID":"b244ceb8-ea1a-401f-bf8b-5a85ed236b21","Type":"ContainerStarted","Data":"2d6b5b87246cabb24d8c8862d79ac351f0d929994a3ee1cdc783358b5de78c77"}
Jul 20 16:29:39.433334 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:39.334703    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5bsgv" event={"ID":"6b7e918b-2d0a-40ca-8b4e-802815b05bc8","Type":"ContainerStarted","Data":"ccfa6a2f7c42c9dac62517407b6aeaf80cf704f4bec9c46993e76ee8d00b2e1b"}
Jul 20 16:29:39.803828 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:39.803724    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1c26b2e-68c2-4421-82c7-7f9e73836624-metrics-certs\") pod \"network-metrics-daemon-2hx9p\" (UID: \"d1c26b2e-68c2-4421-82c7-7f9e73836624\") " pod="openshift-multus/network-metrics-daemon-2hx9p"
Jul 20 16:29:39.803828 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:39.803780    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99t5v\" (UniqueName: \"kubernetes.io/projected/3cb425f5-e1bd-49d1-ab10-38f4431ebc91-kube-api-access-99t5v\") pod \"network-check-target-ss6qh\" (UID: \"3cb425f5-e1bd-49d1-ab10-38f4431ebc91\") " pod="openshift-network-diagnostics/network-check-target-ss6qh"
Jul 20 16:29:39.803828 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:39.803822    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1230abfc-e6e1-432a-b99e-2d700df05d26-original-pull-secret\") pod \"global-pull-secret-syncer-4957k\" (UID: \"1230abfc-e6e1-432a-b99e-2d700df05d26\") " pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:29:39.804114 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:39.803959    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 16:29:39.804114 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:39.804022    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1230abfc-e6e1-432a-b99e-2d700df05d26-original-pull-secret podName:1230abfc-e6e1-432a-b99e-2d700df05d26 nodeName:}" failed. No retries permitted until 2026-07-20 16:29:41.804003927 +0000 UTC m=+5.053497929 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1230abfc-e6e1-432a-b99e-2d700df05d26-original-pull-secret") pod "global-pull-secret-syncer-4957k" (UID: "1230abfc-e6e1-432a-b99e-2d700df05d26") : object "kube-system"/"original-pull-secret" not registered
Jul 20 16:29:39.804682 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:39.804473    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 16:29:39.804682 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:39.804534    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1c26b2e-68c2-4421-82c7-7f9e73836624-metrics-certs podName:d1c26b2e-68c2-4421-82c7-7f9e73836624 nodeName:}" failed. No retries permitted until 2026-07-20 16:29:41.804517132 +0000 UTC m=+5.054011135 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d1c26b2e-68c2-4421-82c7-7f9e73836624-metrics-certs") pod "network-metrics-daemon-2hx9p" (UID: "d1c26b2e-68c2-4421-82c7-7f9e73836624") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 16:29:39.804682 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:39.804616    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 16:29:39.804682 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:39.804648    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 16:29:39.804682 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:39.804662    2577 projected.go:196] Error preparing data for projected volume kube-api-access-99t5v for pod openshift-network-diagnostics/network-check-target-ss6qh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 16:29:39.804978 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:39.804711    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3cb425f5-e1bd-49d1-ab10-38f4431ebc91-kube-api-access-99t5v podName:3cb425f5-e1bd-49d1-ab10-38f4431ebc91 nodeName:}" failed. No retries permitted until 2026-07-20 16:29:41.804685949 +0000 UTC m=+5.054179951 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-99t5v" (UniqueName: "kubernetes.io/projected/3cb425f5-e1bd-49d1-ab10-38f4431ebc91-kube-api-access-99t5v") pod "network-check-target-ss6qh" (UID: "3cb425f5-e1bd-49d1-ab10-38f4431ebc91") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 16:29:40.137294 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:40.137187    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 16:24:38 +0000 UTC" deadline="2028-04-13 10:18:01.860204526 +0000 UTC"
Jul 20 16:29:40.137294 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:40.137234    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15185h48m21.722976285s"
Jul 20 16:29:40.206034 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:40.205998    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hx9p"
Jul 20 16:29:40.206678 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:40.206155    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hx9p" podUID="d1c26b2e-68c2-4421-82c7-7f9e73836624"
Jul 20 16:29:40.206678 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:40.206593    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:29:40.206787 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:40.206693    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4957k" podUID="1230abfc-e6e1-432a-b99e-2d700df05d26"
Jul 20 16:29:40.287043 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:40.286930    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 16:29:41.219277 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:41.218756    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ss6qh"
Jul 20 16:29:41.219277 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:41.218897    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ss6qh" podUID="3cb425f5-e1bd-49d1-ab10-38f4431ebc91"
Jul 20 16:29:41.823762 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:41.823724    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1c26b2e-68c2-4421-82c7-7f9e73836624-metrics-certs\") pod \"network-metrics-daemon-2hx9p\" (UID: \"d1c26b2e-68c2-4421-82c7-7f9e73836624\") " pod="openshift-multus/network-metrics-daemon-2hx9p"
Jul 20 16:29:41.824035 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:41.823778    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99t5v\" (UniqueName: \"kubernetes.io/projected/3cb425f5-e1bd-49d1-ab10-38f4431ebc91-kube-api-access-99t5v\") pod \"network-check-target-ss6qh\" (UID: \"3cb425f5-e1bd-49d1-ab10-38f4431ebc91\") " pod="openshift-network-diagnostics/network-check-target-ss6qh"
Jul 20 16:29:41.824035 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:41.823818    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1230abfc-e6e1-432a-b99e-2d700df05d26-original-pull-secret\") pod \"global-pull-secret-syncer-4957k\" (UID: \"1230abfc-e6e1-432a-b99e-2d700df05d26\") " pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:29:41.824035 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:41.823970    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 16:29:41.824218 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:41.824042    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1230abfc-e6e1-432a-b99e-2d700df05d26-original-pull-secret podName:1230abfc-e6e1-432a-b99e-2d700df05d26 nodeName:}" failed. No retries permitted until 2026-07-20 16:29:45.824021926 +0000 UTC m=+9.073515928 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1230abfc-e6e1-432a-b99e-2d700df05d26-original-pull-secret") pod "global-pull-secret-syncer-4957k" (UID: "1230abfc-e6e1-432a-b99e-2d700df05d26") : object "kube-system"/"original-pull-secret" not registered
Jul 20 16:29:41.824817 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:41.824534    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 16:29:41.824817 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:41.824560    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 16:29:41.824817 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:41.824575    2577 projected.go:196] Error preparing data for projected volume kube-api-access-99t5v for pod openshift-network-diagnostics/network-check-target-ss6qh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 16:29:41.824817 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:41.824630    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3cb425f5-e1bd-49d1-ab10-38f4431ebc91-kube-api-access-99t5v podName:3cb425f5-e1bd-49d1-ab10-38f4431ebc91 nodeName:}" failed. No retries permitted until 2026-07-20 16:29:45.824611448 +0000 UTC m=+9.074105446 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-99t5v" (UniqueName: "kubernetes.io/projected/3cb425f5-e1bd-49d1-ab10-38f4431ebc91-kube-api-access-99t5v") pod "network-check-target-ss6qh" (UID: "3cb425f5-e1bd-49d1-ab10-38f4431ebc91") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 16:29:41.824817 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:41.824701    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 16:29:41.824817 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:41.824737    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1c26b2e-68c2-4421-82c7-7f9e73836624-metrics-certs podName:d1c26b2e-68c2-4421-82c7-7f9e73836624 nodeName:}" failed. No retries permitted until 2026-07-20 16:29:45.824724872 +0000 UTC m=+9.074218891 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d1c26b2e-68c2-4421-82c7-7f9e73836624-metrics-certs") pod "network-metrics-daemon-2hx9p" (UID: "d1c26b2e-68c2-4421-82c7-7f9e73836624") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 16:29:42.206731 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:42.206646    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:29:42.206898 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:42.206797    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4957k" podUID="1230abfc-e6e1-432a-b99e-2d700df05d26"
Jul 20 16:29:42.207215 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:42.207192    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hx9p"
Jul 20 16:29:42.207327 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:42.207309    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hx9p" podUID="d1c26b2e-68c2-4421-82c7-7f9e73836624"
Jul 20 16:29:43.207246 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:43.206765    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ss6qh"
Jul 20 16:29:43.207246 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:43.206898    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ss6qh" podUID="3cb425f5-e1bd-49d1-ab10-38f4431ebc91"
Jul 20 16:29:44.206033 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:44.205992    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hx9p"
Jul 20 16:29:44.206225 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:44.206149    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hx9p" podUID="d1c26b2e-68c2-4421-82c7-7f9e73836624"
Jul 20 16:29:44.206488 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:44.206384    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:29:44.206605 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:44.206493    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4957k" podUID="1230abfc-e6e1-432a-b99e-2d700df05d26"
Jul 20 16:29:45.210826 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:45.210764    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ss6qh"
Jul 20 16:29:45.211389 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:45.210896    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ss6qh" podUID="3cb425f5-e1bd-49d1-ab10-38f4431ebc91"
Jul 20 16:29:45.859666 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:45.859623    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1c26b2e-68c2-4421-82c7-7f9e73836624-metrics-certs\") pod \"network-metrics-daemon-2hx9p\" (UID: \"d1c26b2e-68c2-4421-82c7-7f9e73836624\") " pod="openshift-multus/network-metrics-daemon-2hx9p"
Jul 20 16:29:45.859666 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:45.859670    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99t5v\" (UniqueName: \"kubernetes.io/projected/3cb425f5-e1bd-49d1-ab10-38f4431ebc91-kube-api-access-99t5v\") pod \"network-check-target-ss6qh\" (UID: \"3cb425f5-e1bd-49d1-ab10-38f4431ebc91\") " pod="openshift-network-diagnostics/network-check-target-ss6qh"
Jul 20 16:29:45.859922 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:45.859709    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1230abfc-e6e1-432a-b99e-2d700df05d26-original-pull-secret\") pod \"global-pull-secret-syncer-4957k\" (UID: \"1230abfc-e6e1-432a-b99e-2d700df05d26\") " pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:29:45.859922 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:45.859795    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 16:29:45.859922 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:45.859865    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1c26b2e-68c2-4421-82c7-7f9e73836624-metrics-certs podName:d1c26b2e-68c2-4421-82c7-7f9e73836624 nodeName:}" failed. No retries permitted until 2026-07-20 16:29:53.859845584 +0000 UTC m=+17.109339588 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d1c26b2e-68c2-4421-82c7-7f9e73836624-metrics-certs") pod "network-metrics-daemon-2hx9p" (UID: "d1c26b2e-68c2-4421-82c7-7f9e73836624") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 16:29:45.859922 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:45.859872    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 16:29:45.859922 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:45.859887    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 16:29:45.859922 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:45.859899    2577 projected.go:196] Error preparing data for projected volume kube-api-access-99t5v for pod openshift-network-diagnostics/network-check-target-ss6qh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 16:29:45.860227 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:45.859935    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3cb425f5-e1bd-49d1-ab10-38f4431ebc91-kube-api-access-99t5v podName:3cb425f5-e1bd-49d1-ab10-38f4431ebc91 nodeName:}" failed. No retries permitted until 2026-07-20 16:29:53.859924798 +0000 UTC m=+17.109418796 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-99t5v" (UniqueName: "kubernetes.io/projected/3cb425f5-e1bd-49d1-ab10-38f4431ebc91-kube-api-access-99t5v") pod "network-check-target-ss6qh" (UID: "3cb425f5-e1bd-49d1-ab10-38f4431ebc91") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 16:29:45.860227 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:45.859795    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 16:29:45.860227 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:45.859971    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1230abfc-e6e1-432a-b99e-2d700df05d26-original-pull-secret podName:1230abfc-e6e1-432a-b99e-2d700df05d26 nodeName:}" failed. No retries permitted until 2026-07-20 16:29:53.85996248 +0000 UTC m=+17.109456481 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1230abfc-e6e1-432a-b99e-2d700df05d26-original-pull-secret") pod "global-pull-secret-syncer-4957k" (UID: "1230abfc-e6e1-432a-b99e-2d700df05d26") : object "kube-system"/"original-pull-secret" not registered
Jul 20 16:29:46.206846 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:46.206761    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:29:46.207012 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:46.206922    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4957k" podUID="1230abfc-e6e1-432a-b99e-2d700df05d26"
Jul 20 16:29:46.207012 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:46.206969    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hx9p"
Jul 20 16:29:46.207134 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:46.207062    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hx9p" podUID="d1c26b2e-68c2-4421-82c7-7f9e73836624"
Jul 20 16:29:47.207246 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:47.207208    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ss6qh"
Jul 20 16:29:47.207755 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:47.207357    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ss6qh" podUID="3cb425f5-e1bd-49d1-ab10-38f4431ebc91"
Jul 20 16:29:48.206718 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:48.206686    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hx9p"
Jul 20 16:29:48.206896 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:48.206838    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hx9p" podUID="d1c26b2e-68c2-4421-82c7-7f9e73836624"
Jul 20 16:29:48.207131 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:48.206702    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:29:48.207131 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:48.207096    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4957k" podUID="1230abfc-e6e1-432a-b99e-2d700df05d26"
Jul 20 16:29:49.206382 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:49.206216    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ss6qh"
Jul 20 16:29:49.206382 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:49.206369    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ss6qh" podUID="3cb425f5-e1bd-49d1-ab10-38f4431ebc91"
Jul 20 16:29:50.206493 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:50.206460    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:29:50.206904 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:50.206463    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hx9p"
Jul 20 16:29:50.206904 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:50.206586    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4957k" podUID="1230abfc-e6e1-432a-b99e-2d700df05d26"
Jul 20 16:29:50.206904 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:50.206671    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hx9p" podUID="d1c26b2e-68c2-4421-82c7-7f9e73836624"
Jul 20 16:29:51.206661 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:51.206633    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ss6qh"
Jul 20 16:29:51.207089 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:51.206746    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ss6qh" podUID="3cb425f5-e1bd-49d1-ab10-38f4431ebc91"
Jul 20 16:29:52.206376 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:52.206341    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:29:52.206549 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:52.206347    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hx9p"
Jul 20 16:29:52.206549 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:52.206484    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4957k" podUID="1230abfc-e6e1-432a-b99e-2d700df05d26"
Jul 20 16:29:52.206549 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:52.206541    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hx9p" podUID="d1c26b2e-68c2-4421-82c7-7f9e73836624"
Jul 20 16:29:53.206883 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:53.206840    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ss6qh"
Jul 20 16:29:53.207367 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:53.206978    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ss6qh" podUID="3cb425f5-e1bd-49d1-ab10-38f4431ebc91"
Jul 20 16:29:53.926074 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:53.926032    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1c26b2e-68c2-4421-82c7-7f9e73836624-metrics-certs\") pod \"network-metrics-daemon-2hx9p\" (UID: \"d1c26b2e-68c2-4421-82c7-7f9e73836624\") " pod="openshift-multus/network-metrics-daemon-2hx9p"
Jul 20 16:29:53.926323 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:53.926083    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99t5v\" (UniqueName: \"kubernetes.io/projected/3cb425f5-e1bd-49d1-ab10-38f4431ebc91-kube-api-access-99t5v\") pod \"network-check-target-ss6qh\" (UID: \"3cb425f5-e1bd-49d1-ab10-38f4431ebc91\") " pod="openshift-network-diagnostics/network-check-target-ss6qh"
Jul 20 16:29:53.926323 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:53.926198    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 16:29:53.926323 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:53.926212    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 16:29:53.926323 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:53.926230    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 16:29:53.926323 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:53.926243    2577 projected.go:196] Error preparing data for projected volume kube-api-access-99t5v for pod openshift-network-diagnostics/network-check-target-ss6qh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 16:29:53.926323 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:53.926263    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1230abfc-e6e1-432a-b99e-2d700df05d26-original-pull-secret\") pod \"global-pull-secret-syncer-4957k\" (UID: \"1230abfc-e6e1-432a-b99e-2d700df05d26\") " pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:29:53.926323 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:53.926272    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1c26b2e-68c2-4421-82c7-7f9e73836624-metrics-certs podName:d1c26b2e-68c2-4421-82c7-7f9e73836624 nodeName:}" failed. No retries permitted until 2026-07-20 16:30:09.926252563 +0000 UTC m=+33.175746577 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d1c26b2e-68c2-4421-82c7-7f9e73836624-metrics-certs") pod "network-metrics-daemon-2hx9p" (UID: "d1c26b2e-68c2-4421-82c7-7f9e73836624") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 16:29:53.926323 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:53.926328    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3cb425f5-e1bd-49d1-ab10-38f4431ebc91-kube-api-access-99t5v podName:3cb425f5-e1bd-49d1-ab10-38f4431ebc91 nodeName:}" failed. No retries permitted until 2026-07-20 16:30:09.926309448 +0000 UTC m=+33.175803448 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-99t5v" (UniqueName: "kubernetes.io/projected/3cb425f5-e1bd-49d1-ab10-38f4431ebc91-kube-api-access-99t5v") pod "network-check-target-ss6qh" (UID: "3cb425f5-e1bd-49d1-ab10-38f4431ebc91") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 16:29:53.926688 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:53.926350    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 16:29:53.926688 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:53.926393    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1230abfc-e6e1-432a-b99e-2d700df05d26-original-pull-secret podName:1230abfc-e6e1-432a-b99e-2d700df05d26 nodeName:}" failed. No retries permitted until 2026-07-20 16:30:09.926379058 +0000 UTC m=+33.175873058 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1230abfc-e6e1-432a-b99e-2d700df05d26-original-pull-secret") pod "global-pull-secret-syncer-4957k" (UID: "1230abfc-e6e1-432a-b99e-2d700df05d26") : object "kube-system"/"original-pull-secret" not registered
Jul 20 16:29:54.205988 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:54.205909    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hx9p"
Jul 20 16:29:54.206128 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:54.205909    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:29:54.206128 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:54.206058    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hx9p" podUID="d1c26b2e-68c2-4421-82c7-7f9e73836624"
Jul 20 16:29:54.206128 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:54.206102    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4957k" podUID="1230abfc-e6e1-432a-b99e-2d700df05d26"
Jul 20 16:29:55.206826 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:55.206763    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ss6qh"
Jul 20 16:29:55.207290 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:55.206901    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ss6qh" podUID="3cb425f5-e1bd-49d1-ab10-38f4431ebc91"
Jul 20 16:29:56.206258 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:56.206221    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hx9p"
Jul 20 16:29:56.206427 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:56.206228    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:29:56.206427 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:56.206363    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hx9p" podUID="d1c26b2e-68c2-4421-82c7-7f9e73836624"
Jul 20 16:29:56.206511 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:56.206447    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4957k" podUID="1230abfc-e6e1-432a-b99e-2d700df05d26"
Jul 20 16:29:57.207269 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:57.207235    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ss6qh"
Jul 20 16:29:57.207723 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:57.207348    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ss6qh" podUID="3cb425f5-e1bd-49d1-ab10-38f4431ebc91"
Jul 20 16:29:58.206664 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:58.206624    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hx9p"
Jul 20 16:29:58.206851 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:58.206675    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:29:58.206851 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:58.206749    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hx9p" podUID="d1c26b2e-68c2-4421-82c7-7f9e73836624"
Jul 20 16:29:58.206942 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:58.206844    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4957k" podUID="1230abfc-e6e1-432a-b99e-2d700df05d26"
Jul 20 16:29:59.206101 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:59.206075    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ss6qh"
Jul 20 16:29:59.206500 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:29:59.206214    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ss6qh" podUID="3cb425f5-e1bd-49d1-ab10-38f4431ebc91"
Jul 20 16:29:59.392090 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:59.391987    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tr8vn" event={"ID":"f2e27d15-4856-4628-bd38-eb55b3ae21c3","Type":"ContainerStarted","Data":"912a1ef3631ad37a21351b6d64fbba8b35aa46b870fa9a2729ea309f34513fca"}
Jul 20 16:29:59.393537 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:59.393511    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-85.ec2.internal" event={"ID":"786ae2d571f496bce3302e9386a601e9","Type":"ContainerStarted","Data":"5324ba5a9df5e8a8b33b1c5d4ba6bd05b31247d299990c8cceb3309343e6a54c"}
Jul 20 16:29:59.394831 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:59.394810    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-72dfr" event={"ID":"b244ceb8-ea1a-401f-bf8b-5a85ed236b21","Type":"ContainerStarted","Data":"9ab2a3fb5962ea735ea42405c65781ac8d5b7b1fe4dd481acd1fd3eaeb1ee342"}
Jul 20 16:29:59.409791 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:59.409749    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-tr8vn" podStartSLOduration=1.724897101 podStartE2EDuration="22.409735249s" totalImagesPullingTime="20.684838148s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:29:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:29:59.409139057 +0000 UTC m=+22.658633074" watchObservedRunningTime="2026-07-20 16:29:59.409735249 +0000 UTC m=+22.659229321"
Jul 20 16:29:59.422205 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:59.422170    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-85.ec2.internal" podStartSLOduration=21.422158172 podStartE2EDuration="21.422158172s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:29:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:29:59.421732501 +0000 UTC m=+22.671226520" watchObservedRunningTime="2026-07-20 16:29:59.422158172 +0000 UTC m=+22.671652191"
Jul 20 16:29:59.437886 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:29:59.437851    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-72dfr" podStartSLOduration=0.774498242 podStartE2EDuration="21.437839554s" totalImagesPullingTime="20.663341312s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:29:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:29:59.437617294 +0000 UTC m=+22.687111312" watchObservedRunningTime="2026-07-20 16:29:59.437839554 +0000 UTC m=+22.687333574"
Jul 20 16:30:00.206878 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:00.206699    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:30:00.207546 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:00.206957    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4957k" podUID="1230abfc-e6e1-432a-b99e-2d700df05d26"
Jul 20 16:30:00.207546 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:00.207019    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hx9p"
Jul 20 16:30:00.207546 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:00.207070    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hx9p" podUID="d1c26b2e-68c2-4421-82c7-7f9e73836624"
Jul 20 16:30:00.398162 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:00.397955    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qq95v" event={"ID":"d8418755-37c8-4c02-bdcd-1ac4c903c6ab","Type":"ContainerStarted","Data":"23ff9181d7412a5d52b506289573412dbe4713657966f7b2a960bb0ad1516a77"}
Jul 20 16:30:00.399353 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:00.399323    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgsds" event={"ID":"ed50e3c6-a333-426a-89d8-56f99d082d59","Type":"ContainerStarted","Data":"abdd1371d09741906bb9d7e5aa3c2c5c2c4502d56708139699f431f097c1c2ca"}
Jul 20 16:30:00.400528 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:00.400506    2577 generic.go:356] "Generic (PLEG): container finished" podID="b5c2cca67b804c078f4c05792f7283b5" containerID="4e4a679532254076389f47f569e2de8aeede9de9a20bf1b32b0d955e366ff4d1" exitCode=0
Jul 20 16:30:00.400627 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:00.400577    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-85.ec2.internal" event={"ID":"b5c2cca67b804c078f4c05792f7283b5","Type":"ContainerDied","Data":"4e4a679532254076389f47f569e2de8aeede9de9a20bf1b32b0d955e366ff4d1"}
Jul 20 16:30:00.403310 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:00.403283    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gd4p4_fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c/ovn-acl-logging/0.log"
Jul 20 16:30:00.403621 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:00.403594    2577 generic.go:356] "Generic (PLEG): container finished" podID="fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c" containerID="3cbc09ac7af5657a8e1e7f07be78dc0b749b41af6030563f333241794c5781b5" exitCode=1
Jul 20 16:30:00.403681 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:00.403655    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4" event={"ID":"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c","Type":"ContainerStarted","Data":"d4ab47655ff8315f59dc90479b265942f684ec8c082a8c588102272609d18847"}
Jul 20 16:30:00.403744 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:00.403728    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4" event={"ID":"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c","Type":"ContainerStarted","Data":"8f9578c14fbf6816da0e5c73ffdc1f3d10ad5024887d255c57c4bf7d931879b1"}
Jul 20 16:30:00.403798 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:00.403785    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4" event={"ID":"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c","Type":"ContainerStarted","Data":"f3e0d959e237c124cf01437bbe5841e03a4e4c34115a0e1ba6845ab17e115133"}
Jul 20 16:30:00.403851 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:00.403837    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4" event={"ID":"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c","Type":"ContainerStarted","Data":"6fa08e603bb62f5dd58084189082a59848c1b8be1f4cc470072cd0c32358970f"}
Jul 20 16:30:00.403894 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:00.403883    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4" event={"ID":"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c","Type":"ContainerDied","Data":"3cbc09ac7af5657a8e1e7f07be78dc0b749b41af6030563f333241794c5781b5"}
Jul 20 16:30:00.403940 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:00.403918    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4" event={"ID":"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c","Type":"ContainerStarted","Data":"e32d7a8a79ad33818dc3d91be4aec4bea6b47f97ec69954606b2f150c18c7a5e"}
Jul 20 16:30:00.405021 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:00.404998    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5bsgv" event={"ID":"6b7e918b-2d0a-40ca-8b4e-802815b05bc8","Type":"ContainerStarted","Data":"96f7691f997f3c2422c5756e613489a47720f490c52f6dd57b9c46ec2f6fc52b"}
Jul 20 16:30:00.406292 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:00.406273    2577 generic.go:356] "Generic (PLEG): container finished" podID="b43abd8a-afc1-4551-a69c-94b38eaa6f9b" containerID="5ed6f9026e91bb908d627cc791fa62e2f0c121c628ce31f7ff5770e0111f8f18" exitCode=0
Jul 20 16:30:00.406396 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:00.406330    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gsdf2" event={"ID":"b43abd8a-afc1-4551-a69c-94b38eaa6f9b","Type":"ContainerDied","Data":"5ed6f9026e91bb908d627cc791fa62e2f0c121c628ce31f7ff5770e0111f8f18"}
Jul 20 16:30:00.407739 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:00.407720    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-h568x" event={"ID":"650ecc59-8e80-424e-b8fd-1e285a0cda1a","Type":"ContainerStarted","Data":"27cac3d4b70f6ff6cadd8d6b78ecdcf71085d75906993835496f4cbf844bf14b"}
Jul 20 16:30:00.408956 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:00.408904    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-82fz4" event={"ID":"dc7c0532-d1ad-4a0d-b63b-5a952a7cae21","Type":"ContainerStarted","Data":"d22f9f662c7ae0f2f9106a8d08a8448d39dbcff3da8dccf67871fdf2d03dd321"}
Jul 20 16:30:00.413551 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:00.413516    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qq95v" podStartSLOduration=6.287231677 podStartE2EDuration="22.413504303s" totalImagesPullingTime="16.126272626s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:29:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:30:00.412851037 +0000 UTC m=+23.662345056" watchObservedRunningTime="2026-07-20 16:30:00.413504303 +0000 UTC m=+23.662998319"
Jul 20 16:30:00.430067 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:00.428339    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-h568x" podStartSLOduration=1.8446001619999999 podStartE2EDuration="22.428323383s" totalImagesPullingTime="20.583723221s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:29:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:30:00.427935018 +0000 UTC m=+23.677429040" watchObservedRunningTime="2026-07-20 16:30:00.428323383 +0000 UTC m=+23.677817403"
Jul 20 16:30:00.443663 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:00.443627    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-5bsgv" podStartSLOduration=1.869449514 podStartE2EDuration="22.443614119s" totalImagesPullingTime="20.574164605s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:29:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:30:00.443554313 +0000 UTC m=+23.693048332" watchObservedRunningTime="2026-07-20 16:30:00.443614119 +0000 UTC m=+23.693108138"
Jul 20 16:30:00.500818 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:00.500777    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-82fz4" podStartSLOduration=1.87855257 podStartE2EDuration="22.500760847s" totalImagesPullingTime="20.622208277s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:29:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:30:00.500417829 +0000 UTC m=+23.749911848" watchObservedRunningTime="2026-07-20 16:30:00.500760847 +0000 UTC m=+23.750254866"
Jul 20 16:30:01.206353 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:01.206317    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ss6qh"
Jul 20 16:30:01.206605 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:01.206451    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ss6qh" podUID="3cb425f5-e1bd-49d1-ab10-38f4431ebc91"
Jul 20 16:30:01.413370 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:01.413341    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-85.ec2.internal" event={"ID":"b5c2cca67b804c078f4c05792f7283b5","Type":"ContainerStarted","Data":"242741f1b53680586b756bbb5d7a89ad3f0d559cdbe8be32f07f1e6754e0dd56"}
Jul 20 16:30:01.428813 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:01.428772    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-85.ec2.internal" podStartSLOduration=23.428758111 podStartE2EDuration="23.428758111s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:29:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:30:01.428452754 +0000 UTC m=+24.677946773" watchObservedRunningTime="2026-07-20 16:30:01.428758111 +0000 UTC m=+24.678252129"
Jul 20 16:30:01.447465 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:01.447443    2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 20 16:30:01.599790 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:01.599756    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-82fz4"
Jul 20 16:30:01.600434 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:01.600413    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-82fz4"
Jul 20 16:30:02.151064 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:02.150876    2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-20T16:30:01.447460342Z","UUID":"baf2ee44-7f3f-48d2-b377-b17c4c442f90","Handler":null,"Name":"","Endpoint":""}
Jul 20 16:30:02.152667 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:02.152645    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 20 16:30:02.152788 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:02.152691    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 20 16:30:02.206137 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:02.206110    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hx9p"
Jul 20 16:30:02.206268 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:02.206242    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hx9p" podUID="d1c26b2e-68c2-4421-82c7-7f9e73836624"
Jul 20 16:30:02.206370 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:02.206285    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:30:02.206427 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:02.206374    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4957k" podUID="1230abfc-e6e1-432a-b99e-2d700df05d26"
Jul 20 16:30:02.418782 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:02.418701    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgsds" event={"ID":"ed50e3c6-a333-426a-89d8-56f99d082d59","Type":"ContainerStarted","Data":"2f1a3dfed75f50c8ae73d36be0cd66bd1273b27c95714a70a175673e59324129"}
Jul 20 16:30:02.423107 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:02.423083    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gd4p4_fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c/ovn-acl-logging/0.log"
Jul 20 16:30:02.423686 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:02.423656    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4" event={"ID":"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c","Type":"ContainerStarted","Data":"6b23f9f733ba4388ad77f2e6e870fe5017f2cd93375ddcb90a6e33c3a1a7aa03"}
Jul 20 16:30:02.424039 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:02.423999    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-82fz4"
Jul 20 16:30:02.424500 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:02.424482    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-82fz4"
Jul 20 16:30:03.206451 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:03.206390    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ss6qh"
Jul 20 16:30:03.206591 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:03.206524    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ss6qh" podUID="3cb425f5-e1bd-49d1-ab10-38f4431ebc91"
Jul 20 16:30:03.428522 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:03.428484    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgsds" event={"ID":"ed50e3c6-a333-426a-89d8-56f99d082d59","Type":"ContainerStarted","Data":"4d7030a5899569f3c8c6586b2903036da276c67af0a8c8098cbc47f037cfa57a"}
Jul 20 16:30:03.449797 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:03.449749    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgsds" podStartSLOduration=3.104783609 podStartE2EDuration="26.449728238s" totalImagesPullingTime="23.344944629s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:29:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:30:03.449376993 +0000 UTC m=+26.698871012" watchObservedRunningTime="2026-07-20 16:30:03.449728238 +0000 UTC m=+26.699222258"
Jul 20 16:30:04.206522 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:04.206485    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:30:04.206522 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:04.206529    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hx9p"
Jul 20 16:30:04.206726 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:04.206606    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4957k" podUID="1230abfc-e6e1-432a-b99e-2d700df05d26"
Jul 20 16:30:04.206766 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:04.206727    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hx9p" podUID="d1c26b2e-68c2-4421-82c7-7f9e73836624"
Jul 20 16:30:05.207062 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:05.206873    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ss6qh"
Jul 20 16:30:05.207870 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:05.207166    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ss6qh" podUID="3cb425f5-e1bd-49d1-ab10-38f4431ebc91"
Jul 20 16:30:05.434643 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:05.434607    2577 generic.go:356] "Generic (PLEG): container finished" podID="b43abd8a-afc1-4551-a69c-94b38eaa6f9b" containerID="d2c52d51b5b2ca32b5084540cff87482a8c2c7ec67ad0087d19bf49edeae4cec" exitCode=0
Jul 20 16:30:05.434838 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:05.434692    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gsdf2" event={"ID":"b43abd8a-afc1-4551-a69c-94b38eaa6f9b","Type":"ContainerDied","Data":"d2c52d51b5b2ca32b5084540cff87482a8c2c7ec67ad0087d19bf49edeae4cec"}
Jul 20 16:30:05.438013 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:05.437997    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gd4p4_fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c/ovn-acl-logging/0.log"
Jul 20 16:30:05.438335 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:05.438312    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4" event={"ID":"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c","Type":"ContainerStarted","Data":"bdeac56ae5273d0f4230f1630254654b0208e9e57c91387d9a24ec774ca1b3bb"}
Jul 20 16:30:05.438564 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:05.438548    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:30:05.438635 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:05.438587    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:30:05.438748 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:05.438733    2577 scope.go:117] "RemoveContainer" containerID="3cbc09ac7af5657a8e1e7f07be78dc0b749b41af6030563f333241794c5781b5"
Jul 20 16:30:05.454909 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:05.454890    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:30:06.206538 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:06.206510    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:30:06.206728 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:06.206618    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4957k" podUID="1230abfc-e6e1-432a-b99e-2d700df05d26"
Jul 20 16:30:06.206728 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:06.206671    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hx9p"
Jul 20 16:30:06.206815 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:06.206765    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hx9p" podUID="d1c26b2e-68c2-4421-82c7-7f9e73836624"
Jul 20 16:30:06.444480 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:06.444448    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gd4p4_fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c/ovn-acl-logging/0.log"
Jul 20 16:30:06.444966 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:06.444797    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4" event={"ID":"fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c","Type":"ContainerStarted","Data":"a5f6fac1750195c0bed7811b7e58be4f103c3f01e26f03dc612145d65ac21935"}
Jul 20 16:30:06.445152 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:06.445130    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:30:06.461675 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:06.461608    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:30:06.472799 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:06.472748    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4" podStartSLOduration=7.649865496 podStartE2EDuration="28.472728817s" totalImagesPullingTime="20.822863321s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:29:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:30:06.471407681 +0000 UTC m=+29.720901701" watchObservedRunningTime="2026-07-20 16:30:06.472728817 +0000 UTC m=+29.722222839"
Jul 20 16:30:07.026459 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:07.026390    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4957k"]
Jul 20 16:30:07.026627 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:07.026544    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:30:07.026675 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:07.026630    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4957k" podUID="1230abfc-e6e1-432a-b99e-2d700df05d26"
Jul 20 16:30:07.028475 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:07.028450    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ss6qh"]
Jul 20 16:30:07.028634 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:07.028553    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ss6qh"
Jul 20 16:30:07.028634 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:07.028628    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ss6qh" podUID="3cb425f5-e1bd-49d1-ab10-38f4431ebc91"
Jul 20 16:30:07.029128 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:07.029108    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2hx9p"]
Jul 20 16:30:07.029237 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:07.029209    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hx9p"
Jul 20 16:30:07.029355 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:07.029336    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hx9p" podUID="d1c26b2e-68c2-4421-82c7-7f9e73836624"
Jul 20 16:30:07.448578 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:07.448544    2577 generic.go:356] "Generic (PLEG): container finished" podID="b43abd8a-afc1-4551-a69c-94b38eaa6f9b" containerID="b74ed52a561521523b10b3a5f0cedf7845c0516a2328e3e5d3f5f10bbf62b932" exitCode=0
Jul 20 16:30:07.449024 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:07.448618    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gsdf2" event={"ID":"b43abd8a-afc1-4551-a69c-94b38eaa6f9b","Type":"ContainerDied","Data":"b74ed52a561521523b10b3a5f0cedf7845c0516a2328e3e5d3f5f10bbf62b932"}
Jul 20 16:30:08.206680 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:08.206649    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:30:08.206876 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:08.206770    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4957k" podUID="1230abfc-e6e1-432a-b99e-2d700df05d26"
Jul 20 16:30:09.206669 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:09.206599    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ss6qh"
Jul 20 16:30:09.207031 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:09.206697    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ss6qh" podUID="3cb425f5-e1bd-49d1-ab10-38f4431ebc91"
Jul 20 16:30:09.207031 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:09.206809    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hx9p"
Jul 20 16:30:09.207031 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:09.206931    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hx9p" podUID="d1c26b2e-68c2-4421-82c7-7f9e73836624"
Jul 20 16:30:09.455574 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:09.455539    2577 generic.go:356] "Generic (PLEG): container finished" podID="b43abd8a-afc1-4551-a69c-94b38eaa6f9b" containerID="dc86ddd576a66f0f8da60c5dddba76bfcc5d8e87b40e47b29a66525b73778342" exitCode=0
Jul 20 16:30:09.455728 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:09.455606    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gsdf2" event={"ID":"b43abd8a-afc1-4551-a69c-94b38eaa6f9b","Type":"ContainerDied","Data":"dc86ddd576a66f0f8da60c5dddba76bfcc5d8e87b40e47b29a66525b73778342"}
Jul 20 16:30:09.952549 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:09.952511    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1c26b2e-68c2-4421-82c7-7f9e73836624-metrics-certs\") pod \"network-metrics-daemon-2hx9p\" (UID: \"d1c26b2e-68c2-4421-82c7-7f9e73836624\") " pod="openshift-multus/network-metrics-daemon-2hx9p"
Jul 20 16:30:09.952549 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:09.952550    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99t5v\" (UniqueName: \"kubernetes.io/projected/3cb425f5-e1bd-49d1-ab10-38f4431ebc91-kube-api-access-99t5v\") pod \"network-check-target-ss6qh\" (UID: \"3cb425f5-e1bd-49d1-ab10-38f4431ebc91\") " pod="openshift-network-diagnostics/network-check-target-ss6qh"
Jul 20 16:30:09.952766 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:09.952599    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1230abfc-e6e1-432a-b99e-2d700df05d26-original-pull-secret\") pod \"global-pull-secret-syncer-4957k\" (UID: \"1230abfc-e6e1-432a-b99e-2d700df05d26\") " pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:30:09.952766 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:09.952658    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 16:30:09.952766 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:09.952701    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 16:30:09.952766 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:09.952725    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1c26b2e-68c2-4421-82c7-7f9e73836624-metrics-certs podName:d1c26b2e-68c2-4421-82c7-7f9e73836624 nodeName:}" failed. No retries permitted until 2026-07-20 16:30:41.952702925 +0000 UTC m=+65.202196937 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d1c26b2e-68c2-4421-82c7-7f9e73836624-metrics-certs") pod "network-metrics-daemon-2hx9p" (UID: "d1c26b2e-68c2-4421-82c7-7f9e73836624") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 16:30:09.952766 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:09.952735    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 16:30:09.952766 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:09.952756    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 16:30:09.952766 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:09.952768    2577 projected.go:196] Error preparing data for projected volume kube-api-access-99t5v for pod openshift-network-diagnostics/network-check-target-ss6qh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 16:30:09.953006 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:09.952743    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1230abfc-e6e1-432a-b99e-2d700df05d26-original-pull-secret podName:1230abfc-e6e1-432a-b99e-2d700df05d26 nodeName:}" failed. No retries permitted until 2026-07-20 16:30:41.952734122 +0000 UTC m=+65.202228122 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1230abfc-e6e1-432a-b99e-2d700df05d26-original-pull-secret") pod "global-pull-secret-syncer-4957k" (UID: "1230abfc-e6e1-432a-b99e-2d700df05d26") : object "kube-system"/"original-pull-secret" not registered
Jul 20 16:30:09.953006 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:09.952815    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3cb425f5-e1bd-49d1-ab10-38f4431ebc91-kube-api-access-99t5v podName:3cb425f5-e1bd-49d1-ab10-38f4431ebc91 nodeName:}" failed. No retries permitted until 2026-07-20 16:30:41.952801999 +0000 UTC m=+65.202296003 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-99t5v" (UniqueName: "kubernetes.io/projected/3cb425f5-e1bd-49d1-ab10-38f4431ebc91-kube-api-access-99t5v") pod "network-check-target-ss6qh" (UID: "3cb425f5-e1bd-49d1-ab10-38f4431ebc91") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 16:30:10.206775 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:10.206688    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:30:10.207173 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:10.206797    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4957k" podUID="1230abfc-e6e1-432a-b99e-2d700df05d26"
Jul 20 16:30:11.206362 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:11.206336    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ss6qh"
Jul 20 16:30:11.206557 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:11.206374    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hx9p"
Jul 20 16:30:11.206557 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:11.206441    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ss6qh" podUID="3cb425f5-e1bd-49d1-ab10-38f4431ebc91"
Jul 20 16:30:11.206673 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:11.206568    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2hx9p" podUID="d1c26b2e-68c2-4421-82c7-7f9e73836624"
Jul 20 16:30:12.011623 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.011420    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-85.ec2.internal" event="NodeReady"
Jul 20 16:30:12.011990 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.011755    2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 20 16:30:12.046404 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.046369    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-59985f8c96-m44rk"]
Jul 20 16:30:12.058943 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.058910    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jzd8m"]
Jul 20 16:30:12.059082 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.059044    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:30:12.061638 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.061520    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 20 16:30:12.061638 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.061533    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 20 16:30:12.061809 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.061707    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-lntkl\""
Jul 20 16:30:12.062747 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.062724    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 20 16:30:12.067058 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.067034    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 20 16:30:12.068959 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.068940    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-kv4gz"]
Jul 20 16:30:12.069121 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.069095    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jzd8m"
Jul 20 16:30:12.071341 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.071292    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 20 16:30:12.071449 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.071316    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 20 16:30:12.071449 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.071392    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lwj5v\""
Jul 20 16:30:12.085217 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.085197    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59985f8c96-m44rk"]
Jul 20 16:30:12.085344 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.085233    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kv4gz"]
Jul 20 16:30:12.085344 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.085248    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jzd8m"]
Jul 20 16:30:12.085442 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.085374    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kv4gz"
Jul 20 16:30:12.087829 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.087808    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 20 16:30:12.087829 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.087824    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 20 16:30:12.087973 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.087835    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-djkfh\""
Jul 20 16:30:12.088127 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.088113    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 20 16:30:12.167394 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.167362    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/41c8f82b-bd64-40ff-a6c2-ce412321d3fa-config-volume\") pod \"dns-default-jzd8m\" (UID: \"41c8f82b-bd64-40ff-a6c2-ce412321d3fa\") " pod="openshift-dns/dns-default-jzd8m"
Jul 20 16:30:12.167578 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.167407    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/41c8f82b-bd64-40ff-a6c2-ce412321d3fa-metrics-tls\") pod \"dns-default-jzd8m\" (UID: \"41c8f82b-bd64-40ff-a6c2-ce412321d3fa\") " pod="openshift-dns/dns-default-jzd8m"
Jul 20 16:30:12.167578 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.167439    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bf2f8258-be74-4321-8a3f-5b8bcd376065-registry-certificates\") pod \"image-registry-59985f8c96-m44rk\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") " pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:30:12.167578 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.167467    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9t2j\" (UniqueName: \"kubernetes.io/projected/93c45db6-1931-4880-9920-621adf7376a9-kube-api-access-n9t2j\") pod \"ingress-canary-kv4gz\" (UID: \"93c45db6-1931-4880-9920-621adf7376a9\") " pod="openshift-ingress-canary/ingress-canary-kv4gz"
Jul 20 16:30:12.167578 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.167508    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41c8f82b-bd64-40ff-a6c2-ce412321d3fa-tmp-dir\") pod \"dns-default-jzd8m\" (UID: \"41c8f82b-bd64-40ff-a6c2-ce412321d3fa\") " pod="openshift-dns/dns-default-jzd8m"
Jul 20 16:30:12.167578 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.167541    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-registry-tls\") pod \"image-registry-59985f8c96-m44rk\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") " pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:30:12.167830 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.167582    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bf2f8258-be74-4321-8a3f-5b8bcd376065-installation-pull-secrets\") pod \"image-registry-59985f8c96-m44rk\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") " pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:30:12.167830 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.167627    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bf2f8258-be74-4321-8a3f-5b8bcd376065-ca-trust-extracted\") pod \"image-registry-59985f8c96-m44rk\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") " pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:30:12.167830 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.167649    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/93c45db6-1931-4880-9920-621adf7376a9-cert\") pod \"ingress-canary-kv4gz\" (UID: \"93c45db6-1931-4880-9920-621adf7376a9\") " pod="openshift-ingress-canary/ingress-canary-kv4gz"
Jul 20 16:30:12.167830 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.167676    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lk7cj\" (UniqueName: \"kubernetes.io/projected/41c8f82b-bd64-40ff-a6c2-ce412321d3fa-kube-api-access-lk7cj\") pod \"dns-default-jzd8m\" (UID: \"41c8f82b-bd64-40ff-a6c2-ce412321d3fa\") " pod="openshift-dns/dns-default-jzd8m"
Jul 20 16:30:12.167830 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.167751    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-bound-sa-token\") pod \"image-registry-59985f8c96-m44rk\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") " pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:30:12.167830 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.167816    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cggs\" (UniqueName: \"kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-kube-api-access-2cggs\") pod \"image-registry-59985f8c96-m44rk\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") " pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:30:12.168110 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.167860    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bf2f8258-be74-4321-8a3f-5b8bcd376065-image-registry-private-configuration\") pod \"image-registry-59985f8c96-m44rk\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") " pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:30:12.168110 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.167885    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bf2f8258-be74-4321-8a3f-5b8bcd376065-trusted-ca\") pod \"image-registry-59985f8c96-m44rk\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") " pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:30:12.206593 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.206570    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:30:12.209194 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.209173    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 20 16:30:12.268912 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.268833    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bf2f8258-be74-4321-8a3f-5b8bcd376065-ca-trust-extracted\") pod \"image-registry-59985f8c96-m44rk\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") " pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:30:12.268912 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.268873    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/93c45db6-1931-4880-9920-621adf7376a9-cert\") pod \"ingress-canary-kv4gz\" (UID: \"93c45db6-1931-4880-9920-621adf7376a9\") " pod="openshift-ingress-canary/ingress-canary-kv4gz"
Jul 20 16:30:12.268912 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.268911    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lk7cj\" (UniqueName: \"kubernetes.io/projected/41c8f82b-bd64-40ff-a6c2-ce412321d3fa-kube-api-access-lk7cj\") pod \"dns-default-jzd8m\" (UID: \"41c8f82b-bd64-40ff-a6c2-ce412321d3fa\") " pod="openshift-dns/dns-default-jzd8m"
Jul 20 16:30:12.269162 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.268946    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-bound-sa-token\") pod \"image-registry-59985f8c96-m44rk\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") " pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:30:12.269162 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.268975    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cggs\" (UniqueName: \"kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-kube-api-access-2cggs\") pod \"image-registry-59985f8c96-m44rk\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") " pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:30:12.269162 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.269000    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bf2f8258-be74-4321-8a3f-5b8bcd376065-image-registry-private-configuration\") pod \"image-registry-59985f8c96-m44rk\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") " pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:30:12.269162 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.269027    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bf2f8258-be74-4321-8a3f-5b8bcd376065-trusted-ca\") pod \"image-registry-59985f8c96-m44rk\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") " pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:30:12.269162 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.269050    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/41c8f82b-bd64-40ff-a6c2-ce412321d3fa-config-volume\") pod \"dns-default-jzd8m\" (UID: \"41c8f82b-bd64-40ff-a6c2-ce412321d3fa\") " pod="openshift-dns/dns-default-jzd8m"
Jul 20 16:30:12.269162 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.269079    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/41c8f82b-bd64-40ff-a6c2-ce412321d3fa-metrics-tls\") pod \"dns-default-jzd8m\" (UID: \"41c8f82b-bd64-40ff-a6c2-ce412321d3fa\") " pod="openshift-dns/dns-default-jzd8m"
Jul 20 16:30:12.269162 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.269103    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bf2f8258-be74-4321-8a3f-5b8bcd376065-registry-certificates\") pod \"image-registry-59985f8c96-m44rk\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") " pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:30:12.269162 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.269123    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9t2j\" (UniqueName: \"kubernetes.io/projected/93c45db6-1931-4880-9920-621adf7376a9-kube-api-access-n9t2j\") pod \"ingress-canary-kv4gz\" (UID: \"93c45db6-1931-4880-9920-621adf7376a9\") " pod="openshift-ingress-canary/ingress-canary-kv4gz"
Jul 20 16:30:12.269162 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.269147    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41c8f82b-bd64-40ff-a6c2-ce412321d3fa-tmp-dir\") pod \"dns-default-jzd8m\" (UID: \"41c8f82b-bd64-40ff-a6c2-ce412321d3fa\") " pod="openshift-dns/dns-default-jzd8m"
Jul 20 16:30:12.269562 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.269174    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-registry-tls\") pod \"image-registry-59985f8c96-m44rk\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") " pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:30:12.269562 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.269203    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bf2f8258-be74-4321-8a3f-5b8bcd376065-installation-pull-secrets\") pod \"image-registry-59985f8c96-m44rk\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") " pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:30:12.270096 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.269696    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bf2f8258-be74-4321-8a3f-5b8bcd376065-ca-trust-extracted\") pod \"image-registry-59985f8c96-m44rk\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") " pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:30:12.270096 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.270036    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bf2f8258-be74-4321-8a3f-5b8bcd376065-registry-certificates\") pod \"image-registry-59985f8c96-m44rk\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") " pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:30:12.270096 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.270084    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/41c8f82b-bd64-40ff-a6c2-ce412321d3fa-config-volume\") pod \"dns-default-jzd8m\" (UID: \"41c8f82b-bd64-40ff-a6c2-ce412321d3fa\") " pod="openshift-dns/dns-default-jzd8m"
Jul 20 16:30:12.270359 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:12.270123    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 16:30:12.270359 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:12.270176    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41c8f82b-bd64-40ff-a6c2-ce412321d3fa-metrics-tls podName:41c8f82b-bd64-40ff-a6c2-ce412321d3fa nodeName:}" failed. No retries permitted until 2026-07-20 16:30:12.770158668 +0000 UTC m=+36.019652684 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/41c8f82b-bd64-40ff-a6c2-ce412321d3fa-metrics-tls") pod "dns-default-jzd8m" (UID: "41c8f82b-bd64-40ff-a6c2-ce412321d3fa") : secret "dns-default-metrics-tls" not found
Jul 20 16:30:12.270359 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.270351    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41c8f82b-bd64-40ff-a6c2-ce412321d3fa-tmp-dir\") pod \"dns-default-jzd8m\" (UID: \"41c8f82b-bd64-40ff-a6c2-ce412321d3fa\") " pod="openshift-dns/dns-default-jzd8m"
Jul 20 16:30:12.270556 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:12.270474    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 16:30:12.270556 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:12.270488    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59985f8c96-m44rk: secret "image-registry-tls" not found
Jul 20 16:30:12.270556 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:12.270528    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-registry-tls podName:bf2f8258-be74-4321-8a3f-5b8bcd376065 nodeName:}" failed. No retries permitted until 2026-07-20 16:30:12.770513258 +0000 UTC m=+36.020007276 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-registry-tls") pod "image-registry-59985f8c96-m44rk" (UID: "bf2f8258-be74-4321-8a3f-5b8bcd376065") : secret "image-registry-tls" not found
Jul 20 16:30:12.270705 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.270583    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bf2f8258-be74-4321-8a3f-5b8bcd376065-trusted-ca\") pod \"image-registry-59985f8c96-m44rk\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") " pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:30:12.273671 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.273649    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bf2f8258-be74-4321-8a3f-5b8bcd376065-image-registry-private-configuration\") pod \"image-registry-59985f8c96-m44rk\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") " pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:30:12.273793 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.273676    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bf2f8258-be74-4321-8a3f-5b8bcd376065-installation-pull-secrets\") pod \"image-registry-59985f8c96-m44rk\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") " pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:30:12.273793 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.273694    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/93c45db6-1931-4880-9920-621adf7376a9-cert\") pod \"ingress-canary-kv4gz\" (UID: \"93c45db6-1931-4880-9920-621adf7376a9\") " pod="openshift-ingress-canary/ingress-canary-kv4gz"
Jul 20 16:30:12.277141 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.277115    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lk7cj\" (UniqueName: \"kubernetes.io/projected/41c8f82b-bd64-40ff-a6c2-ce412321d3fa-kube-api-access-lk7cj\") pod \"dns-default-jzd8m\" (UID: \"41c8f82b-bd64-40ff-a6c2-ce412321d3fa\") " pod="openshift-dns/dns-default-jzd8m"
Jul 20 16:30:12.277431 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.277408    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-bound-sa-token\") pod \"image-registry-59985f8c96-m44rk\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") " pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:30:12.277763 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.277740    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cggs\" (UniqueName: \"kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-kube-api-access-2cggs\") pod \"image-registry-59985f8c96-m44rk\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") " pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:30:12.279000 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.278981    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9t2j\" (UniqueName: \"kubernetes.io/projected/93c45db6-1931-4880-9920-621adf7376a9-kube-api-access-n9t2j\") pod \"ingress-canary-kv4gz\" (UID: \"93c45db6-1931-4880-9920-621adf7376a9\") " pod="openshift-ingress-canary/ingress-canary-kv4gz"
Jul 20 16:30:12.394576 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.394542    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kv4gz"
Jul 20 16:30:12.534560 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.534480    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kv4gz"]
Jul 20 16:30:12.772639 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.772608    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/41c8f82b-bd64-40ff-a6c2-ce412321d3fa-metrics-tls\") pod \"dns-default-jzd8m\" (UID: \"41c8f82b-bd64-40ff-a6c2-ce412321d3fa\") " pod="openshift-dns/dns-default-jzd8m"
Jul 20 16:30:12.772821 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:12.772656    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-registry-tls\") pod \"image-registry-59985f8c96-m44rk\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") " pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:30:12.772821 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:12.772787    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 16:30:12.772821 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:12.772804    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 16:30:12.772821 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:12.772820    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59985f8c96-m44rk: secret "image-registry-tls" not found
Jul 20 16:30:12.772985 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:12.772869    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41c8f82b-bd64-40ff-a6c2-ce412321d3fa-metrics-tls podName:41c8f82b-bd64-40ff-a6c2-ce412321d3fa nodeName:}" failed. No retries permitted until 2026-07-20 16:30:13.772848164 +0000 UTC m=+37.022342174 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/41c8f82b-bd64-40ff-a6c2-ce412321d3fa-metrics-tls") pod "dns-default-jzd8m" (UID: "41c8f82b-bd64-40ff-a6c2-ce412321d3fa") : secret "dns-default-metrics-tls" not found
Jul 20 16:30:12.772985 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:12.772889    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-registry-tls podName:bf2f8258-be74-4321-8a3f-5b8bcd376065 nodeName:}" failed. No retries permitted until 2026-07-20 16:30:13.772879366 +0000 UTC m=+37.022373371 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-registry-tls") pod "image-registry-59985f8c96-m44rk" (UID: "bf2f8258-be74-4321-8a3f-5b8bcd376065") : secret "image-registry-tls" not found
Jul 20 16:30:13.214917 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:13.214891    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hx9p"
Jul 20 16:30:13.215688 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:13.215059    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ss6qh"
Jul 20 16:30:13.217849 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:13.217815    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 20 16:30:13.217978 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:13.217916    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 20 16:30:13.218893 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:13.218848    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-f9blf\""
Jul 20 16:30:13.218893 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:13.218888    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 20 16:30:13.219146 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:13.218933    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-k84bh\""
Jul 20 16:30:13.781225 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:13.781184    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/41c8f82b-bd64-40ff-a6c2-ce412321d3fa-metrics-tls\") pod \"dns-default-jzd8m\" (UID: \"41c8f82b-bd64-40ff-a6c2-ce412321d3fa\") " pod="openshift-dns/dns-default-jzd8m"
Jul 20 16:30:13.781409 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:13.781244    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-registry-tls\") pod \"image-registry-59985f8c96-m44rk\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") " pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:30:13.781409 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:13.781355    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 16:30:13.781409 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:13.781364    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 16:30:13.781409 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:13.781377    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59985f8c96-m44rk: secret "image-registry-tls" not found
Jul 20 16:30:13.781546 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:13.781432    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41c8f82b-bd64-40ff-a6c2-ce412321d3fa-metrics-tls podName:41c8f82b-bd64-40ff-a6c2-ce412321d3fa nodeName:}" failed. No retries permitted until 2026-07-20 16:30:15.781412156 +0000 UTC m=+39.030906182 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/41c8f82b-bd64-40ff-a6c2-ce412321d3fa-metrics-tls") pod "dns-default-jzd8m" (UID: "41c8f82b-bd64-40ff-a6c2-ce412321d3fa") : secret "dns-default-metrics-tls" not found
Jul 20 16:30:13.781546 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:13.781456    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-registry-tls podName:bf2f8258-be74-4321-8a3f-5b8bcd376065 nodeName:}" failed. No retries permitted until 2026-07-20 16:30:15.78144516 +0000 UTC m=+39.030939172 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-registry-tls") pod "image-registry-59985f8c96-m44rk" (UID: "bf2f8258-be74-4321-8a3f-5b8bcd376065") : secret "image-registry-tls" not found
Jul 20 16:30:14.017510 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:30:14.017479    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93c45db6_1931_4880_9920_621adf7376a9.slice/crio-c09e74e763d8f8587ccf33165f5ffe7f8b49b1f95d37b4b293380e3b8b2baf71 WatchSource:0}: Error finding container c09e74e763d8f8587ccf33165f5ffe7f8b49b1f95d37b4b293380e3b8b2baf71: Status 404 returned error can't find the container with id c09e74e763d8f8587ccf33165f5ffe7f8b49b1f95d37b4b293380e3b8b2baf71
Jul 20 16:30:14.469279 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:14.468964    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kv4gz" event={"ID":"93c45db6-1931-4880-9920-621adf7376a9","Type":"ContainerStarted","Data":"c09e74e763d8f8587ccf33165f5ffe7f8b49b1f95d37b4b293380e3b8b2baf71"}
Jul 20 16:30:14.472061 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:14.472038    2577 generic.go:356] "Generic (PLEG): container finished" podID="b43abd8a-afc1-4551-a69c-94b38eaa6f9b" containerID="d48117304e72c0d18f8667eb02efc6bc552c15b124f64e072ae853ab7fbd55ab" exitCode=0
Jul 20 16:30:14.472165 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:14.472072    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gsdf2" event={"ID":"b43abd8a-afc1-4551-a69c-94b38eaa6f9b","Type":"ContainerDied","Data":"d48117304e72c0d18f8667eb02efc6bc552c15b124f64e072ae853ab7fbd55ab"}
Jul 20 16:30:15.476948 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:15.476917    2577 generic.go:356] "Generic (PLEG): container finished" podID="b43abd8a-afc1-4551-a69c-94b38eaa6f9b" containerID="aaf1209c60e2bf66ba10c06304846e8bd4dc84bc3d2901a2e98c98ceee37fa96" exitCode=0
Jul 20 16:30:15.477349 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:15.476981    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gsdf2" event={"ID":"b43abd8a-afc1-4551-a69c-94b38eaa6f9b","Type":"ContainerDied","Data":"aaf1209c60e2bf66ba10c06304846e8bd4dc84bc3d2901a2e98c98ceee37fa96"}
Jul 20 16:30:15.799222 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:15.799183    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/41c8f82b-bd64-40ff-a6c2-ce412321d3fa-metrics-tls\") pod \"dns-default-jzd8m\" (UID: \"41c8f82b-bd64-40ff-a6c2-ce412321d3fa\") " pod="openshift-dns/dns-default-jzd8m"
Jul 20 16:30:15.799432 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:15.799244    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-registry-tls\") pod \"image-registry-59985f8c96-m44rk\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") " pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:30:15.799432 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:15.799356    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 16:30:15.799432 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:15.799379    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 16:30:15.799432 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:15.799393    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59985f8c96-m44rk: secret "image-registry-tls" not found
Jul 20 16:30:15.799625 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:15.799436    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41c8f82b-bd64-40ff-a6c2-ce412321d3fa-metrics-tls podName:41c8f82b-bd64-40ff-a6c2-ce412321d3fa nodeName:}" failed. No retries permitted until 2026-07-20 16:30:19.799414514 +0000 UTC m=+43.048908526 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/41c8f82b-bd64-40ff-a6c2-ce412321d3fa-metrics-tls") pod "dns-default-jzd8m" (UID: "41c8f82b-bd64-40ff-a6c2-ce412321d3fa") : secret "dns-default-metrics-tls" not found
Jul 20 16:30:15.799625 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:15.799453    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-registry-tls podName:bf2f8258-be74-4321-8a3f-5b8bcd376065 nodeName:}" failed. No retries permitted until 2026-07-20 16:30:19.799445648 +0000 UTC m=+43.048939644 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-registry-tls") pod "image-registry-59985f8c96-m44rk" (UID: "bf2f8258-be74-4321-8a3f-5b8bcd376065") : secret "image-registry-tls" not found
Jul 20 16:30:16.480869 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:16.480831    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kv4gz" event={"ID":"93c45db6-1931-4880-9920-621adf7376a9","Type":"ContainerStarted","Data":"15d31b64a6803df9d1bac88054534cab469863987436783ec53cc22629940885"}
Jul 20 16:30:16.484473 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:16.484444    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gsdf2" event={"ID":"b43abd8a-afc1-4551-a69c-94b38eaa6f9b","Type":"ContainerStarted","Data":"b81ae8c5281a95798c36d03bad6ccad854e94632e63ddbcb60b7f7c573326d0d"}
Jul 20 16:30:16.496755 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:16.496712    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-kv4gz" podStartSLOduration=2.265852556 podStartE2EDuration="4.496699019s" totalImagesPullingTime="2.230846463s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:30:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:30:16.496038731 +0000 UTC m=+39.745532750" watchObservedRunningTime="2026-07-20 16:30:16.496699019 +0000 UTC m=+39.746193037"
Jul 20 16:30:16.517183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:16.516778    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-gsdf2" podStartSLOduration=6.102286779 podStartE2EDuration="38.516758852s" totalImagesPullingTime="32.414472073s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:29:38 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:30:16.515193437 +0000 UTC m=+39.764687455" watchObservedRunningTime="2026-07-20 16:30:16.516758852 +0000 UTC m=+39.766252872"
Jul 20 16:30:16.687322 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:16.687267    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:16.799646 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:16.799615    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:16.903655 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:16.903626    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:16.969803 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:16.969772    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:18.257308 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:18.257260    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b66cb9dc-r4g7b"]
Jul 20 16:30:18.276889 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:18.276864    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b66cb9dc-r4g7b"]
Jul 20 16:30:18.277021 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:18.276994    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b66cb9dc-r4g7b"
Jul 20 16:30:18.283745 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:18.283713    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 20 16:30:18.283863 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:18.283757    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 20 16:30:18.283863 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:18.283725    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 20 16:30:18.283863 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:18.283857    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 20 16:30:18.419755 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:18.419729    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5773a47e-8e4c-43f5-a4d0-7e145a5cc156-tmp\") pod \"klusterlet-addon-workmgr-69b66cb9dc-r4g7b\" (UID: \"5773a47e-8e4c-43f5-a4d0-7e145a5cc156\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b66cb9dc-r4g7b"
Jul 20 16:30:18.419892 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:18.419761    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5773a47e-8e4c-43f5-a4d0-7e145a5cc156-klusterlet-config\") pod \"klusterlet-addon-workmgr-69b66cb9dc-r4g7b\" (UID: \"5773a47e-8e4c-43f5-a4d0-7e145a5cc156\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b66cb9dc-r4g7b"
Jul 20 16:30:18.419892 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:18.419782    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qr9w8\" (UniqueName: \"kubernetes.io/projected/5773a47e-8e4c-43f5-a4d0-7e145a5cc156-kube-api-access-qr9w8\") pod \"klusterlet-addon-workmgr-69b66cb9dc-r4g7b\" (UID: \"5773a47e-8e4c-43f5-a4d0-7e145a5cc156\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b66cb9dc-r4g7b"
Jul 20 16:30:18.520747 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:18.520683    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5773a47e-8e4c-43f5-a4d0-7e145a5cc156-tmp\") pod \"klusterlet-addon-workmgr-69b66cb9dc-r4g7b\" (UID: \"5773a47e-8e4c-43f5-a4d0-7e145a5cc156\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b66cb9dc-r4g7b"
Jul 20 16:30:18.520747 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:18.520723    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5773a47e-8e4c-43f5-a4d0-7e145a5cc156-klusterlet-config\") pod \"klusterlet-addon-workmgr-69b66cb9dc-r4g7b\" (UID: \"5773a47e-8e4c-43f5-a4d0-7e145a5cc156\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b66cb9dc-r4g7b"
Jul 20 16:30:18.520897 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:18.520753    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qr9w8\" (UniqueName: \"kubernetes.io/projected/5773a47e-8e4c-43f5-a4d0-7e145a5cc156-kube-api-access-qr9w8\") pod \"klusterlet-addon-workmgr-69b66cb9dc-r4g7b\" (UID: \"5773a47e-8e4c-43f5-a4d0-7e145a5cc156\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b66cb9dc-r4g7b"
Jul 20 16:30:18.521104 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:18.521083    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5773a47e-8e4c-43f5-a4d0-7e145a5cc156-tmp\") pod \"klusterlet-addon-workmgr-69b66cb9dc-r4g7b\" (UID: \"5773a47e-8e4c-43f5-a4d0-7e145a5cc156\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b66cb9dc-r4g7b"
Jul 20 16:30:18.524512 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:18.524487    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5773a47e-8e4c-43f5-a4d0-7e145a5cc156-klusterlet-config\") pod \"klusterlet-addon-workmgr-69b66cb9dc-r4g7b\" (UID: \"5773a47e-8e4c-43f5-a4d0-7e145a5cc156\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b66cb9dc-r4g7b"
Jul 20 16:30:18.528233 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:18.528213    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qr9w8\" (UniqueName: \"kubernetes.io/projected/5773a47e-8e4c-43f5-a4d0-7e145a5cc156-kube-api-access-qr9w8\") pod \"klusterlet-addon-workmgr-69b66cb9dc-r4g7b\" (UID: \"5773a47e-8e4c-43f5-a4d0-7e145a5cc156\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b66cb9dc-r4g7b"
Jul 20 16:30:18.586212 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:18.586189    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b66cb9dc-r4g7b"
Jul 20 16:30:18.728107 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:18.728068    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b66cb9dc-r4g7b"]
Jul 20 16:30:18.731289 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:30:18.731261    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5773a47e_8e4c_43f5_a4d0_7e145a5cc156.slice/crio-e4d7de53d1dc16d96ffda9ad19cc850965fe8d9fe361e61ab1dde3f94ef7bcbc WatchSource:0}: Error finding container e4d7de53d1dc16d96ffda9ad19cc850965fe8d9fe361e61ab1dde3f94ef7bcbc: Status 404 returned error can't find the container with id e4d7de53d1dc16d96ffda9ad19cc850965fe8d9fe361e61ab1dde3f94ef7bcbc
Jul 20 16:30:19.492266 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:19.492234    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b66cb9dc-r4g7b" event={"ID":"5773a47e-8e4c-43f5-a4d0-7e145a5cc156","Type":"ContainerStarted","Data":"e4d7de53d1dc16d96ffda9ad19cc850965fe8d9fe361e61ab1dde3f94ef7bcbc"}
Jul 20 16:30:19.829479 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:19.829444    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-registry-tls\") pod \"image-registry-59985f8c96-m44rk\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") " pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:30:19.829632 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:19.829547    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/41c8f82b-bd64-40ff-a6c2-ce412321d3fa-metrics-tls\") pod \"dns-default-jzd8m\" (UID: \"41c8f82b-bd64-40ff-a6c2-ce412321d3fa\") " pod="openshift-dns/dns-default-jzd8m"
Jul 20 16:30:19.829632 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:19.829596    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 16:30:19.829632 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:19.829616    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59985f8c96-m44rk: secret "image-registry-tls" not found
Jul 20 16:30:19.829719 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:19.829641    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 16:30:19.829719 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:19.829671    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-registry-tls podName:bf2f8258-be74-4321-8a3f-5b8bcd376065 nodeName:}" failed. No retries permitted until 2026-07-20 16:30:27.829655431 +0000 UTC m=+51.079149428 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-registry-tls") pod "image-registry-59985f8c96-m44rk" (UID: "bf2f8258-be74-4321-8a3f-5b8bcd376065") : secret "image-registry-tls" not found
Jul 20 16:30:19.829719 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:19.829685    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41c8f82b-bd64-40ff-a6c2-ce412321d3fa-metrics-tls podName:41c8f82b-bd64-40ff-a6c2-ce412321d3fa nodeName:}" failed. No retries permitted until 2026-07-20 16:30:27.829678658 +0000 UTC m=+51.079172654 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/41c8f82b-bd64-40ff-a6c2-ce412321d3fa-metrics-tls") pod "dns-default-jzd8m" (UID: "41c8f82b-bd64-40ff-a6c2-ce412321d3fa") : secret "dns-default-metrics-tls" not found
Jul 20 16:30:21.274238 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:21.274206    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:21.334873 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:21.334836    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:21.399029 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:21.398994    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:21.805359 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:21.805324    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:21.817322 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:21.817252    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ptwkp"]
Jul 20 16:30:21.886492 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:21.886456    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ptwkp"]
Jul 20 16:30:21.886631 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:21.886571    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ptwkp"
Jul 20 16:30:21.888972 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:21.888947    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 20 16:30:21.889090 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:21.889040    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 20 16:30:21.889977 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:21.889941    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-68gfm\""
Jul 20 16:30:21.897873 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:21.897852    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:21.998981 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:21.998941    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:22.046374 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:22.046339    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mprpq\" (UniqueName: \"kubernetes.io/projected/7cd76533-b2e5-43d0-9c6f-815d43eb391f-kube-api-access-mprpq\") pod \"migrator-7cc9c7ffd7-ptwkp\" (UID: \"7cd76533-b2e5-43d0-9c6f-815d43eb391f\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ptwkp"
Jul 20 16:30:22.102041 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:22.101973    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bd59c"]
Jul 20 16:30:22.123644 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:22.123612    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bd59c"]
Jul 20 16:30:22.123757 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:22.123736    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bd59c"
Jul 20 16:30:22.126494 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:22.126467    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 20 16:30:22.126494 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:22.126478    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-w4qdt\""
Jul 20 16:30:22.126685 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:22.126537    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 20 16:30:22.126685 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:22.126537    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 20 16:30:22.126685 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:22.126611    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 20 16:30:22.133248 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:22.133223    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:22.147064 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:22.147038    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mprpq\" (UniqueName: \"kubernetes.io/projected/7cd76533-b2e5-43d0-9c6f-815d43eb391f-kube-api-access-mprpq\") pod \"migrator-7cc9c7ffd7-ptwkp\" (UID: \"7cd76533-b2e5-43d0-9c6f-815d43eb391f\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ptwkp"
Jul 20 16:30:22.156465 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:22.156446    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mprpq\" (UniqueName: \"kubernetes.io/projected/7cd76533-b2e5-43d0-9c6f-815d43eb391f-kube-api-access-mprpq\") pod \"migrator-7cc9c7ffd7-ptwkp\" (UID: \"7cd76533-b2e5-43d0-9c6f-815d43eb391f\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ptwkp"
Jul 20 16:30:22.191926 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:22.191902    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:22.197845 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:22.197824    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ptwkp"
Jul 20 16:30:22.247520 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:22.247482    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-crio-socket\") pod \"insights-runtime-extractor-bd59c\" (UID: \"3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8\") " pod="openshift-insights/insights-runtime-extractor-bd59c"
Jul 20 16:30:22.247673 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:22.247644    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8h5bg\" (UniqueName: \"kubernetes.io/projected/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-kube-api-access-8h5bg\") pod \"insights-runtime-extractor-bd59c\" (UID: \"3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8\") " pod="openshift-insights/insights-runtime-extractor-bd59c"
Jul 20 16:30:22.247728 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:22.247693    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bd59c\" (UID: \"3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8\") " pod="openshift-insights/insights-runtime-extractor-bd59c"
Jul 20 16:30:22.247777 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:22.247728    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-data-volume\") pod \"insights-runtime-extractor-bd59c\" (UID: \"3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8\") " pod="openshift-insights/insights-runtime-extractor-bd59c"
Jul 20 16:30:22.247777 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:22.247764    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bd59c\" (UID: \"3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8\") " pod="openshift-insights/insights-runtime-extractor-bd59c"
Jul 20 16:30:22.343937 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:22.343901    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ptwkp"]
Jul 20 16:30:22.346928 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:30:22.346896    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cd76533_b2e5_43d0_9c6f_815d43eb391f.slice/crio-2338f179058789297a65d360f0001fbbcaea6a4df94a621ed5cb35ef98d20454 WatchSource:0}: Error finding container 2338f179058789297a65d360f0001fbbcaea6a4df94a621ed5cb35ef98d20454: Status 404 returned error can't find the container with id 2338f179058789297a65d360f0001fbbcaea6a4df94a621ed5cb35ef98d20454
Jul 20 16:30:22.347584 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:22.347556    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:22.348139 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:22.348115    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8h5bg\" (UniqueName: \"kubernetes.io/projected/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-kube-api-access-8h5bg\") pod \"insights-runtime-extractor-bd59c\" (UID: \"3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8\") " pod="openshift-insights/insights-runtime-extractor-bd59c"
Jul 20 16:30:22.348254 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:22.348162    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bd59c\" (UID: \"3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8\") " pod="openshift-insights/insights-runtime-extractor-bd59c"
Jul 20 16:30:22.348254 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:22.348202    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-data-volume\") pod \"insights-runtime-extractor-bd59c\" (UID: \"3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8\") " pod="openshift-insights/insights-runtime-extractor-bd59c"
Jul 20 16:30:22.348254 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:22.348243    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bd59c\" (UID: \"3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8\") " pod="openshift-insights/insights-runtime-extractor-bd59c"
Jul 20 16:30:22.348490 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:22.348271    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-crio-socket\") pod \"insights-runtime-extractor-bd59c\" (UID: \"3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8\") " pod="openshift-insights/insights-runtime-extractor-bd59c"
Jul 20 16:30:22.348490 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:22.348316    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 16:30:22.348490 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:22.348380    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-kube-rbac-proxy-cm podName:3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8 nodeName:}" failed. No retries permitted until 2026-07-20 16:30:22.848360611 +0000 UTC m=+46.097854608 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bd59c" (UID: "3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8") : configmap "kube-rbac-proxy" not found
Jul 20 16:30:22.348636 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:22.348511    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-crio-socket\") pod \"insights-runtime-extractor-bd59c\" (UID: \"3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8\") " pod="openshift-insights/insights-runtime-extractor-bd59c"
Jul 20 16:30:22.348636 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:22.348575    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 16:30:22.348636 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:22.348627    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-insights-runtime-extractor-tls podName:3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8 nodeName:}" failed. No retries permitted until 2026-07-20 16:30:22.848611658 +0000 UTC m=+46.098105670 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bd59c" (UID: "3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8") : secret "insights-runtime-extractor-tls" not found
Jul 20 16:30:22.348840 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:22.348754    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-data-volume\") pod \"insights-runtime-extractor-bd59c\" (UID: \"3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8\") " pod="openshift-insights/insights-runtime-extractor-bd59c"
Jul 20 16:30:22.357324 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:22.357262    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8h5bg\" (UniqueName: \"kubernetes.io/projected/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-kube-api-access-8h5bg\") pod \"insights-runtime-extractor-bd59c\" (UID: \"3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8\") " pod="openshift-insights/insights-runtime-extractor-bd59c"
Jul 20 16:30:22.478613 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:22.478579    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:22.501578 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:22.501543    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ptwkp" event={"ID":"7cd76533-b2e5-43d0-9c6f-815d43eb391f","Type":"ContainerStarted","Data":"2338f179058789297a65d360f0001fbbcaea6a4df94a621ed5cb35ef98d20454"}
Jul 20 16:30:22.579795 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:22.579758    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:22.851760 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:22.851724    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bd59c\" (UID: \"3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8\") " pod="openshift-insights/insights-runtime-extractor-bd59c"
Jul 20 16:30:22.851963 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:22.851809    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bd59c\" (UID: \"3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8\") " pod="openshift-insights/insights-runtime-extractor-bd59c"
Jul 20 16:30:22.851963 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:22.851849    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 16:30:22.851963 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:22.851925    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-kube-rbac-proxy-cm podName:3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8 nodeName:}" failed. No retries permitted until 2026-07-20 16:30:23.851903743 +0000 UTC m=+47.101397753 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bd59c" (UID: "3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8") : configmap "kube-rbac-proxy" not found
Jul 20 16:30:22.851963 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:22.851945    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 16:30:22.852200 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:22.852002    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-insights-runtime-extractor-tls podName:3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8 nodeName:}" failed. No retries permitted until 2026-07-20 16:30:23.851989305 +0000 UTC m=+47.101483301 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bd59c" (UID: "3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8") : secret "insights-runtime-extractor-tls" not found
Jul 20 16:30:23.858353 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:23.858316    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bd59c\" (UID: \"3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8\") " pod="openshift-insights/insights-runtime-extractor-bd59c"
Jul 20 16:30:23.858815 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:23.858372    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bd59c\" (UID: \"3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8\") " pod="openshift-insights/insights-runtime-extractor-bd59c"
Jul 20 16:30:23.858815 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:23.858469    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 16:30:23.858815 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:23.858498    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 16:30:23.858815 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:23.858556    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-kube-rbac-proxy-cm podName:3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8 nodeName:}" failed. No retries permitted until 2026-07-20 16:30:25.858534321 +0000 UTC m=+49.108028322 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bd59c" (UID: "3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8") : configmap "kube-rbac-proxy" not found
Jul 20 16:30:23.858815 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:23.858578    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-insights-runtime-extractor-tls podName:3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8 nodeName:}" failed. No retries permitted until 2026-07-20 16:30:25.858568529 +0000 UTC m=+49.108062528 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bd59c" (UID: "3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8") : secret "insights-runtime-extractor-tls" not found
Jul 20 16:30:24.083529 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:24.083493    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-vlgpp"]
Jul 20 16:30:24.101507 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:24.101487    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-vlgpp"]
Jul 20 16:30:24.101621 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:24.101592    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-vlgpp"
Jul 20 16:30:24.104054 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:24.104033    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 20 16:30:24.104280 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:24.104255    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 20 16:30:24.105218 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:24.105122    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 20 16:30:24.105218 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:24.105153    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-m8bgd\""
Jul 20 16:30:24.105218 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:24.105159    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 20 16:30:24.122926 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:24.122899    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:24.160274 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:24.160250    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/579af68b-40bb-4067-aadc-d6151fcc9efb-signing-key\") pod \"service-ca-58598695f7-vlgpp\" (UID: \"579af68b-40bb-4067-aadc-d6151fcc9efb\") " pod="openshift-service-ca/service-ca-58598695f7-vlgpp"
Jul 20 16:30:24.160400 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:24.160336    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ts8kt\" (UniqueName: \"kubernetes.io/projected/579af68b-40bb-4067-aadc-d6151fcc9efb-kube-api-access-ts8kt\") pod \"service-ca-58598695f7-vlgpp\" (UID: \"579af68b-40bb-4067-aadc-d6151fcc9efb\") " pod="openshift-service-ca/service-ca-58598695f7-vlgpp"
Jul 20 16:30:24.160400 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:24.160389    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/579af68b-40bb-4067-aadc-d6151fcc9efb-signing-cabundle\") pod \"service-ca-58598695f7-vlgpp\" (UID: \"579af68b-40bb-4067-aadc-d6151fcc9efb\") " pod="openshift-service-ca/service-ca-58598695f7-vlgpp"
Jul 20 16:30:24.173427 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:24.173404    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:24.260752 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:24.260721    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/579af68b-40bb-4067-aadc-d6151fcc9efb-signing-key\") pod \"service-ca-58598695f7-vlgpp\" (UID: \"579af68b-40bb-4067-aadc-d6151fcc9efb\") " pod="openshift-service-ca/service-ca-58598695f7-vlgpp"
Jul 20 16:30:24.260910 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:24.260809    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ts8kt\" (UniqueName: \"kubernetes.io/projected/579af68b-40bb-4067-aadc-d6151fcc9efb-kube-api-access-ts8kt\") pod \"service-ca-58598695f7-vlgpp\" (UID: \"579af68b-40bb-4067-aadc-d6151fcc9efb\") " pod="openshift-service-ca/service-ca-58598695f7-vlgpp"
Jul 20 16:30:24.260910 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:24.260861    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/579af68b-40bb-4067-aadc-d6151fcc9efb-signing-cabundle\") pod \"service-ca-58598695f7-vlgpp\" (UID: \"579af68b-40bb-4067-aadc-d6151fcc9efb\") " pod="openshift-service-ca/service-ca-58598695f7-vlgpp"
Jul 20 16:30:24.261660 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:24.261635    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/579af68b-40bb-4067-aadc-d6151fcc9efb-signing-cabundle\") pod \"service-ca-58598695f7-vlgpp\" (UID: \"579af68b-40bb-4067-aadc-d6151fcc9efb\") " pod="openshift-service-ca/service-ca-58598695f7-vlgpp"
Jul 20 16:30:24.263468 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:24.263445    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/579af68b-40bb-4067-aadc-d6151fcc9efb-signing-key\") pod \"service-ca-58598695f7-vlgpp\" (UID: \"579af68b-40bb-4067-aadc-d6151fcc9efb\") " pod="openshift-service-ca/service-ca-58598695f7-vlgpp"
Jul 20 16:30:24.269176 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:24.269132    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ts8kt\" (UniqueName: \"kubernetes.io/projected/579af68b-40bb-4067-aadc-d6151fcc9efb-kube-api-access-ts8kt\") pod \"service-ca-58598695f7-vlgpp\" (UID: \"579af68b-40bb-4067-aadc-d6151fcc9efb\") " pod="openshift-service-ca/service-ca-58598695f7-vlgpp"
Jul 20 16:30:24.412419 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:24.412347    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-vlgpp"
Jul 20 16:30:24.508425 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:24.507949    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b66cb9dc-r4g7b" event={"ID":"5773a47e-8e4c-43f5-a4d0-7e145a5cc156","Type":"ContainerStarted","Data":"606794326c599d0e74054b072638510ebeba9793aaf3ecbf2d34cf061a9f56a5"}
Jul 20 16:30:24.508425 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:24.508137    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b66cb9dc-r4g7b"
Jul 20 16:30:24.510559 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:24.510508    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b66cb9dc-r4g7b"
Jul 20 16:30:24.524289 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:24.524233    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b66cb9dc-r4g7b" podStartSLOduration=1.169494868 podStartE2EDuration="6.52421961s" totalImagesPullingTime="5.354724742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:30:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:30:24.523605289 +0000 UTC m=+47.773099349" watchObservedRunningTime="2026-07-20 16:30:24.52421961 +0000 UTC m=+47.773713629"
Jul 20 16:30:24.555230 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:24.555203    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-vlgpp"]
Jul 20 16:30:24.688775 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:24.688710    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-h568x_650ecc59-8e80-424e-b8fd-1e285a0cda1a/dns-node-resolver/0.log"
Jul 20 16:30:24.769140 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:30:24.769107    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod579af68b_40bb_4067_aadc_d6151fcc9efb.slice/crio-e23b7551264236c45081699a3752e43e75d0063f7d2b5129b5e98da5c036f8e9 WatchSource:0}: Error finding container e23b7551264236c45081699a3752e43e75d0063f7d2b5129b5e98da5c036f8e9: Status 404 returned error can't find the container with id e23b7551264236c45081699a3752e43e75d0063f7d2b5129b5e98da5c036f8e9
Jul 20 16:30:25.028036 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:25.027966    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:25.512120 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:25.512082    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-vlgpp" event={"ID":"579af68b-40bb-4067-aadc-d6151fcc9efb","Type":"ContainerStarted","Data":"e23b7551264236c45081699a3752e43e75d0063f7d2b5129b5e98da5c036f8e9"}
Jul 20 16:30:25.514155 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:25.514120    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ptwkp" event={"ID":"7cd76533-b2e5-43d0-9c6f-815d43eb391f","Type":"ContainerStarted","Data":"7ad0acbc8dc2aa3b44174eb107119e94faa1b2512cd4a682652e908c0f03c38b"}
Jul 20 16:30:25.514375 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:25.514173    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ptwkp" event={"ID":"7cd76533-b2e5-43d0-9c6f-815d43eb391f","Type":"ContainerStarted","Data":"a8a88f3c71a24e6d1113b0b870b1124eb653365c9742e48e364de3aa212048c4"}
Jul 20 16:30:25.534487 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:25.534391    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ptwkp" podStartSLOduration=1.760878572 podStartE2EDuration="4.534372485s" totalImagesPullingTime="2.773493913s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:30:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:30:25.533033308 +0000 UTC m=+48.782527329" watchObservedRunningTime="2026-07-20 16:30:25.534372485 +0000 UTC m=+48.783866504"
Jul 20 16:30:25.874752 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:25.874720    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bd59c\" (UID: \"3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8\") " pod="openshift-insights/insights-runtime-extractor-bd59c"
Jul 20 16:30:25.874929 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:25.874774    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bd59c\" (UID: \"3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8\") " pod="openshift-insights/insights-runtime-extractor-bd59c"
Jul 20 16:30:25.874929 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:25.874858    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 16:30:25.874929 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:25.874878    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 16:30:25.875034 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:25.874937    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-kube-rbac-proxy-cm podName:3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8 nodeName:}" failed. No retries permitted until 2026-07-20 16:30:29.874918379 +0000 UTC m=+53.124412410 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bd59c" (UID: "3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8") : configmap "kube-rbac-proxy" not found
Jul 20 16:30:25.875034 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:25.874958    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-insights-runtime-extractor-tls podName:3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8 nodeName:}" failed. No retries permitted until 2026-07-20 16:30:29.874950321 +0000 UTC m=+53.124444317 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bd59c" (UID: "3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8") : secret "insights-runtime-extractor-tls" not found
Jul 20 16:30:25.888783 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:25.888762    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qq95v_d8418755-37c8-4c02-bdcd-1ac4c903c6ab/node-ca/0.log"
Jul 20 16:30:26.688732 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:26.688679    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kv4gz_93c45db6-1931-4880-9920-621adf7376a9/serve-healthcheck-canary/0.log"
Jul 20 16:30:27.088654 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:27.088626    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-ptwkp_7cd76533-b2e5-43d0-9c6f-815d43eb391f/migrator/0.log"
Jul 20 16:30:27.287647 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:27.287618    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-ptwkp_7cd76533-b2e5-43d0-9c6f-815d43eb391f/graceful-termination/0.log"
Jul 20 16:30:27.520490 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:27.520412    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-vlgpp" event={"ID":"579af68b-40bb-4067-aadc-d6151fcc9efb","Type":"ContainerStarted","Data":"2cbeba4788c6ff67617374d6f335c729bb8dfa80bb28c13382042be618a393f2"}
Jul 20 16:30:27.535611 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:27.535567    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-vlgpp" podStartSLOduration=1.17551191 podStartE2EDuration="3.535552854s" totalImagesPullingTime="2.360040944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:30:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:30:27.534873158 +0000 UTC m=+50.784367177" watchObservedRunningTime="2026-07-20 16:30:27.535552854 +0000 UTC m=+50.785046851"
Jul 20 16:30:27.890148 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:27.890114    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/41c8f82b-bd64-40ff-a6c2-ce412321d3fa-metrics-tls\") pod \"dns-default-jzd8m\" (UID: \"41c8f82b-bd64-40ff-a6c2-ce412321d3fa\") " pod="openshift-dns/dns-default-jzd8m"
Jul 20 16:30:27.890598 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:27.890163    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-registry-tls\") pod \"image-registry-59985f8c96-m44rk\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") " pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:30:27.890598 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:27.890285    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 16:30:27.890598 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:27.890317    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59985f8c96-m44rk: secret "image-registry-tls" not found
Jul 20 16:30:27.890598 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:27.890386    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-registry-tls podName:bf2f8258-be74-4321-8a3f-5b8bcd376065 nodeName:}" failed. No retries permitted until 2026-07-20 16:30:43.890367788 +0000 UTC m=+67.139861788 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-registry-tls") pod "image-registry-59985f8c96-m44rk" (UID: "bf2f8258-be74-4321-8a3f-5b8bcd376065") : secret "image-registry-tls" not found
Jul 20 16:30:27.890598 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:27.890386    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 16:30:27.890598 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:27.890442    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41c8f82b-bd64-40ff-a6c2-ce412321d3fa-metrics-tls podName:41c8f82b-bd64-40ff-a6c2-ce412321d3fa nodeName:}" failed. No retries permitted until 2026-07-20 16:30:43.89042679 +0000 UTC m=+67.139920787 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/41c8f82b-bd64-40ff-a6c2-ce412321d3fa-metrics-tls") pod "dns-default-jzd8m" (UID: "41c8f82b-bd64-40ff-a6c2-ce412321d3fa") : secret "dns-default-metrics-tls" not found
Jul 20 16:30:28.170043 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:28.169929    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:28.270816 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:28.270770    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:28.764544 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:28.764483    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:28.864821 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:28.864789    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:29.214959 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:29.214917    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:29.787848 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:29.787814    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:29.888043 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:29.888010    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:29.911844 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:29.911822    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bd59c\" (UID: \"3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8\") " pod="openshift-insights/insights-runtime-extractor-bd59c"
Jul 20 16:30:29.911951 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:29.911916    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bd59c\" (UID: \"3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8\") " pod="openshift-insights/insights-runtime-extractor-bd59c"
Jul 20 16:30:29.912074 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:29.911990    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 16:30:29.912074 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:29.911995    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 16:30:29.912074 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:29.912043    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-kube-rbac-proxy-cm podName:3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8 nodeName:}" failed. No retries permitted until 2026-07-20 16:30:37.9120247 +0000 UTC m=+61.161518715 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bd59c" (UID: "3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8") : configmap "kube-rbac-proxy" not found
Jul 20 16:30:29.912074 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:29.912073    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-insights-runtime-extractor-tls podName:3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8 nodeName:}" failed. No retries permitted until 2026-07-20 16:30:37.912055355 +0000 UTC m=+61.161549367 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bd59c" (UID: "3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8") : secret "insights-runtime-extractor-tls" not found
Jul 20 16:30:30.009926 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:30.009891    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:30.409894 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:30.409858    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:30.622533 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:30.622493    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:31.008641 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:31.008609    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:31.613559 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:31.613527    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:31.738341 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:31.738291    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:31.936849 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:31.936771    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:32.069876 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:32.069844    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:32.211193 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:32.211111    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:32.620600 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:32.620567    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:32.678922 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:32.678892    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:33.414814 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:33.414776    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:33.641197 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:33.641157    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:33.741730 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:33.741651    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:34.251132 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:34.251103    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:34.738120 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:34.738071    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:34.839752 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:34.839722    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:34.939891 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:34.939860    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:35.408691 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:35.408660    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:35.609871 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:35.609840    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:35.819920 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:35.819884    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:36.211455 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:36.211378    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:36.267069 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:36.267039    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:36.811894 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:36.811861    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:37.226886 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:37.226794    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:37.410468 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:37.410435    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:37.808926 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:37.808892    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:37.974981 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:37.974948    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bd59c\" (UID: \"3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8\") " pod="openshift-insights/insights-runtime-extractor-bd59c"
Jul 20 16:30:37.975143 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:37.975015    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bd59c\" (UID: \"3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8\") " pod="openshift-insights/insights-runtime-extractor-bd59c"
Jul 20 16:30:37.975189 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:37.975169    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 16:30:37.975249 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:30:37.975237    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-insights-runtime-extractor-tls podName:3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8 nodeName:}" failed. No retries permitted until 2026-07-20 16:30:53.975220549 +0000 UTC m=+77.224714546 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bd59c" (UID: "3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8") : secret "insights-runtime-extractor-tls" not found
Jul 20 16:30:37.975445 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:37.975426    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bd59c\" (UID: \"3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8\") " pod="openshift-insights/insights-runtime-extractor-bd59c"
Jul 20 16:30:38.465171 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:38.465144    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gd4p4"
Jul 20 16:30:38.624375 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:38.624341    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:38.727157 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:38.727079    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:38.828072 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:38.828035    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:42.004126 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:42.004086    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1230abfc-e6e1-432a-b99e-2d700df05d26-original-pull-secret\") pod \"global-pull-secret-syncer-4957k\" (UID: \"1230abfc-e6e1-432a-b99e-2d700df05d26\") " pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:30:42.004523 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:42.004251    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1c26b2e-68c2-4421-82c7-7f9e73836624-metrics-certs\") pod \"network-metrics-daemon-2hx9p\" (UID: \"d1c26b2e-68c2-4421-82c7-7f9e73836624\") " pod="openshift-multus/network-metrics-daemon-2hx9p"
Jul 20 16:30:42.004523 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:42.004291    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99t5v\" (UniqueName: \"kubernetes.io/projected/3cb425f5-e1bd-49d1-ab10-38f4431ebc91-kube-api-access-99t5v\") pod \"network-check-target-ss6qh\" (UID: \"3cb425f5-e1bd-49d1-ab10-38f4431ebc91\") " pod="openshift-network-diagnostics/network-check-target-ss6qh"
Jul 20 16:30:42.006671 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:42.006650    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 20 16:30:42.006764 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:42.006651    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 20 16:30:42.007366 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:42.007352    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 20 16:30:42.016935 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:42.016918    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 20 16:30:42.017246 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:42.017226    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1c26b2e-68c2-4421-82c7-7f9e73836624-metrics-certs\") pod \"network-metrics-daemon-2hx9p\" (UID: \"d1c26b2e-68c2-4421-82c7-7f9e73836624\") " pod="openshift-multus/network-metrics-daemon-2hx9p"
Jul 20 16:30:42.017341 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:42.017228    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1230abfc-e6e1-432a-b99e-2d700df05d26-original-pull-secret\") pod \"global-pull-secret-syncer-4957k\" (UID: \"1230abfc-e6e1-432a-b99e-2d700df05d26\") " pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:30:42.027642 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:42.027617    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99t5v\" (UniqueName: \"kubernetes.io/projected/3cb425f5-e1bd-49d1-ab10-38f4431ebc91-kube-api-access-99t5v\") pod \"network-check-target-ss6qh\" (UID: \"3cb425f5-e1bd-49d1-ab10-38f4431ebc91\") " pod="openshift-network-diagnostics/network-check-target-ss6qh"
Jul 20 16:30:42.028895 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:42.028880    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-k84bh\""
Jul 20 16:30:42.034506 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:42.034489    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-f9blf\""
Jul 20 16:30:42.037477 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:42.037465    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2hx9p"
Jul 20 16:30:42.042952 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:42.042933    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ss6qh"
Jul 20 16:30:42.164233 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:42.164204    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2hx9p"]
Jul 20 16:30:42.166948 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:30:42.166922    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1c26b2e_68c2_4421_82c7_7f9e73836624.slice/crio-220efa074e1778573aae730b6bb7964cf8a2129d54aa4a322b3c4b35b4efc5fa WatchSource:0}: Error finding container 220efa074e1778573aae730b6bb7964cf8a2129d54aa4a322b3c4b35b4efc5fa: Status 404 returned error can't find the container with id 220efa074e1778573aae730b6bb7964cf8a2129d54aa4a322b3c4b35b4efc5fa
Jul 20 16:30:42.179947 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:42.179923    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ss6qh"]
Jul 20 16:30:42.183248 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:30:42.183226    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cb425f5_e1bd_49d1_ab10_38f4431ebc91.slice/crio-b0167a818b70c7a4a291b377b5d76fffa26812553741c07e580f485ed180e82f WatchSource:0}: Error finding container b0167a818b70c7a4a291b377b5d76fffa26812553741c07e580f485ed180e82f: Status 404 returned error can't find the container with id b0167a818b70c7a4a291b377b5d76fffa26812553741c07e580f485ed180e82f
Jul 20 16:30:42.216510 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:42.216487    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4957k"
Jul 20 16:30:42.330064 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:42.330033    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4957k"]
Jul 20 16:30:42.332467 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:30:42.332435    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1230abfc_e6e1_432a_b99e_2d700df05d26.slice/crio-4c0a32c6daa203a37ced7e51ea7441106b285955f27cb32a66989a82e2c152fc WatchSource:0}: Error finding container 4c0a32c6daa203a37ced7e51ea7441106b285955f27cb32a66989a82e2c152fc: Status 404 returned error can't find the container with id 4c0a32c6daa203a37ced7e51ea7441106b285955f27cb32a66989a82e2c152fc
Jul 20 16:30:42.563664 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:42.563632    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2hx9p" event={"ID":"d1c26b2e-68c2-4421-82c7-7f9e73836624","Type":"ContainerStarted","Data":"220efa074e1778573aae730b6bb7964cf8a2129d54aa4a322b3c4b35b4efc5fa"}
Jul 20 16:30:42.564630 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:42.564607    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4957k" event={"ID":"1230abfc-e6e1-432a-b99e-2d700df05d26","Type":"ContainerStarted","Data":"4c0a32c6daa203a37ced7e51ea7441106b285955f27cb32a66989a82e2c152fc"}
Jul 20 16:30:42.565530 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:42.565506    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ss6qh" event={"ID":"3cb425f5-e1bd-49d1-ab10-38f4431ebc91","Type":"ContainerStarted","Data":"b0167a818b70c7a4a291b377b5d76fffa26812553741c07e580f485ed180e82f"}
Jul 20 16:30:43.572875 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:43.572841    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2hx9p" event={"ID":"d1c26b2e-68c2-4421-82c7-7f9e73836624","Type":"ContainerStarted","Data":"8ea49f36a4995b9e9c024b55f7af2bd0aaca1de4988d6ab10361414f2a054adf"}
Jul 20 16:30:43.919434 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:43.919399    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/41c8f82b-bd64-40ff-a6c2-ce412321d3fa-metrics-tls\") pod \"dns-default-jzd8m\" (UID: \"41c8f82b-bd64-40ff-a6c2-ce412321d3fa\") " pod="openshift-dns/dns-default-jzd8m"
Jul 20 16:30:43.919642 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:43.919453    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-registry-tls\") pod \"image-registry-59985f8c96-m44rk\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") " pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:30:43.922110 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:43.922083    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/41c8f82b-bd64-40ff-a6c2-ce412321d3fa-metrics-tls\") pod \"dns-default-jzd8m\" (UID: \"41c8f82b-bd64-40ff-a6c2-ce412321d3fa\") " pod="openshift-dns/dns-default-jzd8m"
Jul 20 16:30:43.922216 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:43.922162    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-registry-tls\") pod \"image-registry-59985f8c96-m44rk\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") " pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:30:44.174092 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:44.174003    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-lntkl\""
Jul 20 16:30:44.181268 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:44.181235    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lwj5v\""
Jul 20 16:30:44.182285 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:44.182257    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:30:44.189814 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:44.189627    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jzd8m"
Jul 20 16:30:44.341560 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:44.341528    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59985f8c96-m44rk"]
Jul 20 16:30:44.347650 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:30:44.347622    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf2f8258_be74_4321_8a3f_5b8bcd376065.slice/crio-ccba56ad553851d994b2652ff0c9c670e96ca1cd285af232219343b4e3832be9 WatchSource:0}: Error finding container ccba56ad553851d994b2652ff0c9c670e96ca1cd285af232219343b4e3832be9: Status 404 returned error can't find the container with id ccba56ad553851d994b2652ff0c9c670e96ca1cd285af232219343b4e3832be9
Jul 20 16:30:44.366524 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:44.366454    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jzd8m"]
Jul 20 16:30:44.369254 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:30:44.369221    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41c8f82b_bd64_40ff_a6c2_ce412321d3fa.slice/crio-925afa20fb9551243997c74d6bdce677af7a00e60e94f5bb3e478a3d742a15bb WatchSource:0}: Error finding container 925afa20fb9551243997c74d6bdce677af7a00e60e94f5bb3e478a3d742a15bb: Status 404 returned error can't find the container with id 925afa20fb9551243997c74d6bdce677af7a00e60e94f5bb3e478a3d742a15bb
Jul 20 16:30:44.580585 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:44.580382    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2hx9p" event={"ID":"d1c26b2e-68c2-4421-82c7-7f9e73836624","Type":"ContainerStarted","Data":"b1fc66ad7ced9aadb3f49592a950567c0faad9d38c9e478eb1633d30ea04bda0"}
Jul 20 16:30:44.582569 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:44.582538    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jzd8m" event={"ID":"41c8f82b-bd64-40ff-a6c2-ce412321d3fa","Type":"ContainerStarted","Data":"925afa20fb9551243997c74d6bdce677af7a00e60e94f5bb3e478a3d742a15bb"}
Jul 20 16:30:44.585049 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:44.585022    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59985f8c96-m44rk" event={"ID":"bf2f8258-be74-4321-8a3f-5b8bcd376065","Type":"ContainerStarted","Data":"f77a83d0c41ba499bdac4af17b5e6756609c42e6bb232c4a5fbc56af90a3515e"}
Jul 20 16:30:44.585173 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:44.585070    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59985f8c96-m44rk" event={"ID":"bf2f8258-be74-4321-8a3f-5b8bcd376065","Type":"ContainerStarted","Data":"ccba56ad553851d994b2652ff0c9c670e96ca1cd285af232219343b4e3832be9"}
Jul 20 16:30:44.585517 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:44.585473    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:30:44.595952 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:44.595696    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2hx9p" podStartSLOduration=65.325212182 podStartE2EDuration="1m6.595678657s" totalImagesPullingTime="1.270466475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:29:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:30:44.59529068 +0000 UTC m=+67.844784702" watchObservedRunningTime="2026-07-20 16:30:44.595678657 +0000 UTC m=+67.845172677"
Jul 20 16:30:44.614416 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:44.614376    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-59985f8c96-m44rk" podStartSLOduration=65.614363254 podStartE2EDuration="1m5.614363254s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:29:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:30:44.612868164 +0000 UTC m=+67.862362187" watchObservedRunningTime="2026-07-20 16:30:44.614363254 +0000 UTC m=+67.863857273"
Jul 20 16:30:45.778091 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:45.778057    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-59985f8c96-m44rk"]
Jul 20 16:30:45.832756 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:45.832727    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-c685fb89f-tgvnt"]
Jul 20 16:30:45.850478 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:45.850445    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:45.851563 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:45.851540    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c685fb89f-tgvnt"]
Jul 20 16:30:45.851686 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:45.851652    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c685fb89f-tgvnt"
Jul 20 16:30:45.938095 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:45.938055    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5498a318-5a5f-487f-a788-17bb93a8bd4e-trusted-ca\") pod \"image-registry-c685fb89f-tgvnt\" (UID: \"5498a318-5a5f-487f-a788-17bb93a8bd4e\") " pod="openshift-image-registry/image-registry-c685fb89f-tgvnt"
Jul 20 16:30:45.938095 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:45.938094    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trkzs\" (UniqueName: \"kubernetes.io/projected/5498a318-5a5f-487f-a788-17bb93a8bd4e-kube-api-access-trkzs\") pod \"image-registry-c685fb89f-tgvnt\" (UID: \"5498a318-5a5f-487f-a788-17bb93a8bd4e\") " pod="openshift-image-registry/image-registry-c685fb89f-tgvnt"
Jul 20 16:30:45.938434 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:45.938124    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5498a318-5a5f-487f-a788-17bb93a8bd4e-installation-pull-secrets\") pod \"image-registry-c685fb89f-tgvnt\" (UID: \"5498a318-5a5f-487f-a788-17bb93a8bd4e\") " pod="openshift-image-registry/image-registry-c685fb89f-tgvnt"
Jul 20 16:30:45.938434 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:45.938149    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5498a318-5a5f-487f-a788-17bb93a8bd4e-image-registry-private-configuration\") pod \"image-registry-c685fb89f-tgvnt\" (UID: \"5498a318-5a5f-487f-a788-17bb93a8bd4e\") " pod="openshift-image-registry/image-registry-c685fb89f-tgvnt"
Jul 20 16:30:45.938434 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:45.938183    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5498a318-5a5f-487f-a788-17bb93a8bd4e-bound-sa-token\") pod \"image-registry-c685fb89f-tgvnt\" (UID: \"5498a318-5a5f-487f-a788-17bb93a8bd4e\") " pod="openshift-image-registry/image-registry-c685fb89f-tgvnt"
Jul 20 16:30:45.938434 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:45.938203    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5498a318-5a5f-487f-a788-17bb93a8bd4e-ca-trust-extracted\") pod \"image-registry-c685fb89f-tgvnt\" (UID: \"5498a318-5a5f-487f-a788-17bb93a8bd4e\") " pod="openshift-image-registry/image-registry-c685fb89f-tgvnt"
Jul 20 16:30:45.938434 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:45.938219    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5498a318-5a5f-487f-a788-17bb93a8bd4e-registry-tls\") pod \"image-registry-c685fb89f-tgvnt\" (UID: \"5498a318-5a5f-487f-a788-17bb93a8bd4e\") " pod="openshift-image-registry/image-registry-c685fb89f-tgvnt"
Jul 20 16:30:45.938434 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:45.938232    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5498a318-5a5f-487f-a788-17bb93a8bd4e-registry-certificates\") pod \"image-registry-c685fb89f-tgvnt\" (UID: \"5498a318-5a5f-487f-a788-17bb93a8bd4e\") " pod="openshift-image-registry/image-registry-c685fb89f-tgvnt"
Jul 20 16:30:45.948803 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:45.948748    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:46.039241 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:46.039164    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5498a318-5a5f-487f-a788-17bb93a8bd4e-image-registry-private-configuration\") pod \"image-registry-c685fb89f-tgvnt\" (UID: \"5498a318-5a5f-487f-a788-17bb93a8bd4e\") " pod="openshift-image-registry/image-registry-c685fb89f-tgvnt"
Jul 20 16:30:46.039241 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:46.039211    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5498a318-5a5f-487f-a788-17bb93a8bd4e-bound-sa-token\") pod \"image-registry-c685fb89f-tgvnt\" (UID: \"5498a318-5a5f-487f-a788-17bb93a8bd4e\") " pod="openshift-image-registry/image-registry-c685fb89f-tgvnt"
Jul 20 16:30:46.039241 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:46.039233    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5498a318-5a5f-487f-a788-17bb93a8bd4e-ca-trust-extracted\") pod \"image-registry-c685fb89f-tgvnt\" (UID: \"5498a318-5a5f-487f-a788-17bb93a8bd4e\") " pod="openshift-image-registry/image-registry-c685fb89f-tgvnt"
Jul 20 16:30:46.039527 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:46.039255    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5498a318-5a5f-487f-a788-17bb93a8bd4e-registry-tls\") pod \"image-registry-c685fb89f-tgvnt\" (UID: \"5498a318-5a5f-487f-a788-17bb93a8bd4e\") " pod="openshift-image-registry/image-registry-c685fb89f-tgvnt"
Jul 20 16:30:46.039527 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:46.039270    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5498a318-5a5f-487f-a788-17bb93a8bd4e-registry-certificates\") pod \"image-registry-c685fb89f-tgvnt\" (UID: \"5498a318-5a5f-487f-a788-17bb93a8bd4e\") " pod="openshift-image-registry/image-registry-c685fb89f-tgvnt"
Jul 20 16:30:46.039527 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:46.039344    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5498a318-5a5f-487f-a788-17bb93a8bd4e-trusted-ca\") pod \"image-registry-c685fb89f-tgvnt\" (UID: \"5498a318-5a5f-487f-a788-17bb93a8bd4e\") " pod="openshift-image-registry/image-registry-c685fb89f-tgvnt"
Jul 20 16:30:46.039527 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:46.039368    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trkzs\" (UniqueName: \"kubernetes.io/projected/5498a318-5a5f-487f-a788-17bb93a8bd4e-kube-api-access-trkzs\") pod \"image-registry-c685fb89f-tgvnt\" (UID: \"5498a318-5a5f-487f-a788-17bb93a8bd4e\") " pod="openshift-image-registry/image-registry-c685fb89f-tgvnt"
Jul 20 16:30:46.039527 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:46.039397    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5498a318-5a5f-487f-a788-17bb93a8bd4e-installation-pull-secrets\") pod \"image-registry-c685fb89f-tgvnt\" (UID: \"5498a318-5a5f-487f-a788-17bb93a8bd4e\") " pod="openshift-image-registry/image-registry-c685fb89f-tgvnt"
Jul 20 16:30:46.039981 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:46.039934    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5498a318-5a5f-487f-a788-17bb93a8bd4e-ca-trust-extracted\") pod \"image-registry-c685fb89f-tgvnt\" (UID: \"5498a318-5a5f-487f-a788-17bb93a8bd4e\") " pod="openshift-image-registry/image-registry-c685fb89f-tgvnt"
Jul 20 16:30:46.040540 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:46.040491    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5498a318-5a5f-487f-a788-17bb93a8bd4e-registry-certificates\") pod \"image-registry-c685fb89f-tgvnt\" (UID: \"5498a318-5a5f-487f-a788-17bb93a8bd4e\") " pod="openshift-image-registry/image-registry-c685fb89f-tgvnt"
Jul 20 16:30:46.041385 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:46.041355    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5498a318-5a5f-487f-a788-17bb93a8bd4e-trusted-ca\") pod \"image-registry-c685fb89f-tgvnt\" (UID: \"5498a318-5a5f-487f-a788-17bb93a8bd4e\") " pod="openshift-image-registry/image-registry-c685fb89f-tgvnt"
Jul 20 16:30:46.042247 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:46.042223    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5498a318-5a5f-487f-a788-17bb93a8bd4e-installation-pull-secrets\") pod \"image-registry-c685fb89f-tgvnt\" (UID: \"5498a318-5a5f-487f-a788-17bb93a8bd4e\") " pod="openshift-image-registry/image-registry-c685fb89f-tgvnt"
Jul 20 16:30:46.042366 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:46.042269    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5498a318-5a5f-487f-a788-17bb93a8bd4e-image-registry-private-configuration\") pod \"image-registry-c685fb89f-tgvnt\" (UID: \"5498a318-5a5f-487f-a788-17bb93a8bd4e\") " pod="openshift-image-registry/image-registry-c685fb89f-tgvnt"
Jul 20 16:30:46.042475 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:46.042442    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5498a318-5a5f-487f-a788-17bb93a8bd4e-registry-tls\") pod \"image-registry-c685fb89f-tgvnt\" (UID: \"5498a318-5a5f-487f-a788-17bb93a8bd4e\") " pod="openshift-image-registry/image-registry-c685fb89f-tgvnt"
Jul 20 16:30:46.048135 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:46.048110    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5498a318-5a5f-487f-a788-17bb93a8bd4e-bound-sa-token\") pod \"image-registry-c685fb89f-tgvnt\" (UID: \"5498a318-5a5f-487f-a788-17bb93a8bd4e\") " pod="openshift-image-registry/image-registry-c685fb89f-tgvnt"
Jul 20 16:30:46.048459 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:46.048415    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trkzs\" (UniqueName: \"kubernetes.io/projected/5498a318-5a5f-487f-a788-17bb93a8bd4e-kube-api-access-trkzs\") pod \"image-registry-c685fb89f-tgvnt\" (UID: \"5498a318-5a5f-487f-a788-17bb93a8bd4e\") " pod="openshift-image-registry/image-registry-c685fb89f-tgvnt"
Jul 20 16:30:46.050703 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:46.050679    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:46.150898 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:46.150841    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:46.162921 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:46.162825    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c685fb89f-tgvnt"
Jul 20 16:30:48.225427 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:48.224507    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c685fb89f-tgvnt"]
Jul 20 16:30:48.229451 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:30:48.229423    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5498a318_5a5f_487f_a788_17bb93a8bd4e.slice/crio-7a0730bdf519c464b2156917310478649a21e149fb77131155408a135ba94ebd WatchSource:0}: Error finding container 7a0730bdf519c464b2156917310478649a21e149fb77131155408a135ba94ebd: Status 404 returned error can't find the container with id 7a0730bdf519c464b2156917310478649a21e149fb77131155408a135ba94ebd
Jul 20 16:30:48.600456 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:48.600417    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ss6qh" event={"ID":"3cb425f5-e1bd-49d1-ab10-38f4431ebc91","Type":"ContainerStarted","Data":"17bcce61170ce075d40d79c051a99ed097cb03380b9889af81d32c0bead980e9"}
Jul 20 16:30:48.600642 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:48.600533    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-ss6qh"
Jul 20 16:30:48.602267 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:48.602240    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c685fb89f-tgvnt" event={"ID":"5498a318-5a5f-487f-a788-17bb93a8bd4e","Type":"ContainerStarted","Data":"4d818d8df36009ee040326e7e2c54ac8ebf7fb7bc6f84f16a26abb86195273aa"}
Jul 20 16:30:48.602404 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:48.602282    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c685fb89f-tgvnt" event={"ID":"5498a318-5a5f-487f-a788-17bb93a8bd4e","Type":"ContainerStarted","Data":"7a0730bdf519c464b2156917310478649a21e149fb77131155408a135ba94ebd"}
Jul 20 16:30:48.602404 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:48.602317    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-c685fb89f-tgvnt"
Jul 20 16:30:48.603871 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:48.603848    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4957k" event={"ID":"1230abfc-e6e1-432a-b99e-2d700df05d26","Type":"ContainerStarted","Data":"09cffdca4fedaf3a444742408b8369dee2b02cd6cac011d076049f442bf012fc"}
Jul 20 16:30:48.605452 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:48.605429    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jzd8m" event={"ID":"41c8f82b-bd64-40ff-a6c2-ce412321d3fa","Type":"ContainerStarted","Data":"f11b13318b68953d0eba61873482c38e85b85436fcc46478f4c9d2703ace887a"}
Jul 20 16:30:48.605553 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:48.605468    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jzd8m" event={"ID":"41c8f82b-bd64-40ff-a6c2-ce412321d3fa","Type":"ContainerStarted","Data":"256cc0d97875d00208c1b1654ea7ecd90dc4f7914b717c065624efd6c9c3d9b4"}
Jul 20 16:30:48.605615 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:48.605556    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jzd8m"
Jul 20 16:30:48.615660 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:48.615621    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-ss6qh" podStartSLOduration=64.719788049 podStartE2EDuration="1m10.615609892s" totalImagesPullingTime="5.895821843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:29:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:30:48.614684019 +0000 UTC m=+71.864178038" watchObservedRunningTime="2026-07-20 16:30:48.615609892 +0000 UTC m=+71.865103910"
Jul 20 16:30:48.629286 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:48.629245    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4957k" podStartSLOduration=64.882540182 podStartE2EDuration="1m10.629233642s" totalImagesPullingTime="5.74669346s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:29:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:30:48.628422206 +0000 UTC m=+71.877916229" watchObservedRunningTime="2026-07-20 16:30:48.629233642 +0000 UTC m=+71.878727658"
Jul 20 16:30:48.646104 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:48.646049    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-jzd8m" podStartSLOduration=32.909324944 podStartE2EDuration="36.646030683s" totalImagesPullingTime="3.736705739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:30:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:30:48.644651905 +0000 UTC m=+71.894145925" watchObservedRunningTime="2026-07-20 16:30:48.646030683 +0000 UTC m=+71.895524703"
Jul 20 16:30:48.662489 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:48.662443    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-c685fb89f-tgvnt" podStartSLOduration=3.662429677 podStartE2EDuration="3.662429677s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:30:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:30:48.661680228 +0000 UTC m=+71.911174248" watchObservedRunningTime="2026-07-20 16:30:48.662429677 +0000 UTC m=+71.911923695"
Jul 20 16:30:48.750713 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:48.750680    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:48.851168 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:48.851078    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:48.929879 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:48.929845    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:49.251956 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:49.251867    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:49.327257 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:49.327223    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:49.384875 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:49.384847    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:49.485725 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:49.485689    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:49.584665 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:49.584641    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:49.912053 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:49.911967    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:50.480756 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:50.480725    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:50.536691 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:50.536665    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:50.722473 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:50.722442    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:50.823401 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:50.823369    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:50.881551 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:50.881511    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:51.114849 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:51.114736    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:51.650335 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:51.650292    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:51.772279 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:51.772242    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:51.827226 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:51.827202    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:52.130183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:52.130148    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:54.005022 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:54.004987    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bd59c\" (UID: \"3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8\") " pod="openshift-insights/insights-runtime-extractor-bd59c"
Jul 20 16:30:54.007211 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:54.007191    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bd59c\" (UID: \"3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8\") " pod="openshift-insights/insights-runtime-extractor-bd59c"
Jul 20 16:30:54.236847 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:54.236815    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-w4qdt\""
Jul 20 16:30:54.244760 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:54.244734    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bd59c"
Jul 20 16:30:54.448995 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:54.448967    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bd59c"]
Jul 20 16:30:54.452020 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:30:54.451993    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f4098ce_46e2_4f61_8ae3_3a3f71b3abc8.slice/crio-6f0348863890ae3d9cc68c22e0b07ba2e1c972c2b9d53c91166cc9de8e4ae173 WatchSource:0}: Error finding container 6f0348863890ae3d9cc68c22e0b07ba2e1c972c2b9d53c91166cc9de8e4ae173: Status 404 returned error can't find the container with id 6f0348863890ae3d9cc68c22e0b07ba2e1c972c2b9d53c91166cc9de8e4ae173
Jul 20 16:30:54.626685 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:54.626613    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bd59c" event={"ID":"3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8","Type":"ContainerStarted","Data":"6f0348863890ae3d9cc68c22e0b07ba2e1c972c2b9d53c91166cc9de8e4ae173"}
Jul 20 16:30:54.855717 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:54.855681    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:55.403916 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:55.403885    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:57.420517 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:57.420484    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:57.523360 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:57.523322    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:58.338492 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:58.338456    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:30:58.540460 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:58.540423    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-845dd6cc47-c78k5"]
Jul 20 16:30:58.555650 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:58.555625    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-845dd6cc47-c78k5"]
Jul 20 16:30:58.555809 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:58.555714    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-845dd6cc47-c78k5"
Jul 20 16:30:58.558646 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:58.558623    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 20 16:30:58.558772 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:58.558624    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 20 16:30:58.559867 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:58.559843    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 20 16:30:58.559867 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:58.559858    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 20 16:30:58.560050 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:58.560015    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 20 16:30:58.560170 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:58.560142    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 20 16:30:58.560277 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:58.560268    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 20 16:30:58.560396 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:58.560346    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-xvhqd\""
Jul 20 16:30:58.611037 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:58.610974    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jzd8m"
Jul 20 16:30:58.640234 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:58.640202    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-oauth-serving-cert\") pod \"console-845dd6cc47-c78k5\" (UID: \"33aa27a1-16e2-4e72-b8c4-7d76abdb2e51\") " pod="openshift-console/console-845dd6cc47-c78k5"
Jul 20 16:30:58.640441 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:58.640258    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-console-oauth-config\") pod \"console-845dd6cc47-c78k5\" (UID: \"33aa27a1-16e2-4e72-b8c4-7d76abdb2e51\") " pod="openshift-console/console-845dd6cc47-c78k5"
Jul 20 16:30:58.640441 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:58.640330    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-console-serving-cert\") pod \"console-845dd6cc47-c78k5\" (UID: \"33aa27a1-16e2-4e72-b8c4-7d76abdb2e51\") " pod="openshift-console/console-845dd6cc47-c78k5"
Jul 20 16:30:58.640441 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:58.640366    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb7rx\" (UniqueName: \"kubernetes.io/projected/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-kube-api-access-bb7rx\") pod \"console-845dd6cc47-c78k5\" (UID: \"33aa27a1-16e2-4e72-b8c4-7d76abdb2e51\") " pod="openshift-console/console-845dd6cc47-c78k5"
Jul 20 16:30:58.640441 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:58.640394    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-service-ca\") pod \"console-845dd6cc47-c78k5\" (UID: \"33aa27a1-16e2-4e72-b8c4-7d76abdb2e51\") " pod="openshift-console/console-845dd6cc47-c78k5"
Jul 20 16:30:58.640653 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:58.640479    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-console-config\") pod \"console-845dd6cc47-c78k5\" (UID: \"33aa27a1-16e2-4e72-b8c4-7d76abdb2e51\") " pod="openshift-console/console-845dd6cc47-c78k5"
Jul 20 16:30:58.741202 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:58.741168    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-console-oauth-config\") pod \"console-845dd6cc47-c78k5\" (UID: \"33aa27a1-16e2-4e72-b8c4-7d76abdb2e51\") " pod="openshift-console/console-845dd6cc47-c78k5"
Jul 20 16:30:58.741415 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:58.741213    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-console-serving-cert\") pod \"console-845dd6cc47-c78k5\" (UID: \"33aa27a1-16e2-4e72-b8c4-7d76abdb2e51\") " pod="openshift-console/console-845dd6cc47-c78k5"
Jul 20 16:30:58.741415 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:58.741257    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb7rx\" (UniqueName: \"kubernetes.io/projected/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-kube-api-access-bb7rx\") pod \"console-845dd6cc47-c78k5\" (UID: \"33aa27a1-16e2-4e72-b8c4-7d76abdb2e51\") " pod="openshift-console/console-845dd6cc47-c78k5"
Jul 20 16:30:58.741415 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:58.741404    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-service-ca\") pod \"console-845dd6cc47-c78k5\" (UID: \"33aa27a1-16e2-4e72-b8c4-7d76abdb2e51\") " pod="openshift-console/console-845dd6cc47-c78k5"
Jul 20 16:30:58.741585 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:58.741524    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-console-config\") pod \"console-845dd6cc47-c78k5\" (UID: \"33aa27a1-16e2-4e72-b8c4-7d76abdb2e51\") " pod="openshift-console/console-845dd6cc47-c78k5"
Jul 20 16:30:58.741643 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:58.741614    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-oauth-serving-cert\") pod \"console-845dd6cc47-c78k5\" (UID: \"33aa27a1-16e2-4e72-b8c4-7d76abdb2e51\") " pod="openshift-console/console-845dd6cc47-c78k5"
Jul 20 16:30:58.742237 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:58.742211    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-console-config\") pod \"console-845dd6cc47-c78k5\" (UID: \"33aa27a1-16e2-4e72-b8c4-7d76abdb2e51\") " pod="openshift-console/console-845dd6cc47-c78k5"
Jul 20 16:30:58.742393 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:58.742211    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-service-ca\") pod \"console-845dd6cc47-c78k5\" (UID: \"33aa27a1-16e2-4e72-b8c4-7d76abdb2e51\") " pod="openshift-console/console-845dd6cc47-c78k5"
Jul 20 16:30:58.742393 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:58.742314    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-oauth-serving-cert\") pod \"console-845dd6cc47-c78k5\" (UID: \"33aa27a1-16e2-4e72-b8c4-7d76abdb2e51\") " pod="openshift-console/console-845dd6cc47-c78k5"
Jul 20 16:30:58.743958 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:58.743923    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-console-oauth-config\") pod \"console-845dd6cc47-c78k5\" (UID: \"33aa27a1-16e2-4e72-b8c4-7d76abdb2e51\") " pod="openshift-console/console-845dd6cc47-c78k5"
Jul 20 16:30:58.744131 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:58.744074    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-console-serving-cert\") pod \"console-845dd6cc47-c78k5\" (UID: \"33aa27a1-16e2-4e72-b8c4-7d76abdb2e51\") " pod="openshift-console/console-845dd6cc47-c78k5"
Jul 20 16:30:58.748951 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:58.748905    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb7rx\" (UniqueName: \"kubernetes.io/projected/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-kube-api-access-bb7rx\") pod \"console-845dd6cc47-c78k5\" (UID: \"33aa27a1-16e2-4e72-b8c4-7d76abdb2e51\") " pod="openshift-console/console-845dd6cc47-c78k5"
Jul 20 16:30:58.867831 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:58.867748    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-845dd6cc47-c78k5"
Jul 20 16:30:59.308095 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:30:59.308058    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:00.696385 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:00.696362    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-845dd6cc47-c78k5"]
Jul 20 16:31:00.698780 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:31:00.698749    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33aa27a1_16e2_4e72_b8c4_7d76abdb2e51.slice/crio-9779176fe4c8bad8ffd922e51a001ec54170d6986ed8f30583a9773a4158313c WatchSource:0}: Error finding container 9779176fe4c8bad8ffd922e51a001ec54170d6986ed8f30583a9773a4158313c: Status 404 returned error can't find the container with id 9779176fe4c8bad8ffd922e51a001ec54170d6986ed8f30583a9773a4158313c
Jul 20 16:31:00.993009 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:00.992892    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:01.112069 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.112034    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:01.213357 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.213317    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:01.316113 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.316074    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:01.344922 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.344866    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-mn2jt"]
Jul 20 16:31:01.347452 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.347234    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-mn2jt"
Jul 20 16:31:01.351674 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.351344    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 20 16:31:01.351674 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.351457    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 20 16:31:01.351674 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.351492    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 20 16:31:01.351674 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.351559    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 20 16:31:01.351674 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.351661    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 20 16:31:01.352015 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.351754    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-jdh9h\""
Jul 20 16:31:01.352015 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.351901    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 20 16:31:01.364083 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.363702    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-mn2jt"]
Jul 20 16:31:01.370060 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.370037    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-drtfq"]
Jul 20 16:31:01.374975 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.374946    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-drtfq"
Jul 20 16:31:01.377596 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.377576    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 20 16:31:01.377986 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.377807    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-25shj\""
Jul 20 16:31:01.377986 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.377890    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 20 16:31:01.377986 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.377914    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 20 16:31:01.430433 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.430398    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:01.464667 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.464632    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/57efcb32-98cb-44fd-9044-1647dfb1f735-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-mn2jt\" (UID: \"57efcb32-98cb-44fd-9044-1647dfb1f735\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mn2jt"
Jul 20 16:31:01.464822 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.464680    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57efcb32-98cb-44fd-9044-1647dfb1f735-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-mn2jt\" (UID: \"57efcb32-98cb-44fd-9044-1647dfb1f735\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mn2jt"
Jul 20 16:31:01.464822 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.464710    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzckn\" (UniqueName: \"kubernetes.io/projected/57efcb32-98cb-44fd-9044-1647dfb1f735-kube-api-access-kzckn\") pod \"kube-state-metrics-56c5594674-mn2jt\" (UID: \"57efcb32-98cb-44fd-9044-1647dfb1f735\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mn2jt"
Jul 20 16:31:01.464822 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.464757    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/57efcb32-98cb-44fd-9044-1647dfb1f735-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-mn2jt\" (UID: \"57efcb32-98cb-44fd-9044-1647dfb1f735\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mn2jt"
Jul 20 16:31:01.464939 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.464825    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/112aad28-f9b6-415e-ac47-3334d4c2a15e-root\") pod \"node-exporter-drtfq\" (UID: \"112aad28-f9b6-415e-ac47-3334d4c2a15e\") " pod="openshift-monitoring/node-exporter-drtfq"
Jul 20 16:31:01.464939 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.464845    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/112aad28-f9b6-415e-ac47-3334d4c2a15e-node-exporter-textfile\") pod \"node-exporter-drtfq\" (UID: \"112aad28-f9b6-415e-ac47-3334d4c2a15e\") " pod="openshift-monitoring/node-exporter-drtfq"
Jul 20 16:31:01.464939 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.464877    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/112aad28-f9b6-415e-ac47-3334d4c2a15e-node-exporter-accelerators-collector-config\") pod \"node-exporter-drtfq\" (UID: \"112aad28-f9b6-415e-ac47-3334d4c2a15e\") " pod="openshift-monitoring/node-exporter-drtfq"
Jul 20 16:31:01.464939 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.464901    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtxgv\" (UniqueName: \"kubernetes.io/projected/112aad28-f9b6-415e-ac47-3334d4c2a15e-kube-api-access-gtxgv\") pod \"node-exporter-drtfq\" (UID: \"112aad28-f9b6-415e-ac47-3334d4c2a15e\") " pod="openshift-monitoring/node-exporter-drtfq"
Jul 20 16:31:01.465066 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.464962    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/112aad28-f9b6-415e-ac47-3334d4c2a15e-sys\") pod \"node-exporter-drtfq\" (UID: \"112aad28-f9b6-415e-ac47-3334d4c2a15e\") " pod="openshift-monitoring/node-exporter-drtfq"
Jul 20 16:31:01.465066 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.464991    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/112aad28-f9b6-415e-ac47-3334d4c2a15e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-drtfq\" (UID: \"112aad28-f9b6-415e-ac47-3334d4c2a15e\") " pod="openshift-monitoring/node-exporter-drtfq"
Jul 20 16:31:01.465066 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.465033    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/112aad28-f9b6-415e-ac47-3334d4c2a15e-node-exporter-tls\") pod \"node-exporter-drtfq\" (UID: \"112aad28-f9b6-415e-ac47-3334d4c2a15e\") " pod="openshift-monitoring/node-exporter-drtfq"
Jul 20 16:31:01.465066 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.465058    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/112aad28-f9b6-415e-ac47-3334d4c2a15e-node-exporter-wtmp\") pod \"node-exporter-drtfq\" (UID: \"112aad28-f9b6-415e-ac47-3334d4c2a15e\") " pod="openshift-monitoring/node-exporter-drtfq"
Jul 20 16:31:01.465213 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.465098    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/57efcb32-98cb-44fd-9044-1647dfb1f735-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-mn2jt\" (UID: \"57efcb32-98cb-44fd-9044-1647dfb1f735\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mn2jt"
Jul 20 16:31:01.465213 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.465128    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/57efcb32-98cb-44fd-9044-1647dfb1f735-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-mn2jt\" (UID: \"57efcb32-98cb-44fd-9044-1647dfb1f735\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mn2jt"
Jul 20 16:31:01.465213 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.465150    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/112aad28-f9b6-415e-ac47-3334d4c2a15e-metrics-client-ca\") pod \"node-exporter-drtfq\" (UID: \"112aad28-f9b6-415e-ac47-3334d4c2a15e\") " pod="openshift-monitoring/node-exporter-drtfq"
Jul 20 16:31:01.521259 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.521219    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:01.566028 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.565991    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/112aad28-f9b6-415e-ac47-3334d4c2a15e-node-exporter-accelerators-collector-config\") pod \"node-exporter-drtfq\" (UID: \"112aad28-f9b6-415e-ac47-3334d4c2a15e\") " pod="openshift-monitoring/node-exporter-drtfq"
Jul 20 16:31:01.566236 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.566111    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtxgv\" (UniqueName: \"kubernetes.io/projected/112aad28-f9b6-415e-ac47-3334d4c2a15e-kube-api-access-gtxgv\") pod \"node-exporter-drtfq\" (UID: \"112aad28-f9b6-415e-ac47-3334d4c2a15e\") " pod="openshift-monitoring/node-exporter-drtfq"
Jul 20 16:31:01.566236 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.566145    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/112aad28-f9b6-415e-ac47-3334d4c2a15e-sys\") pod \"node-exporter-drtfq\" (UID: \"112aad28-f9b6-415e-ac47-3334d4c2a15e\") " pod="openshift-monitoring/node-exporter-drtfq"
Jul 20 16:31:01.566236 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.566173    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/112aad28-f9b6-415e-ac47-3334d4c2a15e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-drtfq\" (UID: \"112aad28-f9b6-415e-ac47-3334d4c2a15e\") " pod="openshift-monitoring/node-exporter-drtfq"
Jul 20 16:31:01.566236 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.566212    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/112aad28-f9b6-415e-ac47-3334d4c2a15e-node-exporter-tls\") pod \"node-exporter-drtfq\" (UID: \"112aad28-f9b6-415e-ac47-3334d4c2a15e\") " pod="openshift-monitoring/node-exporter-drtfq"
Jul 20 16:31:01.566236 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.566235    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/112aad28-f9b6-415e-ac47-3334d4c2a15e-node-exporter-wtmp\") pod \"node-exporter-drtfq\" (UID: \"112aad28-f9b6-415e-ac47-3334d4c2a15e\") " pod="openshift-monitoring/node-exporter-drtfq"
Jul 20 16:31:01.566544 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.566284    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/57efcb32-98cb-44fd-9044-1647dfb1f735-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-mn2jt\" (UID: \"57efcb32-98cb-44fd-9044-1647dfb1f735\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mn2jt"
Jul 20 16:31:01.566544 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.566337    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/57efcb32-98cb-44fd-9044-1647dfb1f735-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-mn2jt\" (UID: \"57efcb32-98cb-44fd-9044-1647dfb1f735\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mn2jt"
Jul 20 16:31:01.566544 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.566363    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/112aad28-f9b6-415e-ac47-3334d4c2a15e-metrics-client-ca\") pod \"node-exporter-drtfq\" (UID: \"112aad28-f9b6-415e-ac47-3334d4c2a15e\") " pod="openshift-monitoring/node-exporter-drtfq"
Jul 20 16:31:01.566544 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.566393    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/57efcb32-98cb-44fd-9044-1647dfb1f735-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-mn2jt\" (UID: \"57efcb32-98cb-44fd-9044-1647dfb1f735\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mn2jt"
Jul 20 16:31:01.566544 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.566424    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57efcb32-98cb-44fd-9044-1647dfb1f735-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-mn2jt\" (UID: \"57efcb32-98cb-44fd-9044-1647dfb1f735\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mn2jt"
Jul 20 16:31:01.566544 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.566456    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzckn\" (UniqueName: \"kubernetes.io/projected/57efcb32-98cb-44fd-9044-1647dfb1f735-kube-api-access-kzckn\") pod \"kube-state-metrics-56c5594674-mn2jt\" (UID: \"57efcb32-98cb-44fd-9044-1647dfb1f735\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mn2jt"
Jul 20 16:31:01.566544 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.566486    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/57efcb32-98cb-44fd-9044-1647dfb1f735-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-mn2jt\" (UID: \"57efcb32-98cb-44fd-9044-1647dfb1f735\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mn2jt"
Jul 20 16:31:01.566544 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.566516    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/112aad28-f9b6-415e-ac47-3334d4c2a15e-root\") pod \"node-exporter-drtfq\" (UID: \"112aad28-f9b6-415e-ac47-3334d4c2a15e\") " pod="openshift-monitoring/node-exporter-drtfq"
Jul 20 16:31:01.566544 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.566530    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/112aad28-f9b6-415e-ac47-3334d4c2a15e-node-exporter-textfile\") pod \"node-exporter-drtfq\" (UID: \"112aad28-f9b6-415e-ac47-3334d4c2a15e\") " pod="openshift-monitoring/node-exporter-drtfq"
Jul 20 16:31:01.566971 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.566604    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/112aad28-f9b6-415e-ac47-3334d4c2a15e-node-exporter-accelerators-collector-config\") pod \"node-exporter-drtfq\" (UID: \"112aad28-f9b6-415e-ac47-3334d4c2a15e\") " pod="openshift-monitoring/node-exporter-drtfq"
Jul 20 16:31:01.566971 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.566774    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/112aad28-f9b6-415e-ac47-3334d4c2a15e-node-exporter-textfile\") pod \"node-exporter-drtfq\" (UID: \"112aad28-f9b6-415e-ac47-3334d4c2a15e\") " pod="openshift-monitoring/node-exporter-drtfq"
Jul 20 16:31:01.566971 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.566425    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/112aad28-f9b6-415e-ac47-3334d4c2a15e-sys\") pod \"node-exporter-drtfq\" (UID: \"112aad28-f9b6-415e-ac47-3334d4c2a15e\") " pod="openshift-monitoring/node-exporter-drtfq"
Jul 20 16:31:01.567924 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.567191    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/112aad28-f9b6-415e-ac47-3334d4c2a15e-root\") pod \"node-exporter-drtfq\" (UID: \"112aad28-f9b6-415e-ac47-3334d4c2a15e\") " pod="openshift-monitoring/node-exporter-drtfq"
Jul 20 16:31:01.567924 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.567367    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/112aad28-f9b6-415e-ac47-3334d4c2a15e-node-exporter-wtmp\") pod \"node-exporter-drtfq\" (UID: \"112aad28-f9b6-415e-ac47-3334d4c2a15e\") " pod="openshift-monitoring/node-exporter-drtfq"
Jul 20 16:31:01.567924 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.567383    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57efcb32-98cb-44fd-9044-1647dfb1f735-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-mn2jt\" (UID: \"57efcb32-98cb-44fd-9044-1647dfb1f735\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mn2jt"
Jul 20 16:31:01.567924 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.567709    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/57efcb32-98cb-44fd-9044-1647dfb1f735-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-mn2jt\" (UID: \"57efcb32-98cb-44fd-9044-1647dfb1f735\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mn2jt"
Jul 20 16:31:01.567924 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.567881    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/112aad28-f9b6-415e-ac47-3334d4c2a15e-metrics-client-ca\") pod \"node-exporter-drtfq\" (UID: \"112aad28-f9b6-415e-ac47-3334d4c2a15e\") " pod="openshift-monitoring/node-exporter-drtfq"
Jul 20 16:31:01.568238 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.568016    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/57efcb32-98cb-44fd-9044-1647dfb1f735-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-mn2jt\" (UID: \"57efcb32-98cb-44fd-9044-1647dfb1f735\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mn2jt"
Jul 20 16:31:01.570177 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.569191    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/112aad28-f9b6-415e-ac47-3334d4c2a15e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-drtfq\" (UID: \"112aad28-f9b6-415e-ac47-3334d4c2a15e\") " pod="openshift-monitoring/node-exporter-drtfq"
Jul 20 16:31:01.570177 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.570070    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/57efcb32-98cb-44fd-9044-1647dfb1f735-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-mn2jt\" (UID: \"57efcb32-98cb-44fd-9044-1647dfb1f735\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mn2jt"
Jul 20 16:31:01.570177 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.570087    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/57efcb32-98cb-44fd-9044-1647dfb1f735-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-mn2jt\" (UID: \"57efcb32-98cb-44fd-9044-1647dfb1f735\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mn2jt"
Jul 20 16:31:01.570177 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.570151    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/112aad28-f9b6-415e-ac47-3334d4c2a15e-node-exporter-tls\") pod \"node-exporter-drtfq\" (UID: \"112aad28-f9b6-415e-ac47-3334d4c2a15e\") " pod="openshift-monitoring/node-exporter-drtfq"
Jul 20 16:31:01.576170 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.576129    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzckn\" (UniqueName: \"kubernetes.io/projected/57efcb32-98cb-44fd-9044-1647dfb1f735-kube-api-access-kzckn\") pod \"kube-state-metrics-56c5594674-mn2jt\" (UID: \"57efcb32-98cb-44fd-9044-1647dfb1f735\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mn2jt"
Jul 20 16:31:01.577064 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.577023    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtxgv\" (UniqueName: \"kubernetes.io/projected/112aad28-f9b6-415e-ac47-3334d4c2a15e-kube-api-access-gtxgv\") pod \"node-exporter-drtfq\" (UID: \"112aad28-f9b6-415e-ac47-3334d4c2a15e\") " pod="openshift-monitoring/node-exporter-drtfq"
Jul 20 16:31:01.624407 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.624373    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:01.662530 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.659791    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bd59c" event={"ID":"3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8","Type":"ContainerStarted","Data":"899d3ef6126cd21ceff6f3e9718fffee910eea2508db1454d36a8f92f487e3be"}
Jul 20 16:31:01.662530 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.662353    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-845dd6cc47-c78k5" event={"ID":"33aa27a1-16e2-4e72-b8c4-7d76abdb2e51","Type":"ContainerStarted","Data":"9779176fe4c8bad8ffd922e51a001ec54170d6986ed8f30583a9773a4158313c"}
Jul 20 16:31:01.664813 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.664791    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-mn2jt"
Jul 20 16:31:01.687824 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.687798    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-drtfq"
Jul 20 16:31:01.700533 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:31:01.700482    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod112aad28_f9b6_415e_ac47_3334d4c2a15e.slice/crio-6d3a7cfcd520dd785cc85c1697dbc1ec268b08b9c5474d4572715804633cffd8 WatchSource:0}: Error finding container 6d3a7cfcd520dd785cc85c1697dbc1ec268b08b9c5474d4572715804633cffd8: Status 404 returned error can't find the container with id 6d3a7cfcd520dd785cc85c1697dbc1ec268b08b9c5474d4572715804633cffd8
Jul 20 16:31:01.718842 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.718812    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:01.833790 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:01.833764    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-mn2jt"]
Jul 20 16:31:02.049139 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:02.049055    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:02.324624 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:02.323943    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:02.434120 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:02.434036    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:02.570153 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:02.570117    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:02.669700 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:02.669541    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-drtfq" event={"ID":"112aad28-f9b6-415e-ac47-3334d4c2a15e","Type":"ContainerStarted","Data":"6d3a7cfcd520dd785cc85c1697dbc1ec268b08b9c5474d4572715804633cffd8"}
Jul 20 16:31:02.671453 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:02.671386    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-mn2jt" event={"ID":"57efcb32-98cb-44fd-9044-1647dfb1f735","Type":"ContainerStarted","Data":"0ef4c58707cedb7cb1d71220f22f61526f9e38f3619c5a6076e962bf85368a1d"}
Jul 20 16:31:04.257441 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:04.257407    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:04.324643 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:04.324604    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:04.491492 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:04.491451    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:04.588056 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:04.588022    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:04.935496 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:04.935410    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:05.036372 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:05.036337    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:06.081022 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:06.080985    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:06.142058 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:06.142016    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:06.225188 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:06.225145    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:06.283116 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:06.283085    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:06.364849 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:06.364758    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:06.600059 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:06.599992    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:31:06.688980 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:06.688900    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bd59c" event={"ID":"3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8","Type":"ContainerStarted","Data":"1107589ea1216e9577789d0e979101c89696b6d1b8e88bfe59baeee3d8458a6a"}
Jul 20 16:31:06.690685 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:06.690658    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-845dd6cc47-c78k5" event={"ID":"33aa27a1-16e2-4e72-b8c4-7d76abdb2e51","Type":"ContainerStarted","Data":"d5d28b15ef72ba1e66378aee0ccb9bb18269d7255edc96520e4bec317b04773d"}
Jul 20 16:31:06.692410 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:06.692384    2577 generic.go:356] "Generic (PLEG): container finished" podID="112aad28-f9b6-415e-ac47-3334d4c2a15e" containerID="f9c990b8bf660c10a7666c044ea5c559ce8c8cb45848578a96158a09597c7c13" exitCode=0
Jul 20 16:31:06.692540 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:06.692459    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-drtfq" event={"ID":"112aad28-f9b6-415e-ac47-3334d4c2a15e","Type":"ContainerDied","Data":"f9c990b8bf660c10a7666c044ea5c559ce8c8cb45848578a96158a09597c7c13"}
Jul 20 16:31:06.695820 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:06.695785    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-mn2jt" event={"ID":"57efcb32-98cb-44fd-9044-1647dfb1f735","Type":"ContainerStarted","Data":"3957cdc9ae5bab33c8636a308385530829aee81e9d267721e11cfefe5a4f306f"}
Jul 20 16:31:06.695923 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:06.695839    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-mn2jt" event={"ID":"57efcb32-98cb-44fd-9044-1647dfb1f735","Type":"ContainerStarted","Data":"26c5c8cf05da24857900a7263529b52795a12bf677f68368d7aa73c1a8541428"}
Jul 20 16:31:06.695923 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:06.695872    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-mn2jt" event={"ID":"57efcb32-98cb-44fd-9044-1647dfb1f735","Type":"ContainerStarted","Data":"0a96bbe14e83439c0184e117ceaa1d62bc1a7654113e486ad50a7a782c81bdc5"}
Jul 20 16:31:06.745990 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:06.745945    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-845dd6cc47-c78k5" podStartSLOduration=3.04677304 podStartE2EDuration="8.745928754s" totalImagesPullingTime="5.699155714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:30:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:31:06.716695288 +0000 UTC m=+89.966189308" watchObservedRunningTime="2026-07-20 16:31:06.745928754 +0000 UTC m=+89.995422817"
Jul 20 16:31:06.746115 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:06.746034    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-mn2jt" podStartSLOduration=1.1860387239999999 podStartE2EDuration="5.746029852s" totalImagesPullingTime="4.559991128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:31:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:31:06.745202343 +0000 UTC m=+89.994696363" watchObservedRunningTime="2026-07-20 16:31:06.746029852 +0000 UTC m=+89.995523869"
Jul 20 16:31:07.506448 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.506412    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:07.515324 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.514160    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 20 16:31:07.519277 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.519257    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.522280 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.522262    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 20 16:31:07.522407 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.522264    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 20 16:31:07.522602 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.522586    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 20 16:31:07.522808 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.522793    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 20 16:31:07.523069 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.523047    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 20 16:31:07.523140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.523071    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-97dtt1krptne1\""
Jul 20 16:31:07.523561 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.523543    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 20 16:31:07.523652 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.523564    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-9z4d7\""
Jul 20 16:31:07.523957 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.523938    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 20 16:31:07.524692 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.524498    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 20 16:31:07.524692 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.524537    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 20 16:31:07.524692 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.524540    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 20 16:31:07.524692 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.524501    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 20 16:31:07.524692 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.524581    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 20 16:31:07.524692 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.524662    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 20 16:31:07.525006 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.524728    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 20 16:31:07.525687 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.525669    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 20 16:31:07.532888 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.532870    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 20 16:31:07.619704 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.619667    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.619704 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.619712    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.620001 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.619738    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.620001 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.619764    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.620001 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.619819    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-web-config\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.620001 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.619848    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-config\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.620001 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.619872    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.620001 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.619901    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.620001 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.619931    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.620001 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.619956    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.620001 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.619986    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.620471 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.620031    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.620471 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.620093    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4thm6\" (UniqueName: \"kubernetes.io/projected/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-kube-api-access-4thm6\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.620471 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.620460    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.620756 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.620720    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.620756 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.620753    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.620950 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.620802    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-config-out\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.620950 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.620847    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.620950 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.620886    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.620950 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.620918    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.703100 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.703011    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-drtfq" event={"ID":"112aad28-f9b6-415e-ac47-3334d4c2a15e","Type":"ContainerStarted","Data":"356f2158cad336057f7d45ab2a639f0f4697792bce9f3fd0dd4b68d5ef565671"}
Jul 20 16:31:07.703100 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.703085    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-drtfq" event={"ID":"112aad28-f9b6-415e-ac47-3334d4c2a15e","Type":"ContainerStarted","Data":"9158c343cdff7105e84d8c8cc469b016520d8a735f1b654bd4c7a0cfecca3323"}
Jul 20 16:31:07.722027 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.721990    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.722193 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.722034    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.722193 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.722067    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.722193 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.722092    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.722193 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.722115    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-web-config\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.722193 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.722140    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-config\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.722193 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.722164    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.722193 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.722194    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.722563 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.722222    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.722563 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.722248    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.722563 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.722273    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.722563 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.722311    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.722563 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.722334    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4thm6\" (UniqueName: \"kubernetes.io/projected/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-kube-api-access-4thm6\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.722563 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.722375    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.722563 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.722410    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.722563 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.722438    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.722563 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.722465    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-config-out\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.722563 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.722495    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.722563 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.722525    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.722563 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.722555    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.723815 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.722857    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.723815 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.723269    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.727024 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.726998    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.727315 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.727273    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.727515 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.727491    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.727641 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.727620    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-config\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.727983 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.727961    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.727983 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.727972    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.728260 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.728238    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.728616 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.728446    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.728723 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.728534    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.728863 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.728838    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-config-out\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.728953 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.728918    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.729009 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.728971    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.729186 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.729168    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-web-config\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.729531 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.729497    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.730200 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.730164    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.730428 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.730409    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.731556 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.731534    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.745402 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.745357    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4thm6\" (UniqueName: \"kubernetes.io/projected/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-kube-api-access-4thm6\") pod \"prometheus-k8s-0\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.763469 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.763442    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:07.829404 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.829371    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:07.977702 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.977596    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-drtfq" podStartSLOduration=2.201690335 podStartE2EDuration="6.977573337s" totalImagesPullingTime="4.69668442s" totalInitContainerRuntime="79.198582ms" isStatefulPod=true podCreationTimestamp="2026-07-20 16:31:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:31:07.727238402 +0000 UTC m=+90.976732422" watchObservedRunningTime="2026-07-20 16:31:07.977573337 +0000 UTC m=+91.227067365"
Jul 20 16:31:07.979155 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:07.979136    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 20 16:31:07.981004 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:31:07.980976    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5368eaf_95e4_4bfc_96fa_59dd21754d8a.slice/crio-4a9a57621c5fa8d3442faa97bb0bf3716f431196a13b2ec1948ac3f3aed60097 WatchSource:0}: Error finding container 4a9a57621c5fa8d3442faa97bb0bf3716f431196a13b2ec1948ac3f3aed60097: Status 404 returned error can't find the container with id 4a9a57621c5fa8d3442faa97bb0bf3716f431196a13b2ec1948ac3f3aed60097
Jul 20 16:31:08.708746 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:08.708708    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5368eaf-95e4-4bfc-96fa-59dd21754d8a","Type":"ContainerStarted","Data":"4a9a57621c5fa8d3442faa97bb0bf3716f431196a13b2ec1948ac3f3aed60097"}
Jul 20 16:31:08.869005 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:08.868966    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-845dd6cc47-c78k5"
Jul 20 16:31:08.869175 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:08.869040    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-845dd6cc47-c78k5"
Jul 20 16:31:08.874533 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:08.874508    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-845dd6cc47-c78k5"
Jul 20 16:31:09.615142 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:09.615103    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-c685fb89f-tgvnt"
Jul 20 16:31:09.717240 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:09.717174    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-845dd6cc47-c78k5"
Jul 20 16:31:09.736822 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:09.736789    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:10.050788 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:10.050753    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:10.379994 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:10.379904    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:11.621512 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:11.621463    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-59985f8c96-m44rk" podUID="bf2f8258-be74-4321-8a3f-5b8bcd376065" containerName="registry" containerID="cri-o://f77a83d0c41ba499bdac4af17b5e6756609c42e6bb232c4a5fbc56af90a3515e" gracePeriod=30
Jul 20 16:31:11.723489 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:11.723447    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bd59c" event={"ID":"3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8","Type":"ContainerStarted","Data":"cbc8babae9854ed2a2e0441f6c770edc795131f5d1ce20baed73a9f8659e5f1d"}
Jul 20 16:31:11.724941 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:11.724912    2577 generic.go:356] "Generic (PLEG): container finished" podID="f5368eaf-95e4-4bfc-96fa-59dd21754d8a" containerID="249932bf3ed9c3b6fa5fe17aafec74306d2582023cfb85c5cff769cba69a2a76" exitCode=0
Jul 20 16:31:11.725159 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:11.724951    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5368eaf-95e4-4bfc-96fa-59dd21754d8a","Type":"ContainerDied","Data":"249932bf3ed9c3b6fa5fe17aafec74306d2582023cfb85c5cff769cba69a2a76"}
Jul 20 16:31:11.741120 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:11.741069    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-bd59c" podStartSLOduration=33.363696028 podStartE2EDuration="49.741054821s" totalImagesPullingTime="16.377358793s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 16:30:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:31:11.739511513 +0000 UTC m=+94.989005533" watchObservedRunningTime="2026-07-20 16:31:11.741054821 +0000 UTC m=+94.990548840"
Jul 20 16:31:11.850398 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:11.850377    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:31:11.966333 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:11.966291    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bf2f8258-be74-4321-8a3f-5b8bcd376065-image-registry-private-configuration\") pod \"bf2f8258-be74-4321-8a3f-5b8bcd376065\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") "
Jul 20 16:31:11.966426 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:11.966355    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2cggs\" (UniqueName: \"kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-kube-api-access-2cggs\") pod \"bf2f8258-be74-4321-8a3f-5b8bcd376065\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") "
Jul 20 16:31:11.966426 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:11.966395    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bf2f8258-be74-4321-8a3f-5b8bcd376065-trusted-ca\") pod \"bf2f8258-be74-4321-8a3f-5b8bcd376065\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") "
Jul 20 16:31:11.966426 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:11.966416    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bf2f8258-be74-4321-8a3f-5b8bcd376065-ca-trust-extracted\") pod \"bf2f8258-be74-4321-8a3f-5b8bcd376065\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") "
Jul 20 16:31:11.966544 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:11.966454    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bf2f8258-be74-4321-8a3f-5b8bcd376065-installation-pull-secrets\") pod \"bf2f8258-be74-4321-8a3f-5b8bcd376065\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") "
Jul 20 16:31:11.966544 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:11.966503    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-bound-sa-token\") pod \"bf2f8258-be74-4321-8a3f-5b8bcd376065\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") "
Jul 20 16:31:11.966637 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:11.966544    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bf2f8258-be74-4321-8a3f-5b8bcd376065-registry-certificates\") pod \"bf2f8258-be74-4321-8a3f-5b8bcd376065\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") "
Jul 20 16:31:11.966637 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:11.966568    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-registry-tls\") pod \"bf2f8258-be74-4321-8a3f-5b8bcd376065\" (UID: \"bf2f8258-be74-4321-8a3f-5b8bcd376065\") "
Jul 20 16:31:11.966945 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:11.966913    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf2f8258-be74-4321-8a3f-5b8bcd376065-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "bf2f8258-be74-4321-8a3f-5b8bcd376065" (UID: "bf2f8258-be74-4321-8a3f-5b8bcd376065"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:31:11.967052 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:11.966992    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf2f8258-be74-4321-8a3f-5b8bcd376065-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "bf2f8258-be74-4321-8a3f-5b8bcd376065" (UID: "bf2f8258-be74-4321-8a3f-5b8bcd376065"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:31:11.968874 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:11.968844    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf2f8258-be74-4321-8a3f-5b8bcd376065-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "bf2f8258-be74-4321-8a3f-5b8bcd376065" (UID: "bf2f8258-be74-4321-8a3f-5b8bcd376065"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:31:11.969013 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:11.968979    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf2f8258-be74-4321-8a3f-5b8bcd376065-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "bf2f8258-be74-4321-8a3f-5b8bcd376065" (UID: "bf2f8258-be74-4321-8a3f-5b8bcd376065"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:31:11.969117 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:11.969046    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-kube-api-access-2cggs" (OuterVolumeSpecName: "kube-api-access-2cggs") pod "bf2f8258-be74-4321-8a3f-5b8bcd376065" (UID: "bf2f8258-be74-4321-8a3f-5b8bcd376065"). InnerVolumeSpecName "kube-api-access-2cggs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:31:11.969157 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:11.969133    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "bf2f8258-be74-4321-8a3f-5b8bcd376065" (UID: "bf2f8258-be74-4321-8a3f-5b8bcd376065"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:31:11.969363 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:11.969338    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "bf2f8258-be74-4321-8a3f-5b8bcd376065" (UID: "bf2f8258-be74-4321-8a3f-5b8bcd376065"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:31:11.977407 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:11.977381    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf2f8258-be74-4321-8a3f-5b8bcd376065-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "bf2f8258-be74-4321-8a3f-5b8bcd376065" (UID: "bf2f8258-be74-4321-8a3f-5b8bcd376065"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:31:12.068147 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:12.068105    2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bf2f8258-be74-4321-8a3f-5b8bcd376065-registry-certificates\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:31:12.068147 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:12.068140    2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-registry-tls\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:31:12.068147 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:12.068156    2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bf2f8258-be74-4321-8a3f-5b8bcd376065-image-registry-private-configuration\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:31:12.068403 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:12.068169    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2cggs\" (UniqueName: \"kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-kube-api-access-2cggs\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:31:12.068403 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:12.068180    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bf2f8258-be74-4321-8a3f-5b8bcd376065-trusted-ca\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:31:12.068403 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:12.068192    2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bf2f8258-be74-4321-8a3f-5b8bcd376065-ca-trust-extracted\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:31:12.068403 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:12.068202    2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bf2f8258-be74-4321-8a3f-5b8bcd376065-installation-pull-secrets\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:31:12.068403 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:12.068214    2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bf2f8258-be74-4321-8a3f-5b8bcd376065-bound-sa-token\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:31:12.730170 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:12.730130    2577 generic.go:356] "Generic (PLEG): container finished" podID="bf2f8258-be74-4321-8a3f-5b8bcd376065" containerID="f77a83d0c41ba499bdac4af17b5e6756609c42e6bb232c4a5fbc56af90a3515e" exitCode=0
Jul 20 16:31:12.730643 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:12.730200    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59985f8c96-m44rk"
Jul 20 16:31:12.730643 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:12.730224    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59985f8c96-m44rk" event={"ID":"bf2f8258-be74-4321-8a3f-5b8bcd376065","Type":"ContainerDied","Data":"f77a83d0c41ba499bdac4af17b5e6756609c42e6bb232c4a5fbc56af90a3515e"}
Jul 20 16:31:12.730643 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:12.730288    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59985f8c96-m44rk" event={"ID":"bf2f8258-be74-4321-8a3f-5b8bcd376065","Type":"ContainerDied","Data":"ccba56ad553851d994b2652ff0c9c670e96ca1cd285af232219343b4e3832be9"}
Jul 20 16:31:12.730643 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:12.730329    2577 scope.go:117] "RemoveContainer" containerID="f77a83d0c41ba499bdac4af17b5e6756609c42e6bb232c4a5fbc56af90a3515e"
Jul 20 16:31:12.740275 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:12.740251    2577 scope.go:117] "RemoveContainer" containerID="f77a83d0c41ba499bdac4af17b5e6756609c42e6bb232c4a5fbc56af90a3515e"
Jul 20 16:31:12.740631 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:31:12.740600    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f77a83d0c41ba499bdac4af17b5e6756609c42e6bb232c4a5fbc56af90a3515e\": container with ID starting with f77a83d0c41ba499bdac4af17b5e6756609c42e6bb232c4a5fbc56af90a3515e not found: ID does not exist" containerID="f77a83d0c41ba499bdac4af17b5e6756609c42e6bb232c4a5fbc56af90a3515e"
Jul 20 16:31:12.740726 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:12.740641    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f77a83d0c41ba499bdac4af17b5e6756609c42e6bb232c4a5fbc56af90a3515e"} err="failed to get container status \"f77a83d0c41ba499bdac4af17b5e6756609c42e6bb232c4a5fbc56af90a3515e\": rpc error: code = NotFound desc = could not find container \"f77a83d0c41ba499bdac4af17b5e6756609c42e6bb232c4a5fbc56af90a3515e\": container with ID starting with f77a83d0c41ba499bdac4af17b5e6756609c42e6bb232c4a5fbc56af90a3515e not found: ID does not exist"
Jul 20 16:31:12.755551 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:12.755526    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-59985f8c96-m44rk"]
Jul 20 16:31:12.759156 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:12.759135    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-59985f8c96-m44rk"]
Jul 20 16:31:13.211551 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:13.211520    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf2f8258-be74-4321-8a3f-5b8bcd376065" path="/var/lib/kubelet/pods/bf2f8258-be74-4321-8a3f-5b8bcd376065/volumes"
Jul 20 16:31:14.107481 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:14.107446    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:14.166170 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:14.166142    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:15.748659 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:15.747926    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5368eaf-95e4-4bfc-96fa-59dd21754d8a","Type":"ContainerStarted","Data":"4268c38be8ef1fb6299e5b8f56b6d08e298ff6c36e1080fb3da06bae1824e9d9"}
Jul 20 16:31:15.748659 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:15.748449    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5368eaf-95e4-4bfc-96fa-59dd21754d8a","Type":"ContainerStarted","Data":"c15821f8bf553226caa65338f40007d64179be097dd7510bbd630ece5dfbcb06"}
Jul 20 16:31:16.742111 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:16.742022    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:16.795575 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:16.795542    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:17.759880 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:17.759844    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5368eaf-95e4-4bfc-96fa-59dd21754d8a","Type":"ContainerStarted","Data":"963135ce45253063d3832b0b8e50a4a1d6316f6f71da36fe89c05d372a024857"}
Jul 20 16:31:17.760079 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:17.759905    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5368eaf-95e4-4bfc-96fa-59dd21754d8a","Type":"ContainerStarted","Data":"7cc771d5915942c91ffb311b34d1a1006b14fe81246425d2e8f380bd2fe3df05"}
Jul 20 16:31:17.760079 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:17.759933    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5368eaf-95e4-4bfc-96fa-59dd21754d8a","Type":"ContainerStarted","Data":"ff0731116d8e6837b85980eb49ecd86bedef6f72712996b12dfe347b7468544e"}
Jul 20 16:31:17.760079 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:17.759957    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5368eaf-95e4-4bfc-96fa-59dd21754d8a","Type":"ContainerStarted","Data":"38198978ab1410d6482e0667d3d6ba3464031d07ba0c2f78b539d17ecbd69be2"}
Jul 20 16:31:17.789655 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:17.789576    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.8298412920000002 podStartE2EDuration="10.78955527s" totalImagesPullingTime="7.94179922s" totalInitContainerRuntime="17.914758ms" isStatefulPod=false podCreationTimestamp="2026-07-20 16:31:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:31:17.787013524 +0000 UTC m=+101.036507544" watchObservedRunningTime="2026-07-20 16:31:17.78955527 +0000 UTC m=+101.039049290"
Jul 20 16:31:17.829484 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:17.829438    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:31:18.953992 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:18.953956    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:19.612322 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:19.612275    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-ss6qh"
Jul 20 16:31:19.742116 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:19.742052    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:19.792413 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:19.792365    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:21.144938 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:21.144904    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:21.248857 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:21.248800    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:21.336742 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:21.336707    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:28.524371 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:28.524331    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-845dd6cc47-c78k5"]
Jul 20 16:31:28.687712 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:28.687681    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:28.790581 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:28.790503    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:28.859172 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:28.859141    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:36.226013 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:36.225949    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:36.285458 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:36.285423    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:43.240065 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:43.240032    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:43.336766 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:43.336728    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:43.391559 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:43.391532    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:31:53.544288 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:53.544242    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-845dd6cc47-c78k5" podUID="33aa27a1-16e2-4e72-b8c4-7d76abdb2e51" containerName="console" containerID="cri-o://d5d28b15ef72ba1e66378aee0ccb9bb18269d7255edc96520e4bec317b04773d" gracePeriod=15
Jul 20 16:31:53.789431 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:53.789401    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-845dd6cc47-c78k5"
Jul 20 16:31:53.890875 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:53.890790    2577 generic.go:356] "Generic (PLEG): container finished" podID="33aa27a1-16e2-4e72-b8c4-7d76abdb2e51" containerID="d5d28b15ef72ba1e66378aee0ccb9bb18269d7255edc96520e4bec317b04773d" exitCode=0
Jul 20 16:31:53.890875 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:53.890843    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-845dd6cc47-c78k5"
Jul 20 16:31:53.891118 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:53.890878    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-845dd6cc47-c78k5" event={"ID":"33aa27a1-16e2-4e72-b8c4-7d76abdb2e51","Type":"ContainerDied","Data":"d5d28b15ef72ba1e66378aee0ccb9bb18269d7255edc96520e4bec317b04773d"}
Jul 20 16:31:53.891118 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:53.890964    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-845dd6cc47-c78k5" event={"ID":"33aa27a1-16e2-4e72-b8c4-7d76abdb2e51","Type":"ContainerDied","Data":"9779176fe4c8bad8ffd922e51a001ec54170d6986ed8f30583a9773a4158313c"}
Jul 20 16:31:53.891118 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:53.890992    2577 scope.go:117] "RemoveContainer" containerID="d5d28b15ef72ba1e66378aee0ccb9bb18269d7255edc96520e4bec317b04773d"
Jul 20 16:31:53.898480 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:53.898464    2577 scope.go:117] "RemoveContainer" containerID="d5d28b15ef72ba1e66378aee0ccb9bb18269d7255edc96520e4bec317b04773d"
Jul 20 16:31:53.898703 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:31:53.898686    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5d28b15ef72ba1e66378aee0ccb9bb18269d7255edc96520e4bec317b04773d\": container with ID starting with d5d28b15ef72ba1e66378aee0ccb9bb18269d7255edc96520e4bec317b04773d not found: ID does not exist" containerID="d5d28b15ef72ba1e66378aee0ccb9bb18269d7255edc96520e4bec317b04773d"
Jul 20 16:31:53.898742 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:53.898709    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5d28b15ef72ba1e66378aee0ccb9bb18269d7255edc96520e4bec317b04773d"} err="failed to get container status \"d5d28b15ef72ba1e66378aee0ccb9bb18269d7255edc96520e4bec317b04773d\": rpc error: code = NotFound desc = could not find container \"d5d28b15ef72ba1e66378aee0ccb9bb18269d7255edc96520e4bec317b04773d\": container with ID starting with d5d28b15ef72ba1e66378aee0ccb9bb18269d7255edc96520e4bec317b04773d not found: ID does not exist"
Jul 20 16:31:53.913881 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:53.913865    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-console-serving-cert\") pod \"33aa27a1-16e2-4e72-b8c4-7d76abdb2e51\" (UID: \"33aa27a1-16e2-4e72-b8c4-7d76abdb2e51\") "
Jul 20 16:31:53.913951 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:53.913890    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bb7rx\" (UniqueName: \"kubernetes.io/projected/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-kube-api-access-bb7rx\") pod \"33aa27a1-16e2-4e72-b8c4-7d76abdb2e51\" (UID: \"33aa27a1-16e2-4e72-b8c4-7d76abdb2e51\") "
Jul 20 16:31:53.913951 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:53.913929    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-oauth-serving-cert\") pod \"33aa27a1-16e2-4e72-b8c4-7d76abdb2e51\" (UID: \"33aa27a1-16e2-4e72-b8c4-7d76abdb2e51\") "
Jul 20 16:31:53.914058 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:53.914045    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-service-ca\") pod \"33aa27a1-16e2-4e72-b8c4-7d76abdb2e51\" (UID: \"33aa27a1-16e2-4e72-b8c4-7d76abdb2e51\") "
Jul 20 16:31:53.914109 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:53.914077    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-console-oauth-config\") pod \"33aa27a1-16e2-4e72-b8c4-7d76abdb2e51\" (UID: \"33aa27a1-16e2-4e72-b8c4-7d76abdb2e51\") "
Jul 20 16:31:53.914314 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:53.914264    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-console-config\") pod \"33aa27a1-16e2-4e72-b8c4-7d76abdb2e51\" (UID: \"33aa27a1-16e2-4e72-b8c4-7d76abdb2e51\") "
Jul 20 16:31:53.914433 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:53.914334    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "33aa27a1-16e2-4e72-b8c4-7d76abdb2e51" (UID: "33aa27a1-16e2-4e72-b8c4-7d76abdb2e51"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:31:53.914433 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:53.914419    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-service-ca" (OuterVolumeSpecName: "service-ca") pod "33aa27a1-16e2-4e72-b8c4-7d76abdb2e51" (UID: "33aa27a1-16e2-4e72-b8c4-7d76abdb2e51"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:31:53.914556 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:53.914520    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-oauth-serving-cert\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:31:53.914611 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:53.914563    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-service-ca\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:31:53.914691 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:53.914668    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-console-config" (OuterVolumeSpecName: "console-config") pod "33aa27a1-16e2-4e72-b8c4-7d76abdb2e51" (UID: "33aa27a1-16e2-4e72-b8c4-7d76abdb2e51"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:31:53.916039 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:53.916019    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "33aa27a1-16e2-4e72-b8c4-7d76abdb2e51" (UID: "33aa27a1-16e2-4e72-b8c4-7d76abdb2e51"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:31:53.916435 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:53.916414    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-kube-api-access-bb7rx" (OuterVolumeSpecName: "kube-api-access-bb7rx") pod "33aa27a1-16e2-4e72-b8c4-7d76abdb2e51" (UID: "33aa27a1-16e2-4e72-b8c4-7d76abdb2e51"). InnerVolumeSpecName "kube-api-access-bb7rx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:31:53.916487 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:53.916416    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "33aa27a1-16e2-4e72-b8c4-7d76abdb2e51" (UID: "33aa27a1-16e2-4e72-b8c4-7d76abdb2e51"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:31:54.015519 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:54.015489    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-console-config\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:31:54.015519 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:54.015517    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-console-serving-cert\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:31:54.015666 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:54.015531    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bb7rx\" (UniqueName: \"kubernetes.io/projected/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-kube-api-access-bb7rx\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:31:54.015666 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:54.015544    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51-console-oauth-config\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:31:54.211489 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:54.211462    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-845dd6cc47-c78k5"]
Jul 20 16:31:54.215045 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:54.215020    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-845dd6cc47-c78k5"]
Jul 20 16:31:55.210922 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:31:55.210889    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33aa27a1-16e2-4e72-b8c4-7d76abdb2e51" path="/var/lib/kubelet/pods/33aa27a1-16e2-4e72-b8c4-7d76abdb2e51/volumes"
Jul 20 16:32:07.829856 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:07.829818    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:07.848085 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:07.848065    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:07.957767 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:07.957739    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:20.569823 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:20.569787    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:32:20.962922 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:20.962835    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:32:25.950147 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:25.950113    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 20 16:32:25.950570 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:25.950443    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f5368eaf-95e4-4bfc-96fa-59dd21754d8a" containerName="prometheus" containerID="cri-o://c15821f8bf553226caa65338f40007d64179be097dd7510bbd630ece5dfbcb06" gracePeriod=600
Jul 20 16:32:25.950570 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:25.950478    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f5368eaf-95e4-4bfc-96fa-59dd21754d8a" containerName="config-reloader" containerID="cri-o://4268c38be8ef1fb6299e5b8f56b6d08e298ff6c36e1080fb3da06bae1824e9d9" gracePeriod=600
Jul 20 16:32:25.950570 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:25.950498    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f5368eaf-95e4-4bfc-96fa-59dd21754d8a" containerName="kube-rbac-proxy-web" containerID="cri-o://ff0731116d8e6837b85980eb49ecd86bedef6f72712996b12dfe347b7468544e" gracePeriod=600
Jul 20 16:32:25.950570 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:25.950510    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f5368eaf-95e4-4bfc-96fa-59dd21754d8a" containerName="kube-rbac-proxy" containerID="cri-o://7cc771d5915942c91ffb311b34d1a1006b14fe81246425d2e8f380bd2fe3df05" gracePeriod=600
Jul 20 16:32:25.950763 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:25.950466    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f5368eaf-95e4-4bfc-96fa-59dd21754d8a" containerName="kube-rbac-proxy-thanos" containerID="cri-o://963135ce45253063d3832b0b8e50a4a1d6316f6f71da36fe89c05d372a024857" gracePeriod=600
Jul 20 16:32:25.950763 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:25.950476    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f5368eaf-95e4-4bfc-96fa-59dd21754d8a" containerName="thanos-sidecar" containerID="cri-o://38198978ab1410d6482e0667d3d6ba3464031d07ba0c2f78b539d17ecbd69be2" gracePeriod=600
Jul 20 16:32:27.013718 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.013687    2577 generic.go:356] "Generic (PLEG): container finished" podID="f5368eaf-95e4-4bfc-96fa-59dd21754d8a" containerID="963135ce45253063d3832b0b8e50a4a1d6316f6f71da36fe89c05d372a024857" exitCode=0
Jul 20 16:32:27.013718 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.013711    2577 generic.go:356] "Generic (PLEG): container finished" podID="f5368eaf-95e4-4bfc-96fa-59dd21754d8a" containerID="7cc771d5915942c91ffb311b34d1a1006b14fe81246425d2e8f380bd2fe3df05" exitCode=0
Jul 20 16:32:27.013718 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.013720    2577 generic.go:356] "Generic (PLEG): container finished" podID="f5368eaf-95e4-4bfc-96fa-59dd21754d8a" containerID="38198978ab1410d6482e0667d3d6ba3464031d07ba0c2f78b539d17ecbd69be2" exitCode=0
Jul 20 16:32:27.014157 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.013728    2577 generic.go:356] "Generic (PLEG): container finished" podID="f5368eaf-95e4-4bfc-96fa-59dd21754d8a" containerID="4268c38be8ef1fb6299e5b8f56b6d08e298ff6c36e1080fb3da06bae1824e9d9" exitCode=0
Jul 20 16:32:27.014157 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.013738    2577 generic.go:356] "Generic (PLEG): container finished" podID="f5368eaf-95e4-4bfc-96fa-59dd21754d8a" containerID="c15821f8bf553226caa65338f40007d64179be097dd7510bbd630ece5dfbcb06" exitCode=0
Jul 20 16:32:27.014157 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.013759    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5368eaf-95e4-4bfc-96fa-59dd21754d8a","Type":"ContainerDied","Data":"963135ce45253063d3832b0b8e50a4a1d6316f6f71da36fe89c05d372a024857"}
Jul 20 16:32:27.014157 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.013827    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5368eaf-95e4-4bfc-96fa-59dd21754d8a","Type":"ContainerDied","Data":"7cc771d5915942c91ffb311b34d1a1006b14fe81246425d2e8f380bd2fe3df05"}
Jul 20 16:32:27.014157 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.013854    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5368eaf-95e4-4bfc-96fa-59dd21754d8a","Type":"ContainerDied","Data":"38198978ab1410d6482e0667d3d6ba3464031d07ba0c2f78b539d17ecbd69be2"}
Jul 20 16:32:27.014157 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.013879    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5368eaf-95e4-4bfc-96fa-59dd21754d8a","Type":"ContainerDied","Data":"4268c38be8ef1fb6299e5b8f56b6d08e298ff6c36e1080fb3da06bae1824e9d9"}
Jul 20 16:32:27.014157 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.013903    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5368eaf-95e4-4bfc-96fa-59dd21754d8a","Type":"ContainerDied","Data":"c15821f8bf553226caa65338f40007d64179be097dd7510bbd630ece5dfbcb06"}
Jul 20 16:32:27.185973 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.185951    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:27.291965 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.291874    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-prometheus-k8s-rulefiles-1\") pod \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") "
Jul 20 16:32:27.291965 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.291922    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-configmap-metrics-client-ca\") pod \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") "
Jul 20 16:32:27.291965 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.291954    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-configmap-kubelet-serving-ca-bundle\") pod \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") "
Jul 20 16:32:27.292221 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.291978    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4thm6\" (UniqueName: \"kubernetes.io/projected/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-kube-api-access-4thm6\") pod \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") "
Jul 20 16:32:27.292221 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.292005    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") "
Jul 20 16:32:27.292221 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.292040    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-prometheus-k8s-rulefiles-2\") pod \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") "
Jul 20 16:32:27.292221 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.292069    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-prometheus-k8s-rulefiles-0\") pod \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") "
Jul 20 16:32:27.292221 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.292107    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-config\") pod \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") "
Jul 20 16:32:27.292221 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.292144    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-prometheus-k8s-db\") pod \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") "
Jul 20 16:32:27.292221 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.292170    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-web-config\") pod \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") "
Jul 20 16:32:27.292221 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.292195    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-kube-rbac-proxy\") pod \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") "
Jul 20 16:32:27.292641 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.292278    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "f5368eaf-95e4-4bfc-96fa-59dd21754d8a" (UID: "f5368eaf-95e4-4bfc-96fa-59dd21754d8a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:32:27.292641 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.292317    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "f5368eaf-95e4-4bfc-96fa-59dd21754d8a" (UID: "f5368eaf-95e4-4bfc-96fa-59dd21754d8a"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:32:27.292641 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.292357    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-thanos-prometheus-http-client-file\") pod \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") "
Jul 20 16:32:27.292641 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.292374    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "f5368eaf-95e4-4bfc-96fa-59dd21754d8a" (UID: "f5368eaf-95e4-4bfc-96fa-59dd21754d8a"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:32:27.292641 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.292425    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-prometheus-trusted-ca-bundle\") pod \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") "
Jul 20 16:32:27.292641 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.292455    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-grpc-tls\") pod \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") "
Jul 20 16:32:27.292641 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.292583    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-metrics-client-certs\") pod \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") "
Jul 20 16:32:27.292641 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.292615    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-config-out\") pod \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") "
Jul 20 16:32:27.293022 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.292649    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-tls-assets\") pod \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") "
Jul 20 16:32:27.293022 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.292687    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") "
Jul 20 16:32:27.293022 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.292727    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-configmap-serving-certs-ca-bundle\") pod \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") "
Jul 20 16:32:27.293022 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.292757    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-prometheus-k8s-tls\") pod \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\" (UID: \"f5368eaf-95e4-4bfc-96fa-59dd21754d8a\") "
Jul 20 16:32:27.293022 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.292930    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "f5368eaf-95e4-4bfc-96fa-59dd21754d8a" (UID: "f5368eaf-95e4-4bfc-96fa-59dd21754d8a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:32:27.293022 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.292990    2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:32:27.293022 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.293008    2577 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-configmap-metrics-client-ca\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:32:27.293389 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.293026    2577 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:32:27.293389 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.293042    2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:32:27.293681 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.293628    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "f5368eaf-95e4-4bfc-96fa-59dd21754d8a" (UID: "f5368eaf-95e4-4bfc-96fa-59dd21754d8a"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:32:27.294146 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.294118    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "f5368eaf-95e4-4bfc-96fa-59dd21754d8a" (UID: "f5368eaf-95e4-4bfc-96fa-59dd21754d8a"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:32:27.294895 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.294739    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-config" (OuterVolumeSpecName: "config") pod "f5368eaf-95e4-4bfc-96fa-59dd21754d8a" (UID: "f5368eaf-95e4-4bfc-96fa-59dd21754d8a"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:32:27.296344 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.295229    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-kube-api-access-4thm6" (OuterVolumeSpecName: "kube-api-access-4thm6") pod "f5368eaf-95e4-4bfc-96fa-59dd21754d8a" (UID: "f5368eaf-95e4-4bfc-96fa-59dd21754d8a"). InnerVolumeSpecName "kube-api-access-4thm6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:32:27.296344 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.295344    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "f5368eaf-95e4-4bfc-96fa-59dd21754d8a" (UID: "f5368eaf-95e4-4bfc-96fa-59dd21754d8a"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:32:27.296344 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.295609    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "f5368eaf-95e4-4bfc-96fa-59dd21754d8a" (UID: "f5368eaf-95e4-4bfc-96fa-59dd21754d8a"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:32:27.296344 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.295797    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "f5368eaf-95e4-4bfc-96fa-59dd21754d8a" (UID: "f5368eaf-95e4-4bfc-96fa-59dd21754d8a"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:32:27.296344 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.296142    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "f5368eaf-95e4-4bfc-96fa-59dd21754d8a" (UID: "f5368eaf-95e4-4bfc-96fa-59dd21754d8a"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:32:27.296344 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.296185    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "f5368eaf-95e4-4bfc-96fa-59dd21754d8a" (UID: "f5368eaf-95e4-4bfc-96fa-59dd21754d8a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:32:27.296784 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.296759    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "f5368eaf-95e4-4bfc-96fa-59dd21754d8a" (UID: "f5368eaf-95e4-4bfc-96fa-59dd21754d8a"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:32:27.297363 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.297335    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "f5368eaf-95e4-4bfc-96fa-59dd21754d8a" (UID: "f5368eaf-95e4-4bfc-96fa-59dd21754d8a"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:32:27.297559 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.297504    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "f5368eaf-95e4-4bfc-96fa-59dd21754d8a" (UID: "f5368eaf-95e4-4bfc-96fa-59dd21754d8a"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:32:27.297715 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.297690    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "f5368eaf-95e4-4bfc-96fa-59dd21754d8a" (UID: "f5368eaf-95e4-4bfc-96fa-59dd21754d8a"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:32:27.298074 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.298046    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-config-out" (OuterVolumeSpecName: "config-out") pod "f5368eaf-95e4-4bfc-96fa-59dd21754d8a" (UID: "f5368eaf-95e4-4bfc-96fa-59dd21754d8a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:32:27.298228 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.298208    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "f5368eaf-95e4-4bfc-96fa-59dd21754d8a" (UID: "f5368eaf-95e4-4bfc-96fa-59dd21754d8a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:32:27.305337 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.305311    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-web-config" (OuterVolumeSpecName: "web-config") pod "f5368eaf-95e4-4bfc-96fa-59dd21754d8a" (UID: "f5368eaf-95e4-4bfc-96fa-59dd21754d8a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:32:27.393822 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.393775    2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-prometheus-trusted-ca-bundle\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:32:27.393822 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.393819    2577 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-grpc-tls\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:32:27.393822 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.393832    2577 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-metrics-client-certs\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:32:27.394068 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.393846    2577 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-config-out\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:32:27.394068 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.393857    2577 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-tls-assets\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:32:27.394068 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.393869    2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:32:27.394068 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.393882    2577 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:32:27.394068 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.393894    2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-prometheus-k8s-tls\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:32:27.394068 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.393907    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4thm6\" (UniqueName: \"kubernetes.io/projected/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-kube-api-access-4thm6\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:32:27.394068 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.393919    2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:32:27.394068 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.393932    2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:32:27.394068 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.393948    2577 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-config\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:32:27.394068 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.393960    2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-prometheus-k8s-db\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:32:27.394068 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.393971    2577 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-web-config\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:32:27.394068 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.393986    2577 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-secret-kube-rbac-proxy\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:32:27.394068 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:27.393998    2577 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f5368eaf-95e4-4bfc-96fa-59dd21754d8a-thanos-prometheus-http-client-file\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:32:28.020096 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.020060    2577 generic.go:356] "Generic (PLEG): container finished" podID="f5368eaf-95e4-4bfc-96fa-59dd21754d8a" containerID="ff0731116d8e6837b85980eb49ecd86bedef6f72712996b12dfe347b7468544e" exitCode=0
Jul 20 16:32:28.020566 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.020127    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5368eaf-95e4-4bfc-96fa-59dd21754d8a","Type":"ContainerDied","Data":"ff0731116d8e6837b85980eb49ecd86bedef6f72712996b12dfe347b7468544e"}
Jul 20 16:32:28.020566 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.020132    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.020566 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.020204    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5368eaf-95e4-4bfc-96fa-59dd21754d8a","Type":"ContainerDied","Data":"4a9a57621c5fa8d3442faa97bb0bf3716f431196a13b2ec1948ac3f3aed60097"}
Jul 20 16:32:28.020566 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.020259    2577 scope.go:117] "RemoveContainer" containerID="963135ce45253063d3832b0b8e50a4a1d6316f6f71da36fe89c05d372a024857"
Jul 20 16:32:28.027609 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.027579    2577 scope.go:117] "RemoveContainer" containerID="7cc771d5915942c91ffb311b34d1a1006b14fe81246425d2e8f380bd2fe3df05"
Jul 20 16:32:28.034675 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.034659    2577 scope.go:117] "RemoveContainer" containerID="ff0731116d8e6837b85980eb49ecd86bedef6f72712996b12dfe347b7468544e"
Jul 20 16:32:28.041159 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.041143    2577 scope.go:117] "RemoveContainer" containerID="38198978ab1410d6482e0667d3d6ba3464031d07ba0c2f78b539d17ecbd69be2"
Jul 20 16:32:28.047283 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.047250    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 20 16:32:28.050982 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.050952    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 20 16:32:28.053913 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.053893    2577 scope.go:117] "RemoveContainer" containerID="4268c38be8ef1fb6299e5b8f56b6d08e298ff6c36e1080fb3da06bae1824e9d9"
Jul 20 16:32:28.060130 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.060114    2577 scope.go:117] "RemoveContainer" containerID="c15821f8bf553226caa65338f40007d64179be097dd7510bbd630ece5dfbcb06"
Jul 20 16:32:28.067079 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.066932    2577 scope.go:117] "RemoveContainer" containerID="249932bf3ed9c3b6fa5fe17aafec74306d2582023cfb85c5cff769cba69a2a76"
Jul 20 16:32:28.073010 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.072990    2577 scope.go:117] "RemoveContainer" containerID="963135ce45253063d3832b0b8e50a4a1d6316f6f71da36fe89c05d372a024857"
Jul 20 16:32:28.073369 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:32:28.073265    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"963135ce45253063d3832b0b8e50a4a1d6316f6f71da36fe89c05d372a024857\": container with ID starting with 963135ce45253063d3832b0b8e50a4a1d6316f6f71da36fe89c05d372a024857 not found: ID does not exist" containerID="963135ce45253063d3832b0b8e50a4a1d6316f6f71da36fe89c05d372a024857"
Jul 20 16:32:28.073369 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.073332    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"963135ce45253063d3832b0b8e50a4a1d6316f6f71da36fe89c05d372a024857"} err="failed to get container status \"963135ce45253063d3832b0b8e50a4a1d6316f6f71da36fe89c05d372a024857\": rpc error: code = NotFound desc = could not find container \"963135ce45253063d3832b0b8e50a4a1d6316f6f71da36fe89c05d372a024857\": container with ID starting with 963135ce45253063d3832b0b8e50a4a1d6316f6f71da36fe89c05d372a024857 not found: ID does not exist"
Jul 20 16:32:28.073369 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.073361    2577 scope.go:117] "RemoveContainer" containerID="7cc771d5915942c91ffb311b34d1a1006b14fe81246425d2e8f380bd2fe3df05"
Jul 20 16:32:28.073666 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:32:28.073641    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cc771d5915942c91ffb311b34d1a1006b14fe81246425d2e8f380bd2fe3df05\": container with ID starting with 7cc771d5915942c91ffb311b34d1a1006b14fe81246425d2e8f380bd2fe3df05 not found: ID does not exist" containerID="7cc771d5915942c91ffb311b34d1a1006b14fe81246425d2e8f380bd2fe3df05"
Jul 20 16:32:28.073743 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.073676    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cc771d5915942c91ffb311b34d1a1006b14fe81246425d2e8f380bd2fe3df05"} err="failed to get container status \"7cc771d5915942c91ffb311b34d1a1006b14fe81246425d2e8f380bd2fe3df05\": rpc error: code = NotFound desc = could not find container \"7cc771d5915942c91ffb311b34d1a1006b14fe81246425d2e8f380bd2fe3df05\": container with ID starting with 7cc771d5915942c91ffb311b34d1a1006b14fe81246425d2e8f380bd2fe3df05 not found: ID does not exist"
Jul 20 16:32:28.073743 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.073698    2577 scope.go:117] "RemoveContainer" containerID="ff0731116d8e6837b85980eb49ecd86bedef6f72712996b12dfe347b7468544e"
Jul 20 16:32:28.073987 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:32:28.073969    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff0731116d8e6837b85980eb49ecd86bedef6f72712996b12dfe347b7468544e\": container with ID starting with ff0731116d8e6837b85980eb49ecd86bedef6f72712996b12dfe347b7468544e not found: ID does not exist" containerID="ff0731116d8e6837b85980eb49ecd86bedef6f72712996b12dfe347b7468544e"
Jul 20 16:32:28.074036 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.073992    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff0731116d8e6837b85980eb49ecd86bedef6f72712996b12dfe347b7468544e"} err="failed to get container status \"ff0731116d8e6837b85980eb49ecd86bedef6f72712996b12dfe347b7468544e\": rpc error: code = NotFound desc = could not find container \"ff0731116d8e6837b85980eb49ecd86bedef6f72712996b12dfe347b7468544e\": container with ID starting with ff0731116d8e6837b85980eb49ecd86bedef6f72712996b12dfe347b7468544e not found: ID does not exist"
Jul 20 16:32:28.074036 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.074006    2577 scope.go:117] "RemoveContainer" containerID="38198978ab1410d6482e0667d3d6ba3464031d07ba0c2f78b539d17ecbd69be2"
Jul 20 16:32:28.074267 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:32:28.074244    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38198978ab1410d6482e0667d3d6ba3464031d07ba0c2f78b539d17ecbd69be2\": container with ID starting with 38198978ab1410d6482e0667d3d6ba3464031d07ba0c2f78b539d17ecbd69be2 not found: ID does not exist" containerID="38198978ab1410d6482e0667d3d6ba3464031d07ba0c2f78b539d17ecbd69be2"
Jul 20 16:32:28.074428 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.074270    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38198978ab1410d6482e0667d3d6ba3464031d07ba0c2f78b539d17ecbd69be2"} err="failed to get container status \"38198978ab1410d6482e0667d3d6ba3464031d07ba0c2f78b539d17ecbd69be2\": rpc error: code = NotFound desc = could not find container \"38198978ab1410d6482e0667d3d6ba3464031d07ba0c2f78b539d17ecbd69be2\": container with ID starting with 38198978ab1410d6482e0667d3d6ba3464031d07ba0c2f78b539d17ecbd69be2 not found: ID does not exist"
Jul 20 16:32:28.074428 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.074283    2577 scope.go:117] "RemoveContainer" containerID="4268c38be8ef1fb6299e5b8f56b6d08e298ff6c36e1080fb3da06bae1824e9d9"
Jul 20 16:32:28.074581 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:32:28.074560    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4268c38be8ef1fb6299e5b8f56b6d08e298ff6c36e1080fb3da06bae1824e9d9\": container with ID starting with 4268c38be8ef1fb6299e5b8f56b6d08e298ff6c36e1080fb3da06bae1824e9d9 not found: ID does not exist" containerID="4268c38be8ef1fb6299e5b8f56b6d08e298ff6c36e1080fb3da06bae1824e9d9"
Jul 20 16:32:28.074618 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.074589    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4268c38be8ef1fb6299e5b8f56b6d08e298ff6c36e1080fb3da06bae1824e9d9"} err="failed to get container status \"4268c38be8ef1fb6299e5b8f56b6d08e298ff6c36e1080fb3da06bae1824e9d9\": rpc error: code = NotFound desc = could not find container \"4268c38be8ef1fb6299e5b8f56b6d08e298ff6c36e1080fb3da06bae1824e9d9\": container with ID starting with 4268c38be8ef1fb6299e5b8f56b6d08e298ff6c36e1080fb3da06bae1824e9d9 not found: ID does not exist"
Jul 20 16:32:28.074618 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.074609    2577 scope.go:117] "RemoveContainer" containerID="c15821f8bf553226caa65338f40007d64179be097dd7510bbd630ece5dfbcb06"
Jul 20 16:32:28.074834 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:32:28.074819    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c15821f8bf553226caa65338f40007d64179be097dd7510bbd630ece5dfbcb06\": container with ID starting with c15821f8bf553226caa65338f40007d64179be097dd7510bbd630ece5dfbcb06 not found: ID does not exist" containerID="c15821f8bf553226caa65338f40007d64179be097dd7510bbd630ece5dfbcb06"
Jul 20 16:32:28.074888 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.074835    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c15821f8bf553226caa65338f40007d64179be097dd7510bbd630ece5dfbcb06"} err="failed to get container status \"c15821f8bf553226caa65338f40007d64179be097dd7510bbd630ece5dfbcb06\": rpc error: code = NotFound desc = could not find container \"c15821f8bf553226caa65338f40007d64179be097dd7510bbd630ece5dfbcb06\": container with ID starting with c15821f8bf553226caa65338f40007d64179be097dd7510bbd630ece5dfbcb06 not found: ID does not exist"
Jul 20 16:32:28.074888 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.074848    2577 scope.go:117] "RemoveContainer" containerID="249932bf3ed9c3b6fa5fe17aafec74306d2582023cfb85c5cff769cba69a2a76"
Jul 20 16:32:28.075109 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:32:28.075072    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"249932bf3ed9c3b6fa5fe17aafec74306d2582023cfb85c5cff769cba69a2a76\": container with ID starting with 249932bf3ed9c3b6fa5fe17aafec74306d2582023cfb85c5cff769cba69a2a76 not found: ID does not exist" containerID="249932bf3ed9c3b6fa5fe17aafec74306d2582023cfb85c5cff769cba69a2a76"
Jul 20 16:32:28.075147 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.075114    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"249932bf3ed9c3b6fa5fe17aafec74306d2582023cfb85c5cff769cba69a2a76"} err="failed to get container status \"249932bf3ed9c3b6fa5fe17aafec74306d2582023cfb85c5cff769cba69a2a76\": rpc error: code = NotFound desc = could not find container \"249932bf3ed9c3b6fa5fe17aafec74306d2582023cfb85c5cff769cba69a2a76\": container with ID starting with 249932bf3ed9c3b6fa5fe17aafec74306d2582023cfb85c5cff769cba69a2a76 not found: ID does not exist"
Jul 20 16:32:28.084532 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.084512    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 20 16:32:28.084779 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.084767    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33aa27a1-16e2-4e72-b8c4-7d76abdb2e51" containerName="console"
Jul 20 16:32:28.084820 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.084781    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="33aa27a1-16e2-4e72-b8c4-7d76abdb2e51" containerName="console"
Jul 20 16:32:28.084820 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.084789    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5368eaf-95e4-4bfc-96fa-59dd21754d8a" containerName="prometheus"
Jul 20 16:32:28.084820 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.084795    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5368eaf-95e4-4bfc-96fa-59dd21754d8a" containerName="prometheus"
Jul 20 16:32:28.084820 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.084807    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5368eaf-95e4-4bfc-96fa-59dd21754d8a" containerName="config-reloader"
Jul 20 16:32:28.084820 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.084812    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5368eaf-95e4-4bfc-96fa-59dd21754d8a" containerName="config-reloader"
Jul 20 16:32:28.084820 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.084821    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5368eaf-95e4-4bfc-96fa-59dd21754d8a" containerName="init-config-reloader"
Jul 20 16:32:28.085110 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.084827    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5368eaf-95e4-4bfc-96fa-59dd21754d8a" containerName="init-config-reloader"
Jul 20 16:32:28.085110 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.084837    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5368eaf-95e4-4bfc-96fa-59dd21754d8a" containerName="kube-rbac-proxy-web"
Jul 20 16:32:28.085110 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.084842    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5368eaf-95e4-4bfc-96fa-59dd21754d8a" containerName="kube-rbac-proxy-web"
Jul 20 16:32:28.085110 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.084849    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5368eaf-95e4-4bfc-96fa-59dd21754d8a" containerName="kube-rbac-proxy-thanos"
Jul 20 16:32:28.085110 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.084854    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5368eaf-95e4-4bfc-96fa-59dd21754d8a" containerName="kube-rbac-proxy-thanos"
Jul 20 16:32:28.085110 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.084861    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5368eaf-95e4-4bfc-96fa-59dd21754d8a" containerName="kube-rbac-proxy"
Jul 20 16:32:28.085110 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.084867    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5368eaf-95e4-4bfc-96fa-59dd21754d8a" containerName="kube-rbac-proxy"
Jul 20 16:32:28.085110 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.084876    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5368eaf-95e4-4bfc-96fa-59dd21754d8a" containerName="thanos-sidecar"
Jul 20 16:32:28.085110 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.084881    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5368eaf-95e4-4bfc-96fa-59dd21754d8a" containerName="thanos-sidecar"
Jul 20 16:32:28.085110 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.084888    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf2f8258-be74-4321-8a3f-5b8bcd376065" containerName="registry"
Jul 20 16:32:28.085110 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.084893    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf2f8258-be74-4321-8a3f-5b8bcd376065" containerName="registry"
Jul 20 16:32:28.085110 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.084929    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5368eaf-95e4-4bfc-96fa-59dd21754d8a" containerName="kube-rbac-proxy-thanos"
Jul 20 16:32:28.085110 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.084936    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5368eaf-95e4-4bfc-96fa-59dd21754d8a" containerName="config-reloader"
Jul 20 16:32:28.085110 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.084943    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="33aa27a1-16e2-4e72-b8c4-7d76abdb2e51" containerName="console"
Jul 20 16:32:28.085110 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.084949    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5368eaf-95e4-4bfc-96fa-59dd21754d8a" containerName="prometheus"
Jul 20 16:32:28.085110 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.084954    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5368eaf-95e4-4bfc-96fa-59dd21754d8a" containerName="kube-rbac-proxy-web"
Jul 20 16:32:28.085110 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.084959    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf2f8258-be74-4321-8a3f-5b8bcd376065" containerName="registry"
Jul 20 16:32:28.085110 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.084964    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5368eaf-95e4-4bfc-96fa-59dd21754d8a" containerName="thanos-sidecar"
Jul 20 16:32:28.085110 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.084969    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5368eaf-95e4-4bfc-96fa-59dd21754d8a" containerName="kube-rbac-proxy"
Jul 20 16:32:28.089708 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.089692    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.092183 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.092162    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 20 16:32:28.092285 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.092182    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 20 16:32:28.092407 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.092394    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 20 16:32:28.092945 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.092604    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 20 16:32:28.092945 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.092618    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-97dtt1krptne1\""
Jul 20 16:32:28.092945 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.092635    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 20 16:32:28.092945 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.092651    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 20 16:32:28.092945 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.092658    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 20 16:32:28.092945 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.092603    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 20 16:32:28.092945 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.092695    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 20 16:32:28.092945 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.092843    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 20 16:32:28.093374 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.093165    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 20 16:32:28.093374 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.093165    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 20 16:32:28.093374 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.093192    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-9z4d7\""
Jul 20 16:32:28.093374 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.093246    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 20 16:32:28.096696 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.096676    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 20 16:32:28.098843 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.098827    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 20 16:32:28.104856 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.104836    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 20 16:32:28.200797 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.200767    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.200797 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.200798    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.200983 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.200818    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.200983 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.200836    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.200983 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.200903    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.200983 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.200929    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.200983 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.200950    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ncfm\" (UniqueName: \"kubernetes.io/projected/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-kube-api-access-8ncfm\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.200983 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.200972    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.201182 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.200989    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-web-config\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.201182 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.201034    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-config\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.201182 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.201067    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.201182 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.201097    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.201182 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.201122    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.201182 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.201139    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.201182 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.201161    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.201182 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.201181    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.201467 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.201230    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.201467 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.201250    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.201467 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.201269    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.201467 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.201314    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-config-out\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.302208 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.302180    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.302353 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.302225    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.302353 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.302241    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.302353 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.302268    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.302353 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.302315    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.302598 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.302577    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.302664 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.302616    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.302664 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.302649    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.302766 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.302754    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-config-out\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.302816 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.302786    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.302869 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.302812    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.302869 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.302837    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.302980 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.302870    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.302980 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.302906    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.302980 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.302932    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.302980 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.302962    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ncfm\" (UniqueName: \"kubernetes.io/projected/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-kube-api-access-8ncfm\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.303164 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.302995    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.303164 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.303023    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-web-config\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.303164 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.303049    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.303164 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.303054    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-config\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.303164 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.303086    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.303164 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.303103    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.303834 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.303526    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.303834 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.303737    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.304068 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.304043    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.305407 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.305383    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-config-out\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.306088 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.305490    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.306088 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.305524    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.306088 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.305807    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-config\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.306270 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.306216    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.306842 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.306595    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.306842 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.306746    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.306842 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.306746    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.307128 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.307105    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.307855 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.307815    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.308065 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.308044    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.308452 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.308431    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-web-config\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.308531 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.308452    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.308531 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.308482    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.313884 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.313867    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ncfm\" (UniqueName: \"kubernetes.io/projected/7e8dd70e-cbec-4e9a-869a-0a8b56f0b257-kube-api-access-8ncfm\") pod \"prometheus-k8s-0\" (UID: \"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.398914 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.398884    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:28.529488 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:28.529465    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 20 16:32:28.530998 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:32:28.530974    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e8dd70e_cbec_4e9a_869a_0a8b56f0b257.slice/crio-2c2d0b503d41b03ffdad8a2cea7f625feb2eff85a6d561c7e6147c90ec892af6 WatchSource:0}: Error finding container 2c2d0b503d41b03ffdad8a2cea7f625feb2eff85a6d561c7e6147c90ec892af6: Status 404 returned error can't find the container with id 2c2d0b503d41b03ffdad8a2cea7f625feb2eff85a6d561c7e6147c90ec892af6
Jul 20 16:32:29.028175 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:29.028088    2577 generic.go:356] "Generic (PLEG): container finished" podID="7e8dd70e-cbec-4e9a-869a-0a8b56f0b257" containerID="40023b3947f1e1c74722540891e1066caf738da601a7b97004ec57c4cddd069b" exitCode=0
Jul 20 16:32:29.028175 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:29.028125    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257","Type":"ContainerDied","Data":"40023b3947f1e1c74722540891e1066caf738da601a7b97004ec57c4cddd069b"}
Jul 20 16:32:29.028697 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:29.028186    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257","Type":"ContainerStarted","Data":"2c2d0b503d41b03ffdad8a2cea7f625feb2eff85a6d561c7e6147c90ec892af6"}
Jul 20 16:32:29.212280 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:29.212240    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5368eaf-95e4-4bfc-96fa-59dd21754d8a" path="/var/lib/kubelet/pods/f5368eaf-95e4-4bfc-96fa-59dd21754d8a/volumes"
Jul 20 16:32:30.034874 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:30.034840    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257","Type":"ContainerStarted","Data":"dfb6029a7eaf14bc39781479f3213fdc911f9ab07e1275e13160fb8fe1368b5c"}
Jul 20 16:32:30.035234 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:30.034905    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257","Type":"ContainerStarted","Data":"fb237091f94bfb5b2e498a89abe6bcc6339765745168f669ce3a0984586e3bc7"}
Jul 20 16:32:30.035234 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:30.034932    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257","Type":"ContainerStarted","Data":"774f035b0269d21281c4968eced429316e02261c7e6fac465a64fce48b0f78e4"}
Jul 20 16:32:30.035234 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:30.034955    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257","Type":"ContainerStarted","Data":"3ed1db45550f42af4a7f9e5a8296fe2a3daa901293e9cbeed76827c9ec1127e3"}
Jul 20 16:32:30.035234 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:30.034979    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257","Type":"ContainerStarted","Data":"3a8a3c0f0bb72bf93c72425d6a9330084531b98c0664b576aa987840912ac515"}
Jul 20 16:32:30.035234 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:30.035002    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e8dd70e-cbec-4e9a-869a-0a8b56f0b257","Type":"ContainerStarted","Data":"47c9ddf254273bdbfdd6d7ad1f7f96a4d803255f70f52ec7443bddc25c3cc425"}
Jul 20 16:32:30.066994 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:30.066944    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.047958332 podStartE2EDuration="2.066928039s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.969707ms" isStatefulPod=false podCreationTimestamp="2026-07-20 16:32:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:32:30.065736722 +0000 UTC m=+173.315230742" watchObservedRunningTime="2026-07-20 16:32:30.066928039 +0000 UTC m=+173.316422058"
Jul 20 16:32:30.624092 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:30.624052    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:32:30.725851 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:30.725802    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:32:30.824427 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:30.824397    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:32:31.364633 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:31.364592    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:32:31.466918 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:31.466886    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:32:33.399478 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:33.399436    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:32:42.003861 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:42.003824    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:32:42.131620 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:42.131583    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:32:42.183384 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:32:42.183357    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:33:17.771567 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:33:17.771529    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:33:28.400137 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:33:28.400091    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:33:28.415280 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:33:28.415251    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:33:29.249274 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:33:29.249245    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 20 16:33:32.766531 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:33:32.766492    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:33:35.775736 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:33:35.775694    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:33:35.875807 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:33:35.875776    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:33:47.696530 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:33:47.696497    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:33:47.750861 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:33:47.750828    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:33:49.321768 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:33:49.321733    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:33:49.375358 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:33:49.375322    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:33:51.232942 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:33:51.232910    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:33:51.336201 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:33:51.336165    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:33:53.795739 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:33:53.795702    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:34:00.337378 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:34:00.337278    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:34:00.437903 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:34:00.437869    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:34:00.489672 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:34:00.489640    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:34:37.077856 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:34:37.077825    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gd4p4_fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c/ovn-acl-logging/0.log"
Jul 20 16:34:37.084436 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:34:37.084412    2577 kubelet.go:1648] "Image garbage collection succeeded"
Jul 20 16:34:37.084910 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:34:37.084895    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gd4p4_fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c/ovn-acl-logging/0.log"
Jul 20 16:34:40.173056 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:34:40.173027    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:34:41.703847 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:34:41.703807    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:35:06.843882 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:06.843846    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:35:07.607227 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:07.607179    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:35:11.005021 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:11.004988    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:35:11.105386 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:11.105354    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:35:12.796500 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:12.796450    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:35:12.900796 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:12.900767    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:35:13.054883 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:13.054800    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:35:14.594081 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:14.594041    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:35:15.181947 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:15.181904    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:35:17.447874 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:17.447846    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:35:18.121144 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:18.121109    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:35:23.989874 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:23.989832    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:35:24.556263 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:24.556227    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-b2fgk"]
Jul 20 16:35:24.559467 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:24.559444    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-b2fgk"
Jul 20 16:35:24.561902 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:24.561878    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 20 16:35:24.562797 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:24.562778    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-txsqg\""
Jul 20 16:35:24.562862 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:24.562796    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 20 16:35:24.569609 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:24.569585    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-b2fgk"]
Jul 20 16:35:24.670434 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:24.670394    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21a23e61-60d8-4a12-b5bc-3dc5f56f786b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-b2fgk\" (UID: \"21a23e61-60d8-4a12-b5bc-3dc5f56f786b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-b2fgk"
Jul 20 16:35:24.670609 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:24.670448    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdbjk\" (UniqueName: \"kubernetes.io/projected/21a23e61-60d8-4a12-b5bc-3dc5f56f786b-kube-api-access-wdbjk\") pod \"cert-manager-cainjector-559cdd8c87-b2fgk\" (UID: \"21a23e61-60d8-4a12-b5bc-3dc5f56f786b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-b2fgk"
Jul 20 16:35:24.771510 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:24.771469    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdbjk\" (UniqueName: \"kubernetes.io/projected/21a23e61-60d8-4a12-b5bc-3dc5f56f786b-kube-api-access-wdbjk\") pod \"cert-manager-cainjector-559cdd8c87-b2fgk\" (UID: \"21a23e61-60d8-4a12-b5bc-3dc5f56f786b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-b2fgk"
Jul 20 16:35:24.771688 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:24.771552    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21a23e61-60d8-4a12-b5bc-3dc5f56f786b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-b2fgk\" (UID: \"21a23e61-60d8-4a12-b5bc-3dc5f56f786b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-b2fgk"
Jul 20 16:35:24.782427 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:24.782392    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21a23e61-60d8-4a12-b5bc-3dc5f56f786b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-b2fgk\" (UID: \"21a23e61-60d8-4a12-b5bc-3dc5f56f786b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-b2fgk"
Jul 20 16:35:24.782564 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:24.782546    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdbjk\" (UniqueName: \"kubernetes.io/projected/21a23e61-60d8-4a12-b5bc-3dc5f56f786b-kube-api-access-wdbjk\") pod \"cert-manager-cainjector-559cdd8c87-b2fgk\" (UID: \"21a23e61-60d8-4a12-b5bc-3dc5f56f786b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-b2fgk"
Jul 20 16:35:24.879771 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:24.879690    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-b2fgk"
Jul 20 16:35:25.005322 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:25.005276    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-b2fgk"]
Jul 20 16:35:25.005805 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:35:25.005779    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod21a23e61_60d8_4a12_b5bc_3dc5f56f786b.slice/crio-ee03f2eef927bbafb8e82d49c061604ab2d8411471167bf879791ad75d72296d WatchSource:0}: Error finding container ee03f2eef927bbafb8e82d49c061604ab2d8411471167bf879791ad75d72296d: Status 404 returned error can't find the container with id ee03f2eef927bbafb8e82d49c061604ab2d8411471167bf879791ad75d72296d
Jul 20 16:35:25.007769 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:25.007751    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 16:35:25.634465 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:25.634417    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-b2fgk" event={"ID":"21a23e61-60d8-4a12-b5bc-3dc5f56f786b","Type":"ContainerStarted","Data":"ee03f2eef927bbafb8e82d49c061604ab2d8411471167bf879791ad75d72296d"}
Jul 20 16:35:28.571552 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:28.571513    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:35:28.647634 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:28.647594    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-b2fgk" event={"ID":"21a23e61-60d8-4a12-b5bc-3dc5f56f786b","Type":"ContainerStarted","Data":"6b4b6e01d34da65b092d81970b2da9fc08e7e3dcdf2621e3d5da958723c39dd1"}
Jul 20 16:35:28.678477 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:28.678444    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:35:28.679098 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:28.679060    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-b2fgk" podStartSLOduration=1.885938322 podStartE2EDuration="4.679046109s" totalImagesPullingTime="2.793107787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:35:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:35:28.678056901 +0000 UTC m=+351.927550945" watchObservedRunningTime="2026-07-20 16:35:28.679046109 +0000 UTC m=+351.928540128"
Jul 20 16:35:42.543046 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:42.542955    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:35:42.640701 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:42.640655    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:35:46.963926 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:46.963890    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:35:47.065159 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:47.065130    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:35:48.431031 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:48.430983    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:35:48.530553 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:48.530514    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:35:50.844568 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:50.844533    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:35:55.080893 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:55.080854    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:35:55.145173 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:55.145137    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6775c44bcb-msq95"]
Jul 20 16:35:55.148621 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:55.148602    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6775c44bcb-msq95"
Jul 20 16:35:55.151277 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:55.151260    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 20 16:35:55.151492 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:55.151478    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 20 16:35:55.152683 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:55.152644    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 20 16:35:55.152798 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:55.152706    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 20 16:35:55.152798 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:55.152654    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 20 16:35:55.152798 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:55.152654    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-54bn7\""
Jul 20 16:35:55.158756 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:55.158723    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6775c44bcb-msq95"]
Jul 20 16:35:55.161977 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:55.161956    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:35:55.329984 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:55.329947    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a9c37908-6c71-4dae-a2ca-2c097a3022a2-cert\") pod \"lws-controller-manager-6775c44bcb-msq95\" (UID: \"a9c37908-6c71-4dae-a2ca-2c097a3022a2\") " pod="openshift-lws-operator/lws-controller-manager-6775c44bcb-msq95"
Jul 20 16:35:55.329984 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:55.329994    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a9c37908-6c71-4dae-a2ca-2c097a3022a2-metrics-cert\") pod \"lws-controller-manager-6775c44bcb-msq95\" (UID: \"a9c37908-6c71-4dae-a2ca-2c097a3022a2\") " pod="openshift-lws-operator/lws-controller-manager-6775c44bcb-msq95"
Jul 20 16:35:55.330197 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:55.330064    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a9c37908-6c71-4dae-a2ca-2c097a3022a2-manager-config\") pod \"lws-controller-manager-6775c44bcb-msq95\" (UID: \"a9c37908-6c71-4dae-a2ca-2c097a3022a2\") " pod="openshift-lws-operator/lws-controller-manager-6775c44bcb-msq95"
Jul 20 16:35:55.330197 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:55.330094    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6vwl\" (UniqueName: \"kubernetes.io/projected/a9c37908-6c71-4dae-a2ca-2c097a3022a2-kube-api-access-n6vwl\") pod \"lws-controller-manager-6775c44bcb-msq95\" (UID: \"a9c37908-6c71-4dae-a2ca-2c097a3022a2\") " pod="openshift-lws-operator/lws-controller-manager-6775c44bcb-msq95"
Jul 20 16:35:55.431226 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:55.431137    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a9c37908-6c71-4dae-a2ca-2c097a3022a2-manager-config\") pod \"lws-controller-manager-6775c44bcb-msq95\" (UID: \"a9c37908-6c71-4dae-a2ca-2c097a3022a2\") " pod="openshift-lws-operator/lws-controller-manager-6775c44bcb-msq95"
Jul 20 16:35:55.431226 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:55.431195    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6vwl\" (UniqueName: \"kubernetes.io/projected/a9c37908-6c71-4dae-a2ca-2c097a3022a2-kube-api-access-n6vwl\") pod \"lws-controller-manager-6775c44bcb-msq95\" (UID: \"a9c37908-6c71-4dae-a2ca-2c097a3022a2\") " pod="openshift-lws-operator/lws-controller-manager-6775c44bcb-msq95"
Jul 20 16:35:55.431467 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:55.431247    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a9c37908-6c71-4dae-a2ca-2c097a3022a2-cert\") pod \"lws-controller-manager-6775c44bcb-msq95\" (UID: \"a9c37908-6c71-4dae-a2ca-2c097a3022a2\") " pod="openshift-lws-operator/lws-controller-manager-6775c44bcb-msq95"
Jul 20 16:35:55.431467 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:55.431278    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a9c37908-6c71-4dae-a2ca-2c097a3022a2-metrics-cert\") pod \"lws-controller-manager-6775c44bcb-msq95\" (UID: \"a9c37908-6c71-4dae-a2ca-2c097a3022a2\") " pod="openshift-lws-operator/lws-controller-manager-6775c44bcb-msq95"
Jul 20 16:35:55.431996 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:55.431968    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a9c37908-6c71-4dae-a2ca-2c097a3022a2-manager-config\") pod \"lws-controller-manager-6775c44bcb-msq95\" (UID: \"a9c37908-6c71-4dae-a2ca-2c097a3022a2\") " pod="openshift-lws-operator/lws-controller-manager-6775c44bcb-msq95"
Jul 20 16:35:55.433843 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:55.433812    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a9c37908-6c71-4dae-a2ca-2c097a3022a2-cert\") pod \"lws-controller-manager-6775c44bcb-msq95\" (UID: \"a9c37908-6c71-4dae-a2ca-2c097a3022a2\") " pod="openshift-lws-operator/lws-controller-manager-6775c44bcb-msq95"
Jul 20 16:35:55.434158 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:55.434138    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a9c37908-6c71-4dae-a2ca-2c097a3022a2-metrics-cert\") pod \"lws-controller-manager-6775c44bcb-msq95\" (UID: \"a9c37908-6c71-4dae-a2ca-2c097a3022a2\") " pod="openshift-lws-operator/lws-controller-manager-6775c44bcb-msq95"
Jul 20 16:35:55.440339 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:55.440277    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6vwl\" (UniqueName: \"kubernetes.io/projected/a9c37908-6c71-4dae-a2ca-2c097a3022a2-kube-api-access-n6vwl\") pod \"lws-controller-manager-6775c44bcb-msq95\" (UID: \"a9c37908-6c71-4dae-a2ca-2c097a3022a2\") " pod="openshift-lws-operator/lws-controller-manager-6775c44bcb-msq95"
Jul 20 16:35:55.457708 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:55.457683    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6775c44bcb-msq95"
Jul 20 16:35:55.588956 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:55.588923    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6775c44bcb-msq95"]
Jul 20 16:35:55.591685 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:35:55.591659    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9c37908_6c71_4dae_a2ca_2c097a3022a2.slice/crio-65db5519109f5bb652ddcdecc7da8bfee5c13a2a189e430cc8b50b04d34d86ed WatchSource:0}: Error finding container 65db5519109f5bb652ddcdecc7da8bfee5c13a2a189e430cc8b50b04d34d86ed: Status 404 returned error can't find the container with id 65db5519109f5bb652ddcdecc7da8bfee5c13a2a189e430cc8b50b04d34d86ed
Jul 20 16:35:55.748513 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:55.748422    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6775c44bcb-msq95" event={"ID":"a9c37908-6c71-4dae-a2ca-2c097a3022a2","Type":"ContainerStarted","Data":"65db5519109f5bb652ddcdecc7da8bfee5c13a2a189e430cc8b50b04d34d86ed"}
Jul 20 16:35:56.164169 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:56.164136    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:35:56.238204 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:35:56.238167    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:00.779376 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:00.779334    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6775c44bcb-msq95" event={"ID":"a9c37908-6c71-4dae-a2ca-2c097a3022a2","Type":"ContainerStarted","Data":"fbabe85a65250856261cbc1b734750ec5875945758cbda0657deca6a61c2bf9a"}
Jul 20 16:36:00.779764 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:00.779501    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6775c44bcb-msq95"
Jul 20 16:36:00.796811 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:00.796761    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6775c44bcb-msq95" podStartSLOduration=1.333450881 podStartE2EDuration="5.796745473s" totalImagesPullingTime="4.463294592s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:35:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:36:00.795576414 +0000 UTC m=+384.045070434" watchObservedRunningTime="2026-07-20 16:36:00.796745473 +0000 UTC m=+384.046239557"
Jul 20 16:36:06.516073 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:06.516037    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:06.613134 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:06.613097    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:07.149292 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:07.149260    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:07.366743 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:07.366697    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:07.466054 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:07.465969    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:07.510045 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:07.510007    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9d55867bb-nsh92"]
Jul 20 16:36:07.514379 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:07.514356    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9d55867bb-nsh92"
Jul 20 16:36:07.517072 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:07.517038    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 20 16:36:07.517483 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:07.517093    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-gt9k8\""
Jul 20 16:36:07.517483 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:07.517108    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 20 16:36:07.517483 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:07.517093    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 20 16:36:07.517483 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:07.517473    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 20 16:36:07.528369 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:07.528347    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9d55867bb-nsh92"]
Jul 20 16:36:07.533615 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:07.533590    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b-webhook-cert\") pod \"opendatahub-operator-controller-manager-9d55867bb-nsh92\" (UID: \"dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b\") " pod="opendatahub/opendatahub-operator-controller-manager-9d55867bb-nsh92"
Jul 20 16:36:07.533755 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:07.533636    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b-tmp\") pod \"opendatahub-operator-controller-manager-9d55867bb-nsh92\" (UID: \"dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b\") " pod="opendatahub/opendatahub-operator-controller-manager-9d55867bb-nsh92"
Jul 20 16:36:07.533755 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:07.533695    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b-manifests\") pod \"opendatahub-operator-controller-manager-9d55867bb-nsh92\" (UID: \"dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b\") " pod="opendatahub/opendatahub-operator-controller-manager-9d55867bb-nsh92"
Jul 20 16:36:07.533755 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:07.533730    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxs65\" (UniqueName: \"kubernetes.io/projected/dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b-kube-api-access-sxs65\") pod \"opendatahub-operator-controller-manager-9d55867bb-nsh92\" (UID: \"dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b\") " pod="opendatahub/opendatahub-operator-controller-manager-9d55867bb-nsh92"
Jul 20 16:36:07.533926 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:07.533786    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9d55867bb-nsh92\" (UID: \"dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b\") " pod="opendatahub/opendatahub-operator-controller-manager-9d55867bb-nsh92"
Jul 20 16:36:07.621160 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:07.621124    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:07.634516 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:07.634489    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b-manifests\") pod \"opendatahub-operator-controller-manager-9d55867bb-nsh92\" (UID: \"dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b\") " pod="opendatahub/opendatahub-operator-controller-manager-9d55867bb-nsh92"
Jul 20 16:36:07.634641 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:07.634522    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxs65\" (UniqueName: \"kubernetes.io/projected/dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b-kube-api-access-sxs65\") pod \"opendatahub-operator-controller-manager-9d55867bb-nsh92\" (UID: \"dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b\") " pod="opendatahub/opendatahub-operator-controller-manager-9d55867bb-nsh92"
Jul 20 16:36:07.634641 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:07.634560    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9d55867bb-nsh92\" (UID: \"dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b\") " pod="opendatahub/opendatahub-operator-controller-manager-9d55867bb-nsh92"
Jul 20 16:36:07.634641 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:07.634599    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b-webhook-cert\") pod \"opendatahub-operator-controller-manager-9d55867bb-nsh92\" (UID: \"dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b\") " pod="opendatahub/opendatahub-operator-controller-manager-9d55867bb-nsh92"
Jul 20 16:36:07.634641 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:07.634619    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b-tmp\") pod \"opendatahub-operator-controller-manager-9d55867bb-nsh92\" (UID: \"dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b\") " pod="opendatahub/opendatahub-operator-controller-manager-9d55867bb-nsh92"
Jul 20 16:36:07.634953 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:07.634934    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b-manifests\") pod \"opendatahub-operator-controller-manager-9d55867bb-nsh92\" (UID: \"dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b\") " pod="opendatahub/opendatahub-operator-controller-manager-9d55867bb-nsh92"
Jul 20 16:36:07.637051 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:07.637018    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b-tmp\") pod \"opendatahub-operator-controller-manager-9d55867bb-nsh92\" (UID: \"dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b\") " pod="opendatahub/opendatahub-operator-controller-manager-9d55867bb-nsh92"
Jul 20 16:36:07.637152 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:07.637085    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9d55867bb-nsh92\" (UID: \"dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b\") " pod="opendatahub/opendatahub-operator-controller-manager-9d55867bb-nsh92"
Jul 20 16:36:07.637152 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:07.637124    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b-webhook-cert\") pod \"opendatahub-operator-controller-manager-9d55867bb-nsh92\" (UID: \"dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b\") " pod="opendatahub/opendatahub-operator-controller-manager-9d55867bb-nsh92"
Jul 20 16:36:07.645030 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:07.645007    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxs65\" (UniqueName: \"kubernetes.io/projected/dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b-kube-api-access-sxs65\") pod \"opendatahub-operator-controller-manager-9d55867bb-nsh92\" (UID: \"dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b\") " pod="opendatahub/opendatahub-operator-controller-manager-9d55867bb-nsh92"
Jul 20 16:36:07.720454 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:07.720373    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:07.825224 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:07.825195    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9d55867bb-nsh92"
Jul 20 16:36:07.830553 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:07.830528    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:07.925988 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:07.925961    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:07.958664 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:07.958635    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9d55867bb-nsh92"]
Jul 20 16:36:07.961127 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:36:07.961102    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddab53aaf_8e6f_44a2_9e2c_668cc09bbb8b.slice/crio-567879454b9c4359e9191f45c078aac95f08031e289485a4c6377b55c8976754 WatchSource:0}: Error finding container 567879454b9c4359e9191f45c078aac95f08031e289485a4c6377b55c8976754: Status 404 returned error can't find the container with id 567879454b9c4359e9191f45c078aac95f08031e289485a4c6377b55c8976754
Jul 20 16:36:08.017949 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:08.017859    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:08.813040 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:08.813003    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9d55867bb-nsh92" event={"ID":"dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b","Type":"ContainerStarted","Data":"567879454b9c4359e9191f45c078aac95f08031e289485a4c6377b55c8976754"}
Jul 20 16:36:11.785944 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:11.785914    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6775c44bcb-msq95"
Jul 20 16:36:11.827149 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:11.827102    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9d55867bb-nsh92" event={"ID":"dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b","Type":"ContainerStarted","Data":"43efe9dfae69d8315b4b44301e762fd24d22edb5e3f5308fd13f7b3a1eb5e450"}
Jul 20 16:36:12.832846 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:12.832810    2577 generic.go:356] "Generic (PLEG): container finished" podID="dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b" containerID="43efe9dfae69d8315b4b44301e762fd24d22edb5e3f5308fd13f7b3a1eb5e450" exitCode=0
Jul 20 16:36:12.833236 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:12.832896    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9d55867bb-nsh92" event={"ID":"dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b","Type":"ContainerDied","Data":"43efe9dfae69d8315b4b44301e762fd24d22edb5e3f5308fd13f7b3a1eb5e450"}
Jul 20 16:36:13.838762 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:13.838723    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9d55867bb-nsh92" event={"ID":"dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b","Type":"ContainerStarted","Data":"dbcd4e5fbd35dd2ef8418511a184002335f2e7bb1af4731a3cfb5b12afc5da22"}
Jul 20 16:36:13.839223 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:13.838873    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-9d55867bb-nsh92"
Jul 20 16:36:13.869008 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:13.868955    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-9d55867bb-nsh92" podStartSLOduration=2.376991143 podStartE2EDuration="6.868937822s" totalImagesPullingTime="3.409195814s" totalInitContainerRuntime="1.082750865s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:36:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:36:13.866337419 +0000 UTC m=+397.115831436" watchObservedRunningTime="2026-07-20 16:36:13.868937822 +0000 UTC m=+397.118431840"
Jul 20 16:36:19.052165 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:19.052129    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:19.152082 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:19.152046    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:19.987352 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:19.987317    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:20.088885 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:20.088850    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:20.426016 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:20.425962    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:20.527241 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:20.527208    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:24.193038 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:24.193006    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:24.845549 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:24.845519    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-9d55867bb-nsh92"
Jul 20 16:36:26.516084 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:26.516055    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:26.699514 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:26.699476    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:27.313946 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:27.313894    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:27.420232 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:27.420193    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:27.526582 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:27.526546    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:27.810234 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:27.810204    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:28.187122 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:28.187041    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:32.497663 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:32.497619    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:32.603785 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:32.603752    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-745b54bffd-zgzf9"]
Jul 20 16:36:32.607108 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:32.607082    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-745b54bffd-zgzf9"
Jul 20 16:36:32.609733 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:32.609699    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 20 16:36:32.609879 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:32.609700    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 20 16:36:32.609879 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:32.609747    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 20 16:36:32.610645 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:32.610623    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-kxjfw\""
Jul 20 16:36:32.610773 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:32.610653    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 20 16:36:32.618576 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:32.618555    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-745b54bffd-zgzf9"]
Jul 20 16:36:32.757404 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:32.757285    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dstxk\" (UniqueName: \"kubernetes.io/projected/7f988b64-f729-4c51-afee-366220e6ffcf-kube-api-access-dstxk\") pod \"kube-auth-proxy-745b54bffd-zgzf9\" (UID: \"7f988b64-f729-4c51-afee-366220e6ffcf\") " pod="openshift-ingress/kube-auth-proxy-745b54bffd-zgzf9"
Jul 20 16:36:32.757582 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:32.757448    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f988b64-f729-4c51-afee-366220e6ffcf-tls-certs\") pod \"kube-auth-proxy-745b54bffd-zgzf9\" (UID: \"7f988b64-f729-4c51-afee-366220e6ffcf\") " pod="openshift-ingress/kube-auth-proxy-745b54bffd-zgzf9"
Jul 20 16:36:32.757582 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:32.757482    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7f988b64-f729-4c51-afee-366220e6ffcf-tmp\") pod \"kube-auth-proxy-745b54bffd-zgzf9\" (UID: \"7f988b64-f729-4c51-afee-366220e6ffcf\") " pod="openshift-ingress/kube-auth-proxy-745b54bffd-zgzf9"
Jul 20 16:36:32.858971 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:32.858925    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f988b64-f729-4c51-afee-366220e6ffcf-tls-certs\") pod \"kube-auth-proxy-745b54bffd-zgzf9\" (UID: \"7f988b64-f729-4c51-afee-366220e6ffcf\") " pod="openshift-ingress/kube-auth-proxy-745b54bffd-zgzf9"
Jul 20 16:36:32.858971 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:32.858976    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7f988b64-f729-4c51-afee-366220e6ffcf-tmp\") pod \"kube-auth-proxy-745b54bffd-zgzf9\" (UID: \"7f988b64-f729-4c51-afee-366220e6ffcf\") " pod="openshift-ingress/kube-auth-proxy-745b54bffd-zgzf9"
Jul 20 16:36:32.859217 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:32.859059    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dstxk\" (UniqueName: \"kubernetes.io/projected/7f988b64-f729-4c51-afee-366220e6ffcf-kube-api-access-dstxk\") pod \"kube-auth-proxy-745b54bffd-zgzf9\" (UID: \"7f988b64-f729-4c51-afee-366220e6ffcf\") " pod="openshift-ingress/kube-auth-proxy-745b54bffd-zgzf9"
Jul 20 16:36:32.859217 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:36:32.859071    2577 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 20 16:36:32.859217 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:36:32.859132    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f988b64-f729-4c51-afee-366220e6ffcf-tls-certs podName:7f988b64-f729-4c51-afee-366220e6ffcf nodeName:}" failed. No retries permitted until 2026-07-20 16:36:33.359115314 +0000 UTC m=+416.608609315 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/7f988b64-f729-4c51-afee-366220e6ffcf-tls-certs") pod "kube-auth-proxy-745b54bffd-zgzf9" (UID: "7f988b64-f729-4c51-afee-366220e6ffcf") : secret "kube-auth-proxy-tls" not found
Jul 20 16:36:32.861281 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:32.861249    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7f988b64-f729-4c51-afee-366220e6ffcf-tmp\") pod \"kube-auth-proxy-745b54bffd-zgzf9\" (UID: \"7f988b64-f729-4c51-afee-366220e6ffcf\") " pod="openshift-ingress/kube-auth-proxy-745b54bffd-zgzf9"
Jul 20 16:36:32.869889 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:32.869862    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dstxk\" (UniqueName: \"kubernetes.io/projected/7f988b64-f729-4c51-afee-366220e6ffcf-kube-api-access-dstxk\") pod \"kube-auth-proxy-745b54bffd-zgzf9\" (UID: \"7f988b64-f729-4c51-afee-366220e6ffcf\") " pod="openshift-ingress/kube-auth-proxy-745b54bffd-zgzf9"
Jul 20 16:36:33.141812 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:33.141778    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:33.363639 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:33.363602    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f988b64-f729-4c51-afee-366220e6ffcf-tls-certs\") pod \"kube-auth-proxy-745b54bffd-zgzf9\" (UID: \"7f988b64-f729-4c51-afee-366220e6ffcf\") " pod="openshift-ingress/kube-auth-proxy-745b54bffd-zgzf9"
Jul 20 16:36:33.366084 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:33.366050    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f988b64-f729-4c51-afee-366220e6ffcf-tls-certs\") pod \"kube-auth-proxy-745b54bffd-zgzf9\" (UID: \"7f988b64-f729-4c51-afee-366220e6ffcf\") " pod="openshift-ingress/kube-auth-proxy-745b54bffd-zgzf9"
Jul 20 16:36:33.519352 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:33.519254    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-745b54bffd-zgzf9"
Jul 20 16:36:33.720222 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:33.720125    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-745b54bffd-zgzf9"]
Jul 20 16:36:33.722615 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:36:33.722584    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f988b64_f729_4c51_afee_366220e6ffcf.slice/crio-8d90b15e477b1883477aa0db6f9b3b524088623fdd62b97d142dc9058a12ec35 WatchSource:0}: Error finding container 8d90b15e477b1883477aa0db6f9b3b524088623fdd62b97d142dc9058a12ec35: Status 404 returned error can't find the container with id 8d90b15e477b1883477aa0db6f9b3b524088623fdd62b97d142dc9058a12ec35
Jul 20 16:36:33.826230 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:33.826191    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:33.915574 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:33.915537    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-745b54bffd-zgzf9" event={"ID":"7f988b64-f729-4c51-afee-366220e6ffcf","Type":"ContainerStarted","Data":"8d90b15e477b1883477aa0db6f9b3b524088623fdd62b97d142dc9058a12ec35"}
Jul 20 16:36:33.958793 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:33.958757    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:35.110143 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:35.110113    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:36.599055 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:36.599021    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:36.741664 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:36.741631    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:36.923266 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:36.923174    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:36.928619 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:36.928592    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-745b54bffd-zgzf9" event={"ID":"7f988b64-f729-4c51-afee-366220e6ffcf","Type":"ContainerStarted","Data":"c408a4c8edbe78266a8363503a8c06ba883740c885884e130a49137e57beaa23"}
Jul 20 16:36:36.948552 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:36.948496    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-745b54bffd-zgzf9" podStartSLOduration=2.052375461 podStartE2EDuration="4.948475545s" totalImagesPullingTime="2.896100084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:36:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:36:36.947534473 +0000 UTC m=+420.197028496" watchObservedRunningTime="2026-07-20 16:36:36.948475545 +0000 UTC m=+420.197969566"
Jul 20 16:36:37.031993 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:37.031958    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:37.179466 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:37.179391    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:37.307003 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:37.306973    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:38.221630 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:38.221592    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:38.397820 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:38.397793    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:38.507957 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:38.507875    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:39.301932 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:39.301898    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:39.544606 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:39.544571    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:39.740130 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:39.740050    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:42.793336 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:42.793282    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:42.894150 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:42.894121    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:36:53.628032 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:36:53.627998    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:37:28.021810 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:37:28.021777    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:37:28.199609 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:37:28.199574    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:37:28.265652 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:37:28.265622    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:37:28.931896 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:37:28.931865    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:37:29.133198 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:37:29.133164    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:37:37.869623 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:37:37.869588    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:37:37.970511 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:37:37.970474    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:37:48.526140 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:37:48.526102    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:37:48.626521 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:37:48.626486    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:37:48.728099 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:37:48.728063    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:37:50.875190 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:37:50.875157    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:37:56.843567 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:37:56.843533    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:00.729712 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:00.729670    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:02.886927 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:02.886891    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:04.756315 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:04.756263    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:04.884919 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:04.884881    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:07.114032 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:07.113996    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:07.259059 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:07.259025    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:08.064507 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:08.064478    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:09.482075 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:09.482040    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:12.006468 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:12.006432    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-t79zm"]
Jul 20 16:38:12.009356 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:12.009334    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-t79zm"
Jul 20 16:38:12.012096 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:12.012074    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 20 16:38:12.012225 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:12.012139    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 20 16:38:12.012328 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:12.012235    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 20 16:38:12.012328 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:12.012240    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-fgs6k\""
Jul 20 16:38:12.013103 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:12.013087    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 20 16:38:12.019624 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:12.019601    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:12.020435 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:12.020416    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-t79zm"]
Jul 20 16:38:12.064954 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:12.064924    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/cea7bc8f-75a2-4f2b-a14d-8ead22da6a5d-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-t79zm\" (UID: \"cea7bc8f-75a2-4f2b-a14d-8ead22da6a5d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-t79zm"
Jul 20 16:38:12.065086 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:12.064967    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cea7bc8f-75a2-4f2b-a14d-8ead22da6a5d-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-t79zm\" (UID: \"cea7bc8f-75a2-4f2b-a14d-8ead22da6a5d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-t79zm"
Jul 20 16:38:12.065086 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:12.065043    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdf54\" (UniqueName: \"kubernetes.io/projected/cea7bc8f-75a2-4f2b-a14d-8ead22da6a5d-kube-api-access-gdf54\") pod \"kuadrant-console-plugin-66b5774985-t79zm\" (UID: \"cea7bc8f-75a2-4f2b-a14d-8ead22da6a5d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-t79zm"
Jul 20 16:38:12.166244 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:12.166209    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cea7bc8f-75a2-4f2b-a14d-8ead22da6a5d-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-t79zm\" (UID: \"cea7bc8f-75a2-4f2b-a14d-8ead22da6a5d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-t79zm"
Jul 20 16:38:12.166409 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:12.166266    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdf54\" (UniqueName: \"kubernetes.io/projected/cea7bc8f-75a2-4f2b-a14d-8ead22da6a5d-kube-api-access-gdf54\") pod \"kuadrant-console-plugin-66b5774985-t79zm\" (UID: \"cea7bc8f-75a2-4f2b-a14d-8ead22da6a5d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-t79zm"
Jul 20 16:38:12.166409 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:12.166331    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/cea7bc8f-75a2-4f2b-a14d-8ead22da6a5d-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-t79zm\" (UID: \"cea7bc8f-75a2-4f2b-a14d-8ead22da6a5d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-t79zm"
Jul 20 16:38:12.166482 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:38:12.166415    2577 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 20 16:38:12.166482 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:38:12.166471    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cea7bc8f-75a2-4f2b-a14d-8ead22da6a5d-plugin-serving-cert podName:cea7bc8f-75a2-4f2b-a14d-8ead22da6a5d nodeName:}" failed. No retries permitted until 2026-07-20 16:38:12.666454472 +0000 UTC m=+515.915948473 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/cea7bc8f-75a2-4f2b-a14d-8ead22da6a5d-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-t79zm" (UID: "cea7bc8f-75a2-4f2b-a14d-8ead22da6a5d") : secret "plugin-serving-cert" not found
Jul 20 16:38:12.166973 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:12.166953    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cea7bc8f-75a2-4f2b-a14d-8ead22da6a5d-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-t79zm\" (UID: \"cea7bc8f-75a2-4f2b-a14d-8ead22da6a5d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-t79zm"
Jul 20 16:38:12.180197 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:12.180166    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdf54\" (UniqueName: \"kubernetes.io/projected/cea7bc8f-75a2-4f2b-a14d-8ead22da6a5d-kube-api-access-gdf54\") pod \"kuadrant-console-plugin-66b5774985-t79zm\" (UID: \"cea7bc8f-75a2-4f2b-a14d-8ead22da6a5d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-t79zm"
Jul 20 16:38:12.467442 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:12.467407    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:12.569677 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:12.569641    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:12.669731 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:12.669691    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/cea7bc8f-75a2-4f2b-a14d-8ead22da6a5d-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-t79zm\" (UID: \"cea7bc8f-75a2-4f2b-a14d-8ead22da6a5d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-t79zm"
Jul 20 16:38:12.672063 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:12.672045    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/cea7bc8f-75a2-4f2b-a14d-8ead22da6a5d-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-t79zm\" (UID: \"cea7bc8f-75a2-4f2b-a14d-8ead22da6a5d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-t79zm"
Jul 20 16:38:12.919181 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:12.919153    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-t79zm"
Jul 20 16:38:13.049851 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:13.049819    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-t79zm"]
Jul 20 16:38:13.052902 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:38:13.052875    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcea7bc8f_75a2_4f2b_a14d_8ead22da6a5d.slice/crio-1cc7a2dcf8a329e7ea564bad940fa16d0736cd860fb26b0e2323e20bcac5244c WatchSource:0}: Error finding container 1cc7a2dcf8a329e7ea564bad940fa16d0736cd860fb26b0e2323e20bcac5244c: Status 404 returned error can't find the container with id 1cc7a2dcf8a329e7ea564bad940fa16d0736cd860fb26b0e2323e20bcac5244c
Jul 20 16:38:13.301479 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:13.301445    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-t79zm" event={"ID":"cea7bc8f-75a2-4f2b-a14d-8ead22da6a5d","Type":"ContainerStarted","Data":"1cc7a2dcf8a329e7ea564bad940fa16d0736cd860fb26b0e2323e20bcac5244c"}
Jul 20 16:38:20.116977 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:20.116932    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:20.215957 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:20.215909    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:20.890255 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:20.890216    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:24.896918 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:24.896885    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:24.998366 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:24.998320    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:29.678551 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:29.678235    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:29.778028 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:29.777987    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:33.877047 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:33.877004    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:33.978271 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:33.978236    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:37.410979 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:37.410945    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-t79zm" event={"ID":"cea7bc8f-75a2-4f2b-a14d-8ead22da6a5d","Type":"ContainerStarted","Data":"161e545137b2f80bb11f859724e56c9ccbeae95da824892494866f11999a2ee5"}
Jul 20 16:38:37.426653 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:37.426599    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-t79zm" podStartSLOduration=3.018740265 podStartE2EDuration="26.426572873s" totalImagesPullingTime="23.407832608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:38:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:38:37.426180318 +0000 UTC m=+540.675674339" watchObservedRunningTime="2026-07-20 16:38:37.426572873 +0000 UTC m=+540.676066891"
Jul 20 16:38:41.071786 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:41.071752    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:41.172282 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:41.172250    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:45.869589 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:45.869548    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:45.972905 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:45.972843    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:49.228371 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:49.228334    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:49.328497 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:49.328461    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:50.670798 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:50.670761    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:50.771652 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:50.771618    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:51.002036 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:51.001951    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:51.160162 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:51.160130    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:51.268888 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:51.268787    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:51.326027 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:51.325994    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:53.499888 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:53.499854    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8g8w5"]
Jul 20 16:38:53.533005 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:53.532973    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:53.535710 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:53.535685    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8g8w5"]
Jul 20 16:38:53.535824 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:53.535778    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8g8w5"
Jul 20 16:38:53.537989 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:53.537971    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 20 16:38:53.594805 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:53.594769    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8g8w5"]
Jul 20 16:38:53.597728 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:53.597706    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:53.622025 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:53.622000    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/239f76e0-a9cf-46fc-ab02-9bf58def2c54-config-file\") pod \"limitador-limitador-64c94fd64f-8g8w5\" (UID: \"239f76e0-a9cf-46fc-ab02-9bf58def2c54\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8g8w5"
Jul 20 16:38:53.622159 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:53.622047    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxlhj\" (UniqueName: \"kubernetes.io/projected/239f76e0-a9cf-46fc-ab02-9bf58def2c54-kube-api-access-kxlhj\") pod \"limitador-limitador-64c94fd64f-8g8w5\" (UID: \"239f76e0-a9cf-46fc-ab02-9bf58def2c54\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8g8w5"
Jul 20 16:38:53.659882 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:53.659849    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:53.723012 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:53.722982    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxlhj\" (UniqueName: \"kubernetes.io/projected/239f76e0-a9cf-46fc-ab02-9bf58def2c54-kube-api-access-kxlhj\") pod \"limitador-limitador-64c94fd64f-8g8w5\" (UID: \"239f76e0-a9cf-46fc-ab02-9bf58def2c54\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8g8w5"
Jul 20 16:38:53.723184 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:53.723043    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/239f76e0-a9cf-46fc-ab02-9bf58def2c54-config-file\") pod \"limitador-limitador-64c94fd64f-8g8w5\" (UID: \"239f76e0-a9cf-46fc-ab02-9bf58def2c54\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8g8w5"
Jul 20 16:38:53.723647 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:53.723628    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/239f76e0-a9cf-46fc-ab02-9bf58def2c54-config-file\") pod \"limitador-limitador-64c94fd64f-8g8w5\" (UID: \"239f76e0-a9cf-46fc-ab02-9bf58def2c54\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8g8w5"
Jul 20 16:38:53.730610 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:53.730590    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxlhj\" (UniqueName: \"kubernetes.io/projected/239f76e0-a9cf-46fc-ab02-9bf58def2c54-kube-api-access-kxlhj\") pod \"limitador-limitador-64c94fd64f-8g8w5\" (UID: \"239f76e0-a9cf-46fc-ab02-9bf58def2c54\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-8g8w5"
Jul 20 16:38:53.845266 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:53.845239    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8g8w5"
Jul 20 16:38:53.969925 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:53.969894    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8g8w5"]
Jul 20 16:38:53.973272 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:38:53.973240    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod239f76e0_a9cf_46fc_ab02_9bf58def2c54.slice/crio-e407a3341608d496e9f3abf48da9e4f42f8db5977fd80cc8613e113efc44b3a2 WatchSource:0}: Error finding container e407a3341608d496e9f3abf48da9e4f42f8db5977fd80cc8613e113efc44b3a2: Status 404 returned error can't find the container with id e407a3341608d496e9f3abf48da9e4f42f8db5977fd80cc8613e113efc44b3a2
Jul 20 16:38:54.358452 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:54.358396    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:54.470473 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:54.470431    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:54.479098 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:54.479067    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8g8w5" event={"ID":"239f76e0-a9cf-46fc-ab02-9bf58def2c54","Type":"ContainerStarted","Data":"e407a3341608d496e9f3abf48da9e4f42f8db5977fd80cc8613e113efc44b3a2"}
Jul 20 16:38:56.917731 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:56.917644    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:56.967784 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:56.967744    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:38:57.495002 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:57.494965    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8g8w5" event={"ID":"239f76e0-a9cf-46fc-ab02-9bf58def2c54","Type":"ContainerStarted","Data":"d7def861f09fbcf344b43c275e1962ce906fbd55291ecaf6536866b5515495dc"}
Jul 20 16:38:57.495206 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:57.495084    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-8g8w5"
Jul 20 16:38:57.517700 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:38:57.517632    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-8g8w5" podStartSLOduration=1.877050966 podStartE2EDuration="4.517617479s" totalImagesPullingTime="2.640566513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:38:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:38:57.51594133 +0000 UTC m=+560.765435361" watchObservedRunningTime="2026-07-20 16:38:57.517617479 +0000 UTC m=+560.767111497"
Jul 20 16:39:08.499058 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:08.499022    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-8g8w5"
Jul 20 16:39:09.849436 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:09.849399    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8g8w5"]
Jul 20 16:39:09.849803 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:09.849570    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-8g8w5" podUID="239f76e0-a9cf-46fc-ab02-9bf58def2c54" containerName="limitador" containerID="cri-o://d7def861f09fbcf344b43c275e1962ce906fbd55291ecaf6536866b5515495dc" gracePeriod=30
Jul 20 16:39:10.408168 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:10.408144    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8g8w5"
Jul 20 16:39:10.466919 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:10.466845    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/239f76e0-a9cf-46fc-ab02-9bf58def2c54-config-file\") pod \"239f76e0-a9cf-46fc-ab02-9bf58def2c54\" (UID: \"239f76e0-a9cf-46fc-ab02-9bf58def2c54\") "
Jul 20 16:39:10.466919 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:10.466902    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxlhj\" (UniqueName: \"kubernetes.io/projected/239f76e0-a9cf-46fc-ab02-9bf58def2c54-kube-api-access-kxlhj\") pod \"239f76e0-a9cf-46fc-ab02-9bf58def2c54\" (UID: \"239f76e0-a9cf-46fc-ab02-9bf58def2c54\") "
Jul 20 16:39:10.467229 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:10.467200    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/239f76e0-a9cf-46fc-ab02-9bf58def2c54-config-file" (OuterVolumeSpecName: "config-file") pod "239f76e0-a9cf-46fc-ab02-9bf58def2c54" (UID: "239f76e0-a9cf-46fc-ab02-9bf58def2c54"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:39:10.469029 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:10.469005    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/239f76e0-a9cf-46fc-ab02-9bf58def2c54-kube-api-access-kxlhj" (OuterVolumeSpecName: "kube-api-access-kxlhj") pod "239f76e0-a9cf-46fc-ab02-9bf58def2c54" (UID: "239f76e0-a9cf-46fc-ab02-9bf58def2c54"). InnerVolumeSpecName "kube-api-access-kxlhj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:39:10.556806 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:10.556775    2577 generic.go:356] "Generic (PLEG): container finished" podID="239f76e0-a9cf-46fc-ab02-9bf58def2c54" containerID="d7def861f09fbcf344b43c275e1962ce906fbd55291ecaf6536866b5515495dc" exitCode=0
Jul 20 16:39:10.556960 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:10.556818    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8g8w5" event={"ID":"239f76e0-a9cf-46fc-ab02-9bf58def2c54","Type":"ContainerDied","Data":"d7def861f09fbcf344b43c275e1962ce906fbd55291ecaf6536866b5515495dc"}
Jul 20 16:39:10.556960 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:10.556838    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-8g8w5"
Jul 20 16:39:10.556960 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:10.556880    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-8g8w5" event={"ID":"239f76e0-a9cf-46fc-ab02-9bf58def2c54","Type":"ContainerDied","Data":"e407a3341608d496e9f3abf48da9e4f42f8db5977fd80cc8613e113efc44b3a2"}
Jul 20 16:39:10.556960 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:10.556902    2577 scope.go:117] "RemoveContainer" containerID="d7def861f09fbcf344b43c275e1962ce906fbd55291ecaf6536866b5515495dc"
Jul 20 16:39:10.565238 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:10.565212    2577 scope.go:117] "RemoveContainer" containerID="d7def861f09fbcf344b43c275e1962ce906fbd55291ecaf6536866b5515495dc"
Jul 20 16:39:10.565496 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:39:10.565478    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7def861f09fbcf344b43c275e1962ce906fbd55291ecaf6536866b5515495dc\": container with ID starting with d7def861f09fbcf344b43c275e1962ce906fbd55291ecaf6536866b5515495dc not found: ID does not exist" containerID="d7def861f09fbcf344b43c275e1962ce906fbd55291ecaf6536866b5515495dc"
Jul 20 16:39:10.565544 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:10.565503    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7def861f09fbcf344b43c275e1962ce906fbd55291ecaf6536866b5515495dc"} err="failed to get container status \"d7def861f09fbcf344b43c275e1962ce906fbd55291ecaf6536866b5515495dc\": rpc error: code = NotFound desc = could not find container \"d7def861f09fbcf344b43c275e1962ce906fbd55291ecaf6536866b5515495dc\": container with ID starting with d7def861f09fbcf344b43c275e1962ce906fbd55291ecaf6536866b5515495dc not found: ID does not exist"
Jul 20 16:39:10.567808 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:10.567790    2577 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/239f76e0-a9cf-46fc-ab02-9bf58def2c54-config-file\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:39:10.567895 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:10.567816    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxlhj\" (UniqueName: \"kubernetes.io/projected/239f76e0-a9cf-46fc-ab02-9bf58def2c54-kube-api-access-kxlhj\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:39:10.576827 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:10.576804    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8g8w5"]
Jul 20 16:39:10.580646 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:10.580628    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-8g8w5"]
Jul 20 16:39:11.211873 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:11.211836    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="239f76e0-a9cf-46fc-ab02-9bf58def2c54" path="/var/lib/kubelet/pods/239f76e0-a9cf-46fc-ab02-9bf58def2c54/volumes"
Jul 20 16:39:14.752227 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:14.752193    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:39:14.851709 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:14.851678    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:39:15.211903 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:15.211871    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-qhrgc"]
Jul 20 16:39:15.212228 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:15.212210    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="239f76e0-a9cf-46fc-ab02-9bf58def2c54" containerName="limitador"
Jul 20 16:39:15.212340 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:15.212230    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="239f76e0-a9cf-46fc-ab02-9bf58def2c54" containerName="limitador"
Jul 20 16:39:15.212404 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:15.212351    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="239f76e0-a9cf-46fc-ab02-9bf58def2c54" containerName="limitador"
Jul 20 16:39:15.216801 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:15.216773    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qhrgc"
Jul 20 16:39:15.218595 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:15.218571    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-qhrgc"]
Jul 20 16:39:15.219651 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:15.219632    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-mc8vn\""
Jul 20 16:39:15.219767 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:15.219655    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 16:39:15.219767 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:15.219635    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 20 16:39:15.220707 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:15.220688    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 20 16:39:15.239413 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:15.239389    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:39:15.314762 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:15.314718    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfvgq\" (UniqueName: \"kubernetes.io/projected/826b421a-e9dc-44c8-bfe1-5ebffaf759a6-kube-api-access-gfvgq\") pod \"postgres-69fb4594cf-qhrgc\" (UID: \"826b421a-e9dc-44c8-bfe1-5ebffaf759a6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qhrgc"
Jul 20 16:39:15.314924 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:15.314825    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/826b421a-e9dc-44c8-bfe1-5ebffaf759a6-data\") pod \"postgres-69fb4594cf-qhrgc\" (UID: \"826b421a-e9dc-44c8-bfe1-5ebffaf759a6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qhrgc"
Jul 20 16:39:15.415540 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:15.415502    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/826b421a-e9dc-44c8-bfe1-5ebffaf759a6-data\") pod \"postgres-69fb4594cf-qhrgc\" (UID: \"826b421a-e9dc-44c8-bfe1-5ebffaf759a6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qhrgc"
Jul 20 16:39:15.415733 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:15.415561    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfvgq\" (UniqueName: \"kubernetes.io/projected/826b421a-e9dc-44c8-bfe1-5ebffaf759a6-kube-api-access-gfvgq\") pod \"postgres-69fb4594cf-qhrgc\" (UID: \"826b421a-e9dc-44c8-bfe1-5ebffaf759a6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qhrgc"
Jul 20 16:39:15.415933 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:15.415910    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/826b421a-e9dc-44c8-bfe1-5ebffaf759a6-data\") pod \"postgres-69fb4594cf-qhrgc\" (UID: \"826b421a-e9dc-44c8-bfe1-5ebffaf759a6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qhrgc"
Jul 20 16:39:15.425499 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:15.425465    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfvgq\" (UniqueName: \"kubernetes.io/projected/826b421a-e9dc-44c8-bfe1-5ebffaf759a6-kube-api-access-gfvgq\") pod \"postgres-69fb4594cf-qhrgc\" (UID: \"826b421a-e9dc-44c8-bfe1-5ebffaf759a6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qhrgc"
Jul 20 16:39:15.527952 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:15.527864    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qhrgc"
Jul 20 16:39:15.655135 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:15.655104    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-qhrgc"]
Jul 20 16:39:15.656925 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:39:15.656899    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod826b421a_e9dc_44c8_bfe1_5ebffaf759a6.slice/crio-d98ab28d23931e9223ae2ffc79778b4de9940e4157cc85ee315207e1a6660ad7 WatchSource:0}: Error finding container d98ab28d23931e9223ae2ffc79778b4de9940e4157cc85ee315207e1a6660ad7: Status 404 returned error can't find the container with id d98ab28d23931e9223ae2ffc79778b4de9940e4157cc85ee315207e1a6660ad7
Jul 20 16:39:16.583945 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:16.583910    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qhrgc" event={"ID":"826b421a-e9dc-44c8-bfe1-5ebffaf759a6","Type":"ContainerStarted","Data":"d98ab28d23931e9223ae2ffc79778b4de9940e4157cc85ee315207e1a6660ad7"}
Jul 20 16:39:20.087569 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:20.087532    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:39:21.609861 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:21.609821    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qhrgc" event={"ID":"826b421a-e9dc-44c8-bfe1-5ebffaf759a6","Type":"ContainerStarted","Data":"25bbdb8c2e6d4e2f83cf2193fb5bd1c73de1be3a5d70a8095817108386a528dc"}
Jul 20 16:39:21.610344 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:21.609947    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qhrgc"
Jul 20 16:39:21.627021 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:21.626970    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qhrgc" podStartSLOduration=1.088492037 podStartE2EDuration="6.626954922s" totalImagesPullingTime="5.538462885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:39:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:39:21.62569157 +0000 UTC m=+584.875185600" watchObservedRunningTime="2026-07-20 16:39:21.626954922 +0000 UTC m=+584.876448960"
Jul 20 16:39:27.643066 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:27.643034    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qhrgc"
Jul 20 16:39:27.945623 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:27.945510    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:39:28.189794 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:28.189762    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:39:34.698113 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:34.698078    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:39:34.780152 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:34.780119    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:39:34.843908 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:34.843873    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:39:35.229086 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:35.229049    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:39:37.107205 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:37.107173    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gd4p4_fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c/ovn-acl-logging/0.log"
Jul 20 16:39:37.110421 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:37.110398    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gd4p4_fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c/ovn-acl-logging/0.log"
Jul 20 16:39:38.453484 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:38.453440    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:39:38.549593 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:39:38.549562    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:40:10.089990 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:10.089917    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:40:11.256132 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:11.256099    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-64flz"]
Jul 20 16:40:11.259222 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:11.259203    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-64flz"
Jul 20 16:40:11.261585 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:11.261567    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 20 16:40:11.261685 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:11.261638    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-9p8x4\""
Jul 20 16:40:11.273254 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:11.273229    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-64flz"]
Jul 20 16:40:11.394118 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:11.394070    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbkcv\" (UniqueName: \"kubernetes.io/projected/750883b9-48a6-4050-b1ee-28f3d7d9eda8-kube-api-access-rbkcv\") pod \"payload-processing-5bf47cd988-64flz\" (UID: \"750883b9-48a6-4050-b1ee-28f3d7d9eda8\") " pod="openshift-ingress/payload-processing-5bf47cd988-64flz"
Jul 20 16:40:11.394336 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:11.394192    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/750883b9-48a6-4050-b1ee-28f3d7d9eda8-plugins-config-volume\") pod \"payload-processing-5bf47cd988-64flz\" (UID: \"750883b9-48a6-4050-b1ee-28f3d7d9eda8\") " pod="openshift-ingress/payload-processing-5bf47cd988-64flz"
Jul 20 16:40:11.495340 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:11.495281    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/750883b9-48a6-4050-b1ee-28f3d7d9eda8-plugins-config-volume\") pod \"payload-processing-5bf47cd988-64flz\" (UID: \"750883b9-48a6-4050-b1ee-28f3d7d9eda8\") " pod="openshift-ingress/payload-processing-5bf47cd988-64flz"
Jul 20 16:40:11.495507 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:11.495404    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbkcv\" (UniqueName: \"kubernetes.io/projected/750883b9-48a6-4050-b1ee-28f3d7d9eda8-kube-api-access-rbkcv\") pod \"payload-processing-5bf47cd988-64flz\" (UID: \"750883b9-48a6-4050-b1ee-28f3d7d9eda8\") " pod="openshift-ingress/payload-processing-5bf47cd988-64flz"
Jul 20 16:40:11.496062 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:11.496043    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/750883b9-48a6-4050-b1ee-28f3d7d9eda8-plugins-config-volume\") pod \"payload-processing-5bf47cd988-64flz\" (UID: \"750883b9-48a6-4050-b1ee-28f3d7d9eda8\") " pod="openshift-ingress/payload-processing-5bf47cd988-64flz"
Jul 20 16:40:11.503696 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:11.503674    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbkcv\" (UniqueName: \"kubernetes.io/projected/750883b9-48a6-4050-b1ee-28f3d7d9eda8-kube-api-access-rbkcv\") pod \"payload-processing-5bf47cd988-64flz\" (UID: \"750883b9-48a6-4050-b1ee-28f3d7d9eda8\") " pod="openshift-ingress/payload-processing-5bf47cd988-64flz"
Jul 20 16:40:11.568556 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:11.568533    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-64flz"
Jul 20 16:40:11.690015 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:11.689974    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-64flz"]
Jul 20 16:40:11.692636 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:40:11.692608    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod750883b9_48a6_4050_b1ee_28f3d7d9eda8.slice/crio-a39de963de96d876377ec0ed874bbe761e2a4ca2a0cfc43147dfa88b2927ca01 WatchSource:0}: Error finding container a39de963de96d876377ec0ed874bbe761e2a4ca2a0cfc43147dfa88b2927ca01: Status 404 returned error can't find the container with id a39de963de96d876377ec0ed874bbe761e2a4ca2a0cfc43147dfa88b2927ca01
Jul 20 16:40:11.816442 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:11.816410    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-64flz" event={"ID":"750883b9-48a6-4050-b1ee-28f3d7d9eda8","Type":"ContainerStarted","Data":"a39de963de96d876377ec0ed874bbe761e2a4ca2a0cfc43147dfa88b2927ca01"}
Jul 20 16:40:12.442347 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:12.442290    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:40:13.826514 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:13.826477    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-64flz" event={"ID":"750883b9-48a6-4050-b1ee-28f3d7d9eda8","Type":"ContainerStarted","Data":"8d578d4bd093f0679c7676a82175b28d972d1bffbe7387589e6bb8233ffaa261"}
Jul 20 16:40:13.826888 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:13.826558    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-64flz"
Jul 20 16:40:13.842250 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:13.842205    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-64flz" podStartSLOduration=1.144105988 podStartE2EDuration="2.842189456s" totalImagesPullingTime="1.698083468s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:40:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:40:13.840408215 +0000 UTC m=+637.089902238" watchObservedRunningTime="2026-07-20 16:40:13.842189456 +0000 UTC m=+637.091683474"
Jul 20 16:40:22.862243 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:22.862203    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:40:22.949240 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:22.949207    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:40:24.832358 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:24.832324    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-64flz"
Jul 20 16:40:40.191460 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:40.191423    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs"]
Jul 20 16:40:40.195663 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:40.195640    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs"
Jul 20 16:40:40.199015 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:40.198966    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 20 16:40:40.199015 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:40.199006    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-m2kx8\""
Jul 20 16:40:40.199252 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:40.199055    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 20 16:40:40.199252 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:40.199006    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 20 16:40:40.203062 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:40.203036    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs"]
Jul 20 16:40:40.226950 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:40.226916    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:40:40.334045 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:40.334007    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e53b8d7-5bb7-4d2e-8524-a755abd79661-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs\" (UID: \"3e53b8d7-5bb7-4d2e-8524-a755abd79661\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs"
Jul 20 16:40:40.334217 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:40.334062    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e53b8d7-5bb7-4d2e-8524-a755abd79661-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs\" (UID: \"3e53b8d7-5bb7-4d2e-8524-a755abd79661\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs"
Jul 20 16:40:40.334217 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:40.334106    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e53b8d7-5bb7-4d2e-8524-a755abd79661-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs\" (UID: \"3e53b8d7-5bb7-4d2e-8524-a755abd79661\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs"
Jul 20 16:40:40.334217 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:40.334124    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e53b8d7-5bb7-4d2e-8524-a755abd79661-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs\" (UID: \"3e53b8d7-5bb7-4d2e-8524-a755abd79661\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs"
Jul 20 16:40:40.334217 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:40.334144    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvhnj\" (UniqueName: \"kubernetes.io/projected/3e53b8d7-5bb7-4d2e-8524-a755abd79661-kube-api-access-kvhnj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs\" (UID: \"3e53b8d7-5bb7-4d2e-8524-a755abd79661\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs"
Jul 20 16:40:40.334217 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:40.334165    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e53b8d7-5bb7-4d2e-8524-a755abd79661-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs\" (UID: \"3e53b8d7-5bb7-4d2e-8524-a755abd79661\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs"
Jul 20 16:40:40.435360 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:40.435321    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e53b8d7-5bb7-4d2e-8524-a755abd79661-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs\" (UID: \"3e53b8d7-5bb7-4d2e-8524-a755abd79661\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs"
Jul 20 16:40:40.435360 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:40.435361    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e53b8d7-5bb7-4d2e-8524-a755abd79661-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs\" (UID: \"3e53b8d7-5bb7-4d2e-8524-a755abd79661\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs"
Jul 20 16:40:40.435563 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:40.435384    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvhnj\" (UniqueName: \"kubernetes.io/projected/3e53b8d7-5bb7-4d2e-8524-a755abd79661-kube-api-access-kvhnj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs\" (UID: \"3e53b8d7-5bb7-4d2e-8524-a755abd79661\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs"
Jul 20 16:40:40.435563 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:40.435412    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e53b8d7-5bb7-4d2e-8524-a755abd79661-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs\" (UID: \"3e53b8d7-5bb7-4d2e-8524-a755abd79661\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs"
Jul 20 16:40:40.435563 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:40.435486    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e53b8d7-5bb7-4d2e-8524-a755abd79661-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs\" (UID: \"3e53b8d7-5bb7-4d2e-8524-a755abd79661\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs"
Jul 20 16:40:40.435563 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:40.435535    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e53b8d7-5bb7-4d2e-8524-a755abd79661-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs\" (UID: \"3e53b8d7-5bb7-4d2e-8524-a755abd79661\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs"
Jul 20 16:40:40.435765 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:40.435715    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e53b8d7-5bb7-4d2e-8524-a755abd79661-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs\" (UID: \"3e53b8d7-5bb7-4d2e-8524-a755abd79661\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs"
Jul 20 16:40:40.435818 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:40.435781    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e53b8d7-5bb7-4d2e-8524-a755abd79661-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs\" (UID: \"3e53b8d7-5bb7-4d2e-8524-a755abd79661\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs"
Jul 20 16:40:40.435918 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:40.435897    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e53b8d7-5bb7-4d2e-8524-a755abd79661-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs\" (UID: \"3e53b8d7-5bb7-4d2e-8524-a755abd79661\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs"
Jul 20 16:40:40.437766 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:40.437749    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e53b8d7-5bb7-4d2e-8524-a755abd79661-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs\" (UID: \"3e53b8d7-5bb7-4d2e-8524-a755abd79661\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs"
Jul 20 16:40:40.437922 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:40.437905    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e53b8d7-5bb7-4d2e-8524-a755abd79661-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs\" (UID: \"3e53b8d7-5bb7-4d2e-8524-a755abd79661\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs"
Jul 20 16:40:40.443444 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:40.443392    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvhnj\" (UniqueName: \"kubernetes.io/projected/3e53b8d7-5bb7-4d2e-8524-a755abd79661-kube-api-access-kvhnj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs\" (UID: \"3e53b8d7-5bb7-4d2e-8524-a755abd79661\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs"
Jul 20 16:40:40.507812 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:40.507773    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs"
Jul 20 16:40:40.640655 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:40.640628    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs"]
Jul 20 16:40:40.643157 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:40:40.643127    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e53b8d7_5bb7_4d2e_8524_a755abd79661.slice/crio-0c90bf277921d118034247414e76eb356bfe0c64b0da745d35b1a4be5787e273 WatchSource:0}: Error finding container 0c90bf277921d118034247414e76eb356bfe0c64b0da745d35b1a4be5787e273: Status 404 returned error can't find the container with id 0c90bf277921d118034247414e76eb356bfe0c64b0da745d35b1a4be5787e273
Jul 20 16:40:40.644715 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:40.644695    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 16:40:40.933720 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:40.933681    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs" event={"ID":"3e53b8d7-5bb7-4d2e-8524-a755abd79661","Type":"ContainerStarted","Data":"0c90bf277921d118034247414e76eb356bfe0c64b0da745d35b1a4be5787e273"}
Jul 20 16:40:41.939625 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:41.939589    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs" event={"ID":"3e53b8d7-5bb7-4d2e-8524-a755abd79661","Type":"ContainerStarted","Data":"d88ddea8689884418b128acfaa46c066724a07e4ed19defc7b0af17d9b4e5fb8"}
Jul 20 16:40:41.956995 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:41.956914    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs" podStartSLOduration=0.761445794 podStartE2EDuration="1.956896175s" totalImagesPullingTime="1.195450381s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:40:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:40:41.955282762 +0000 UTC m=+665.204776795" watchObservedRunningTime="2026-07-20 16:40:41.956896175 +0000 UTC m=+665.206390193"
Jul 20 16:40:50.508363 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:50.508319    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs"
Jul 20 16:40:50.508867 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:50.508390    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs"
Jul 20 16:40:50.520699 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:50.520676    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs"
Jul 20 16:40:50.987841 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:40:50.987812    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs"
Jul 20 16:41:09.980121 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:09.980087    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:41:15.420106 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:15.420070    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:41:23.315522 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:23.315483    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:41:30.734142 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:30.734061    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:41:40.084242 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:40.084199    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg"]
Jul 20 16:41:40.087496 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:40.087476    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg"
Jul 20 16:41:40.089915 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:40.089897    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 20 16:41:40.098331 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:40.098309    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg"]
Jul 20 16:41:40.109349 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:40.109327    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:41:40.251187 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:40.251152    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jgn4\" (UniqueName: \"kubernetes.io/projected/e12a1a47-4c70-4eb3-ad22-10711533dc83-kube-api-access-2jgn4\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg\" (UID: \"e12a1a47-4c70-4eb3-ad22-10711533dc83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg"
Jul 20 16:41:40.251187 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:40.251195    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e12a1a47-4c70-4eb3-ad22-10711533dc83-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg\" (UID: \"e12a1a47-4c70-4eb3-ad22-10711533dc83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg"
Jul 20 16:41:40.251460 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:40.251323    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e12a1a47-4c70-4eb3-ad22-10711533dc83-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg\" (UID: \"e12a1a47-4c70-4eb3-ad22-10711533dc83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg"
Jul 20 16:41:40.251460 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:40.251371    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e12a1a47-4c70-4eb3-ad22-10711533dc83-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg\" (UID: \"e12a1a47-4c70-4eb3-ad22-10711533dc83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg"
Jul 20 16:41:40.251460 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:40.251390    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e12a1a47-4c70-4eb3-ad22-10711533dc83-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg\" (UID: \"e12a1a47-4c70-4eb3-ad22-10711533dc83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg"
Jul 20 16:41:40.251460 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:40.251415    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e12a1a47-4c70-4eb3-ad22-10711533dc83-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg\" (UID: \"e12a1a47-4c70-4eb3-ad22-10711533dc83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg"
Jul 20 16:41:40.352699 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:40.352606    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jgn4\" (UniqueName: \"kubernetes.io/projected/e12a1a47-4c70-4eb3-ad22-10711533dc83-kube-api-access-2jgn4\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg\" (UID: \"e12a1a47-4c70-4eb3-ad22-10711533dc83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg"
Jul 20 16:41:40.352699 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:40.352652    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e12a1a47-4c70-4eb3-ad22-10711533dc83-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg\" (UID: \"e12a1a47-4c70-4eb3-ad22-10711533dc83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg"
Jul 20 16:41:40.352699 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:40.352691    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e12a1a47-4c70-4eb3-ad22-10711533dc83-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg\" (UID: \"e12a1a47-4c70-4eb3-ad22-10711533dc83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg"
Jul 20 16:41:40.352976 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:40.352714    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e12a1a47-4c70-4eb3-ad22-10711533dc83-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg\" (UID: \"e12a1a47-4c70-4eb3-ad22-10711533dc83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg"
Jul 20 16:41:40.352976 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:40.352733    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e12a1a47-4c70-4eb3-ad22-10711533dc83-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg\" (UID: \"e12a1a47-4c70-4eb3-ad22-10711533dc83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg"
Jul 20 16:41:40.352976 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:40.352884    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e12a1a47-4c70-4eb3-ad22-10711533dc83-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg\" (UID: \"e12a1a47-4c70-4eb3-ad22-10711533dc83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg"
Jul 20 16:41:40.353117 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:40.353100    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e12a1a47-4c70-4eb3-ad22-10711533dc83-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg\" (UID: \"e12a1a47-4c70-4eb3-ad22-10711533dc83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg"
Jul 20 16:41:40.353160 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:40.353147    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e12a1a47-4c70-4eb3-ad22-10711533dc83-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg\" (UID: \"e12a1a47-4c70-4eb3-ad22-10711533dc83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg"
Jul 20 16:41:40.353201 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:40.353172    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e12a1a47-4c70-4eb3-ad22-10711533dc83-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg\" (UID: \"e12a1a47-4c70-4eb3-ad22-10711533dc83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg"
Jul 20 16:41:40.355095 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:40.355067    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e12a1a47-4c70-4eb3-ad22-10711533dc83-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg\" (UID: \"e12a1a47-4c70-4eb3-ad22-10711533dc83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg"
Jul 20 16:41:40.355289 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:40.355274    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e12a1a47-4c70-4eb3-ad22-10711533dc83-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg\" (UID: \"e12a1a47-4c70-4eb3-ad22-10711533dc83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg"
Jul 20 16:41:40.360262 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:40.360236    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jgn4\" (UniqueName: \"kubernetes.io/projected/e12a1a47-4c70-4eb3-ad22-10711533dc83-kube-api-access-2jgn4\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg\" (UID: \"e12a1a47-4c70-4eb3-ad22-10711533dc83\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg"
Jul 20 16:41:40.396103 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:40.396076    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg"
Jul 20 16:41:40.518638 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:40.518615    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg"]
Jul 20 16:41:40.521473 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:41:40.521450    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode12a1a47_4c70_4eb3_ad22_10711533dc83.slice/crio-d0ba0f89cf7a597b07f71c0dcfd31d98d4227a6370b3ca3c0f7291631dd566a4 WatchSource:0}: Error finding container d0ba0f89cf7a597b07f71c0dcfd31d98d4227a6370b3ca3c0f7291631dd566a4: Status 404 returned error can't find the container with id d0ba0f89cf7a597b07f71c0dcfd31d98d4227a6370b3ca3c0f7291631dd566a4
Jul 20 16:41:41.179123 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:41.179085    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg" event={"ID":"e12a1a47-4c70-4eb3-ad22-10711533dc83","Type":"ContainerStarted","Data":"e795852f11984c2ffe2d7b13280bab9d9719607c3e22e8f9b1f52d3b6b92f1d6"}
Jul 20 16:41:41.179123 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:41.179126    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg" event={"ID":"e12a1a47-4c70-4eb3-ad22-10711533dc83","Type":"ContainerStarted","Data":"d0ba0f89cf7a597b07f71c0dcfd31d98d4227a6370b3ca3c0f7291631dd566a4"}
Jul 20 16:41:41.194953 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:41.194907    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg" podStartSLOduration=1.001491241 podStartE2EDuration="1.194893264s" totalImagesPullingTime="193.402023ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:41:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:41:41.193929509 +0000 UTC m=+724.443423542" watchObservedRunningTime="2026-07-20 16:41:41.194893264 +0000 UTC m=+724.444387309"
Jul 20 16:41:50.396483 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:50.396429    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg"
Jul 20 16:41:50.396483 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:50.396490    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg"
Jul 20 16:41:50.408917 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:50.408889    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg"
Jul 20 16:41:51.231486 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:41:51.231454    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg"
Jul 20 16:42:25.283399 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:42:25.283352    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:42:30.470203 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:42:30.470163    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:42:40.558425 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:42:40.558385    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:42:40.656891 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:42:40.656852    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:42:52.487147 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:42:52.487112    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:42:52.554092 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:42:52.554060    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:42:53.659267 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:42:53.659230    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:42:53.760945 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:42:53.760907    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:43:31.001855 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:43:31.001817    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:44:37.133454 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:44:37.133384    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gd4p4_fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c/ovn-acl-logging/0.log"
Jul 20 16:44:37.134405 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:44:37.134382    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gd4p4_fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c/ovn-acl-logging/0.log"
Jul 20 16:44:58.649618 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:44:58.649569    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:44:59.859676 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:44:59.859640    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:45:00.138656 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:45:00.138571    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742765-7x5tj"]
Jul 20 16:45:00.141914 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:45:00.141896    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742765-7x5tj"
Jul 20 16:45:00.144269 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:45:00.144248    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d9prm\""
Jul 20 16:45:00.149237 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:45:00.149213    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742765-7x5tj"]
Jul 20 16:45:00.152500 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:45:00.152481    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742765-7x5tj"
Jul 20 16:45:00.279244 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:45:00.279222    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742765-7x5tj"]
Jul 20 16:45:00.281956 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:45:00.281929    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bfc237b_41c2_40b3_82f9_596643dcb4eb.slice/crio-6435c2e6de33d25927f7434e21649b1f92d4f3d0badd0b89a1e43accb48b9ffc WatchSource:0}: Error finding container 6435c2e6de33d25927f7434e21649b1f92d4f3d0badd0b89a1e43accb48b9ffc: Status 404 returned error can't find the container with id 6435c2e6de33d25927f7434e21649b1f92d4f3d0badd0b89a1e43accb48b9ffc
Jul 20 16:45:00.996594 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:45:00.996559    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742765-7x5tj" event={"ID":"8bfc237b-41c2-40b3-82f9-596643dcb4eb","Type":"ContainerStarted","Data":"6435c2e6de33d25927f7434e21649b1f92d4f3d0badd0b89a1e43accb48b9ffc"}
Jul 20 16:45:03.007963 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:45:03.007872    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742765-7x5tj" event={"ID":"8bfc237b-41c2-40b3-82f9-596643dcb4eb","Type":"ContainerStarted","Data":"c4b044f16a9fae2e964f7171ff48a0c1dcc462934f7153cfea5cad0582d5cad9"}
Jul 20 16:45:03.021147 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:45:03.021094    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742765-7x5tj" podStartSLOduration=0.707565326 podStartE2EDuration="3.021078252s" totalImagesPullingTime="2.313512926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:45:03.020853867 +0000 UTC m=+926.270347884" watchObservedRunningTime="2026-07-20 16:45:03.021078252 +0000 UTC m=+926.270572270"
Jul 20 16:45:04.014274 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:45:04.014241    2577 generic.go:356] "Generic (PLEG): container finished" podID="8bfc237b-41c2-40b3-82f9-596643dcb4eb" containerID="c4b044f16a9fae2e964f7171ff48a0c1dcc462934f7153cfea5cad0582d5cad9" exitCode=0
Jul 20 16:45:04.014697 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:45:04.014330    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742765-7x5tj" event={"ID":"8bfc237b-41c2-40b3-82f9-596643dcb4eb","Type":"ContainerDied","Data":"c4b044f16a9fae2e964f7171ff48a0c1dcc462934f7153cfea5cad0582d5cad9"}
Jul 20 16:45:05.139556 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:45:05.139530    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742765-7x5tj"
Jul 20 16:45:06.024205 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:45:06.024172    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742765-7x5tj"
Jul 20 16:45:06.024369 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:45:06.024174    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742765-7x5tj" event={"ID":"8bfc237b-41c2-40b3-82f9-596643dcb4eb","Type":"ContainerDied","Data":"6435c2e6de33d25927f7434e21649b1f92d4f3d0badd0b89a1e43accb48b9ffc"}
Jul 20 16:45:06.024369 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:45:06.024324    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6435c2e6de33d25927f7434e21649b1f92d4f3d0badd0b89a1e43accb48b9ffc"
Jul 20 16:45:36.558270 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:45:36.558233    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:45:39.558598 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:45:39.558565    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:45:39.658432 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:45:39.658394    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:45:39.813231 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:45:39.813149    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:45:45.356460 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:45:45.356424    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:45:45.462089 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:45:45.462060    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:45:45.556737 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:45:45.556705    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:45:45.717534 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:45:45.717432    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:45:51.376009 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:45:51.375969    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:45:51.487474 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:45:51.487438    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:47:18.801753 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:47:18.801706    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:48:32.470502 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:48:32.470465    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:48:32.530200 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:48:32.530170    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:49:37.161488 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:49:37.161460    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gd4p4_fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c/ovn-acl-logging/0.log"
Jul 20 16:49:37.162036 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:49:37.161571    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gd4p4_fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c/ovn-acl-logging/0.log"
Jul 20 16:49:49.388273 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:49:49.388234    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:51:02.358635 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:51:02.358595    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:52:46.829079 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:52:46.829047    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:53:42.392425 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:42.392351    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld"]
Jul 20 16:53:42.392845 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:42.392666    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bfc237b-41c2-40b3-82f9-596643dcb4eb" containerName="cleanup"
Jul 20 16:53:42.392845 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:42.392677    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bfc237b-41c2-40b3-82f9-596643dcb4eb" containerName="cleanup"
Jul 20 16:53:42.392845 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:42.392745    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bfc237b-41c2-40b3-82f9-596643dcb4eb" containerName="cleanup"
Jul 20 16:53:42.395498 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:42.395480    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld"
Jul 20 16:53:42.398001 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:42.397977    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-9a8027cc-kserve-self-signed-certs\""
Jul 20 16:53:42.410926 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:42.410904    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld"]
Jul 20 16:53:42.422319 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:42.422281    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:53:42.455609 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:42.455576    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05724bd3-4b2a-45a6-8e97-bf187352205d-tmp-dir\") pod \"e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld\" (UID: \"05724bd3-4b2a-45a6-8e97-bf187352205d\") " pod="llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld"
Jul 20 16:53:42.455734 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:42.455618    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjpqv\" (UniqueName: \"kubernetes.io/projected/05724bd3-4b2a-45a6-8e97-bf187352205d-kube-api-access-rjpqv\") pod \"e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld\" (UID: \"05724bd3-4b2a-45a6-8e97-bf187352205d\") " pod="llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld"
Jul 20 16:53:42.455734 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:42.455662    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05724bd3-4b2a-45a6-8e97-bf187352205d-dshm\") pod \"e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld\" (UID: \"05724bd3-4b2a-45a6-8e97-bf187352205d\") " pod="llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld"
Jul 20 16:53:42.455734 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:42.455713    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05724bd3-4b2a-45a6-8e97-bf187352205d-model-cache\") pod \"e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld\" (UID: \"05724bd3-4b2a-45a6-8e97-bf187352205d\") " pod="llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld"
Jul 20 16:53:42.455874 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:42.455838    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05724bd3-4b2a-45a6-8e97-bf187352205d-tls-certs\") pod \"e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld\" (UID: \"05724bd3-4b2a-45a6-8e97-bf187352205d\") " pod="llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld"
Jul 20 16:53:42.455918 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:42.455899    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05724bd3-4b2a-45a6-8e97-bf187352205d-home\") pod \"e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld\" (UID: \"05724bd3-4b2a-45a6-8e97-bf187352205d\") " pod="llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld"
Jul 20 16:53:42.557184 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:42.557152    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05724bd3-4b2a-45a6-8e97-bf187352205d-tls-certs\") pod \"e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld\" (UID: \"05724bd3-4b2a-45a6-8e97-bf187352205d\") " pod="llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld"
Jul 20 16:53:42.557337 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:42.557201    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05724bd3-4b2a-45a6-8e97-bf187352205d-home\") pod \"e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld\" (UID: \"05724bd3-4b2a-45a6-8e97-bf187352205d\") " pod="llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld"
Jul 20 16:53:42.557337 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:42.557231    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05724bd3-4b2a-45a6-8e97-bf187352205d-tmp-dir\") pod \"e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld\" (UID: \"05724bd3-4b2a-45a6-8e97-bf187352205d\") " pod="llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld"
Jul 20 16:53:42.557337 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:42.557248    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjpqv\" (UniqueName: \"kubernetes.io/projected/05724bd3-4b2a-45a6-8e97-bf187352205d-kube-api-access-rjpqv\") pod \"e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld\" (UID: \"05724bd3-4b2a-45a6-8e97-bf187352205d\") " pod="llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld"
Jul 20 16:53:42.557337 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:42.557272    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05724bd3-4b2a-45a6-8e97-bf187352205d-dshm\") pod \"e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld\" (UID: \"05724bd3-4b2a-45a6-8e97-bf187352205d\") " pod="llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld"
Jul 20 16:53:42.557337 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:42.557327    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05724bd3-4b2a-45a6-8e97-bf187352205d-model-cache\") pod \"e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld\" (UID: \"05724bd3-4b2a-45a6-8e97-bf187352205d\") " pod="llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld"
Jul 20 16:53:42.557758 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:42.557735    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05724bd3-4b2a-45a6-8e97-bf187352205d-home\") pod \"e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld\" (UID: \"05724bd3-4b2a-45a6-8e97-bf187352205d\") " pod="llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld"
Jul 20 16:53:42.557834 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:42.557774    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05724bd3-4b2a-45a6-8e97-bf187352205d-model-cache\") pod \"e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld\" (UID: \"05724bd3-4b2a-45a6-8e97-bf187352205d\") " pod="llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld"
Jul 20 16:53:42.557834 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:42.557799    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05724bd3-4b2a-45a6-8e97-bf187352205d-tmp-dir\") pod \"e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld\" (UID: \"05724bd3-4b2a-45a6-8e97-bf187352205d\") " pod="llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld"
Jul 20 16:53:42.559474 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:42.559447    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05724bd3-4b2a-45a6-8e97-bf187352205d-dshm\") pod \"e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld\" (UID: \"05724bd3-4b2a-45a6-8e97-bf187352205d\") " pod="llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld"
Jul 20 16:53:42.559697 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:42.559677    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05724bd3-4b2a-45a6-8e97-bf187352205d-tls-certs\") pod \"e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld\" (UID: \"05724bd3-4b2a-45a6-8e97-bf187352205d\") " pod="llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld"
Jul 20 16:53:42.564968 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:42.564947    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjpqv\" (UniqueName: \"kubernetes.io/projected/05724bd3-4b2a-45a6-8e97-bf187352205d-kube-api-access-rjpqv\") pod \"e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld\" (UID: \"05724bd3-4b2a-45a6-8e97-bf187352205d\") " pod="llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld"
Jul 20 16:53:42.704136 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:42.704061    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld"
Jul 20 16:53:42.830049 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:42.830024    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld"]
Jul 20 16:53:42.832650 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:53:42.832625    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod05724bd3_4b2a_45a6_8e97_bf187352205d.slice/crio-69e0f0cb58fd2c786a65d947e79ae731b48bab118719aaad70eb282afdabe56b WatchSource:0}: Error finding container 69e0f0cb58fd2c786a65d947e79ae731b48bab118719aaad70eb282afdabe56b: Status 404 returned error can't find the container with id 69e0f0cb58fd2c786a65d947e79ae731b48bab118719aaad70eb282afdabe56b
Jul 20 16:53:43.178053 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:43.178017    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld" event={"ID":"05724bd3-4b2a-45a6-8e97-bf187352205d","Type":"ContainerStarted","Data":"d5efc39db9963bd8c37418cc4d0565516d974494505bbc903cf099c9aedc94e8"}
Jul 20 16:53:43.178226 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:43.178062    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld" event={"ID":"05724bd3-4b2a-45a6-8e97-bf187352205d","Type":"ContainerStarted","Data":"69e0f0cb58fd2c786a65d947e79ae731b48bab118719aaad70eb282afdabe56b"}
Jul 20 16:53:43.196837 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:43.196792    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld" podStartSLOduration=1.19677624 podStartE2EDuration="1.19677624s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:53:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:53:43.194817526 +0000 UTC m=+1446.444311546" watchObservedRunningTime="2026-07-20 16:53:43.19677624 +0000 UTC m=+1446.446270259"
Jul 20 16:53:52.704205 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:52.704120    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld"
Jul 20 16:53:52.704205 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:52.704197    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld"
Jul 20 16:53:52.723867 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:52.723786    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld"
Jul 20 16:53:53.230186 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:53:53.230158    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld"
Jul 20 16:54:01.383998 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:01.383954    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq"]
Jul 20 16:54:01.387365 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:01.387342    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq"
Jul 20 16:54:01.390464 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:01.390445    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-9a8027cc-kserve-self-signed-certs\""
Jul 20 16:54:01.397326 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:01.397287    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq"]
Jul 20 16:54:01.532030 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:01.531991    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-model-cache\") pod \"e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq\" (UID: \"f5fd7a65-edf5-4082-bb6b-f9595e0f36a9\") " pod="llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq"
Jul 20 16:54:01.532030 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:01.532030    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-tls-certs\") pod \"e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq\" (UID: \"f5fd7a65-edf5-4082-bb6b-f9595e0f36a9\") " pod="llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq"
Jul 20 16:54:01.532239 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:01.532053    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv4nn\" (UniqueName: \"kubernetes.io/projected/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-kube-api-access-tv4nn\") pod \"e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq\" (UID: \"f5fd7a65-edf5-4082-bb6b-f9595e0f36a9\") " pod="llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq"
Jul 20 16:54:01.532239 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:01.532106    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-home\") pod \"e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq\" (UID: \"f5fd7a65-edf5-4082-bb6b-f9595e0f36a9\") " pod="llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq"
Jul 20 16:54:01.532239 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:01.532194    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-dshm\") pod \"e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq\" (UID: \"f5fd7a65-edf5-4082-bb6b-f9595e0f36a9\") " pod="llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq"
Jul 20 16:54:01.532239 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:01.532209    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-tmp-dir\") pod \"e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq\" (UID: \"f5fd7a65-edf5-4082-bb6b-f9595e0f36a9\") " pod="llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq"
Jul 20 16:54:01.633506 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:01.633474    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-model-cache\") pod \"e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq\" (UID: \"f5fd7a65-edf5-4082-bb6b-f9595e0f36a9\") " pod="llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq"
Jul 20 16:54:01.633506 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:01.633509    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-tls-certs\") pod \"e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq\" (UID: \"f5fd7a65-edf5-4082-bb6b-f9595e0f36a9\") " pod="llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq"
Jul 20 16:54:01.633702 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:01.633547    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv4nn\" (UniqueName: \"kubernetes.io/projected/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-kube-api-access-tv4nn\") pod \"e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq\" (UID: \"f5fd7a65-edf5-4082-bb6b-f9595e0f36a9\") " pod="llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq"
Jul 20 16:54:01.633702 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:01.633565    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-home\") pod \"e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq\" (UID: \"f5fd7a65-edf5-4082-bb6b-f9595e0f36a9\") " pod="llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq"
Jul 20 16:54:01.633702 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:01.633603    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-dshm\") pod \"e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq\" (UID: \"f5fd7a65-edf5-4082-bb6b-f9595e0f36a9\") " pod="llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq"
Jul 20 16:54:01.633702 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:01.633617    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-tmp-dir\") pod \"e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq\" (UID: \"f5fd7a65-edf5-4082-bb6b-f9595e0f36a9\") " pod="llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq"
Jul 20 16:54:01.633955 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:01.633932    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-model-cache\") pod \"e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq\" (UID: \"f5fd7a65-edf5-4082-bb6b-f9595e0f36a9\") " pod="llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq"
Jul 20 16:54:01.634046 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:01.633944    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-home\") pod \"e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq\" (UID: \"f5fd7a65-edf5-4082-bb6b-f9595e0f36a9\") " pod="llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq"
Jul 20 16:54:01.634046 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:01.634005    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-tmp-dir\") pod \"e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq\" (UID: \"f5fd7a65-edf5-4082-bb6b-f9595e0f36a9\") " pod="llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq"
Jul 20 16:54:01.635891 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:01.635869    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-dshm\") pod \"e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq\" (UID: \"f5fd7a65-edf5-4082-bb6b-f9595e0f36a9\") " pod="llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq"
Jul 20 16:54:01.636004 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:01.635987    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-tls-certs\") pod \"e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq\" (UID: \"f5fd7a65-edf5-4082-bb6b-f9595e0f36a9\") " pod="llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq"
Jul 20 16:54:01.641454 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:01.641434    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv4nn\" (UniqueName: \"kubernetes.io/projected/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-kube-api-access-tv4nn\") pod \"e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq\" (UID: \"f5fd7a65-edf5-4082-bb6b-f9595e0f36a9\") " pod="llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq"
Jul 20 16:54:01.698342 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:01.698291    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq"
Jul 20 16:54:01.824483 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:01.824458    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq"]
Jul 20 16:54:01.826493 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:54:01.826464    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf5fd7a65_edf5_4082_bb6b_f9595e0f36a9.slice/crio-f67104c3942d1372993c7b4f50890110e0bc30f51399732aa320ee68b64829b1 WatchSource:0}: Error finding container f67104c3942d1372993c7b4f50890110e0bc30f51399732aa320ee68b64829b1: Status 404 returned error can't find the container with id f67104c3942d1372993c7b4f50890110e0bc30f51399732aa320ee68b64829b1
Jul 20 16:54:02.260524 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:02.260491    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq" event={"ID":"f5fd7a65-edf5-4082-bb6b-f9595e0f36a9","Type":"ContainerStarted","Data":"def7459c4f5a537e69b6b861f0cc53e75d3ac41dbbcbe4f739802ec4d6a4963a"}
Jul 20 16:54:02.260524 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:02.260530    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq" event={"ID":"f5fd7a65-edf5-4082-bb6b-f9595e0f36a9","Type":"ContainerStarted","Data":"f67104c3942d1372993c7b4f50890110e0bc30f51399732aa320ee68b64829b1"}
Jul 20 16:54:02.301880 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:02.301832    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq" podStartSLOduration=1.301818261 podStartE2EDuration="1.301818261s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:54:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:54:02.300741874 +0000 UTC m=+1465.550235893" watchObservedRunningTime="2026-07-20 16:54:02.301818261 +0000 UTC m=+1465.551312279"
Jul 20 16:54:02.390638 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:02.390601    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:54:11.698801 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:11.698764    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq"
Jul 20 16:54:11.699249 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:11.698817    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq"
Jul 20 16:54:11.711393 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:11.711364    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq"
Jul 20 16:54:12.314195 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:12.314163    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq"
Jul 20 16:54:22.159005 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:22.158964    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld"]
Jul 20 16:54:22.180127 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:22.180084    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:54:23.631888 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:23.631848    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq"]
Jul 20 16:54:23.652879 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:23.652851    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:54:37.179715 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:37.179650    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld" podUID="05724bd3-4b2a-45a6-8e97-bf187352205d" containerName="main" containerID="cri-o://d5efc39db9963bd8c37418cc4d0565516d974494505bbc903cf099c9aedc94e8" gracePeriod=45
Jul 20 16:54:37.187199 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:37.187163    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gd4p4_fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c/ovn-acl-logging/0.log"
Jul 20 16:54:37.188920 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:37.188900    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gd4p4_fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c/ovn-acl-logging/0.log"
Jul 20 16:54:37.414655 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:37.414628    2577 generic.go:356] "Generic (PLEG): container finished" podID="05724bd3-4b2a-45a6-8e97-bf187352205d" containerID="d5efc39db9963bd8c37418cc4d0565516d974494505bbc903cf099c9aedc94e8" exitCode=0
Jul 20 16:54:37.414799 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:37.414704    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld" event={"ID":"05724bd3-4b2a-45a6-8e97-bf187352205d","Type":"ContainerDied","Data":"d5efc39db9963bd8c37418cc4d0565516d974494505bbc903cf099c9aedc94e8"}
Jul 20 16:54:37.429332 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:37.429291    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld"
Jul 20 16:54:37.550516 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:37.550490    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjpqv\" (UniqueName: \"kubernetes.io/projected/05724bd3-4b2a-45a6-8e97-bf187352205d-kube-api-access-rjpqv\") pod \"05724bd3-4b2a-45a6-8e97-bf187352205d\" (UID: \"05724bd3-4b2a-45a6-8e97-bf187352205d\") "
Jul 20 16:54:37.550679 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:37.550526    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05724bd3-4b2a-45a6-8e97-bf187352205d-dshm\") pod \"05724bd3-4b2a-45a6-8e97-bf187352205d\" (UID: \"05724bd3-4b2a-45a6-8e97-bf187352205d\") "
Jul 20 16:54:37.550679 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:37.550600    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05724bd3-4b2a-45a6-8e97-bf187352205d-model-cache\") pod \"05724bd3-4b2a-45a6-8e97-bf187352205d\" (UID: \"05724bd3-4b2a-45a6-8e97-bf187352205d\") "
Jul 20 16:54:37.550679 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:37.550616    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05724bd3-4b2a-45a6-8e97-bf187352205d-home\") pod \"05724bd3-4b2a-45a6-8e97-bf187352205d\" (UID: \"05724bd3-4b2a-45a6-8e97-bf187352205d\") "
Jul 20 16:54:37.550679 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:37.550644    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05724bd3-4b2a-45a6-8e97-bf187352205d-tmp-dir\") pod \"05724bd3-4b2a-45a6-8e97-bf187352205d\" (UID: \"05724bd3-4b2a-45a6-8e97-bf187352205d\") "
Jul 20 16:54:37.550679 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:37.550674    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05724bd3-4b2a-45a6-8e97-bf187352205d-tls-certs\") pod \"05724bd3-4b2a-45a6-8e97-bf187352205d\" (UID: \"05724bd3-4b2a-45a6-8e97-bf187352205d\") "
Jul 20 16:54:37.550947 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:37.550905    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05724bd3-4b2a-45a6-8e97-bf187352205d-home" (OuterVolumeSpecName: "home") pod "05724bd3-4b2a-45a6-8e97-bf187352205d" (UID: "05724bd3-4b2a-45a6-8e97-bf187352205d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:54:37.550947 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:37.550916    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05724bd3-4b2a-45a6-8e97-bf187352205d-model-cache" (OuterVolumeSpecName: "model-cache") pod "05724bd3-4b2a-45a6-8e97-bf187352205d" (UID: "05724bd3-4b2a-45a6-8e97-bf187352205d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:54:37.551052 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:37.550951    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05724bd3-4b2a-45a6-8e97-bf187352205d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "05724bd3-4b2a-45a6-8e97-bf187352205d" (UID: "05724bd3-4b2a-45a6-8e97-bf187352205d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:54:37.551052 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:37.551013    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05724bd3-4b2a-45a6-8e97-bf187352205d-model-cache\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:54:37.551131 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:37.551062    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05724bd3-4b2a-45a6-8e97-bf187352205d-home\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:54:37.552753 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:37.552722    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05724bd3-4b2a-45a6-8e97-bf187352205d-dshm" (OuterVolumeSpecName: "dshm") pod "05724bd3-4b2a-45a6-8e97-bf187352205d" (UID: "05724bd3-4b2a-45a6-8e97-bf187352205d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:54:37.552862 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:37.552842    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05724bd3-4b2a-45a6-8e97-bf187352205d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "05724bd3-4b2a-45a6-8e97-bf187352205d" (UID: "05724bd3-4b2a-45a6-8e97-bf187352205d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:54:37.552968 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:37.552945    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05724bd3-4b2a-45a6-8e97-bf187352205d-kube-api-access-rjpqv" (OuterVolumeSpecName: "kube-api-access-rjpqv") pod "05724bd3-4b2a-45a6-8e97-bf187352205d" (UID: "05724bd3-4b2a-45a6-8e97-bf187352205d"). InnerVolumeSpecName "kube-api-access-rjpqv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:54:37.651825 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:37.651791    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05724bd3-4b2a-45a6-8e97-bf187352205d-tmp-dir\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:54:37.651825 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:37.651817    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05724bd3-4b2a-45a6-8e97-bf187352205d-tls-certs\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:54:37.651825 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:37.651828    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjpqv\" (UniqueName: \"kubernetes.io/projected/05724bd3-4b2a-45a6-8e97-bf187352205d-kube-api-access-rjpqv\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:54:37.652045 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:37.651837    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05724bd3-4b2a-45a6-8e97-bf187352205d-dshm\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:54:38.419847 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:38.419822    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld"
Jul 20 16:54:38.420250 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:38.419846    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld" event={"ID":"05724bd3-4b2a-45a6-8e97-bf187352205d","Type":"ContainerDied","Data":"69e0f0cb58fd2c786a65d947e79ae731b48bab118719aaad70eb282afdabe56b"}
Jul 20 16:54:38.420250 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:38.419896    2577 scope.go:117] "RemoveContainer" containerID="d5efc39db9963bd8c37418cc4d0565516d974494505bbc903cf099c9aedc94e8"
Jul 20 16:54:38.440153 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:38.440128    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld"]
Jul 20 16:54:38.445721 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:38.445701    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-9a8027cc-kserve-85b5895d8c-zxbld"]
Jul 20 16:54:38.654984 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:38.654944    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq" podUID="f5fd7a65-edf5-4082-bb6b-f9595e0f36a9" containerName="main" containerID="cri-o://def7459c4f5a537e69b6b861f0cc53e75d3ac41dbbcbe4f739802ec4d6a4963a" gracePeriod=45
Jul 20 16:54:38.901452 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:38.901426    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq"
Jul 20 16:54:38.964042 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:38.963968    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-tls-certs\") pod \"f5fd7a65-edf5-4082-bb6b-f9595e0f36a9\" (UID: \"f5fd7a65-edf5-4082-bb6b-f9595e0f36a9\") "
Jul 20 16:54:38.964042 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:38.964014    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tv4nn\" (UniqueName: \"kubernetes.io/projected/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-kube-api-access-tv4nn\") pod \"f5fd7a65-edf5-4082-bb6b-f9595e0f36a9\" (UID: \"f5fd7a65-edf5-4082-bb6b-f9595e0f36a9\") "
Jul 20 16:54:38.964227 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:38.964048    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-home\") pod \"f5fd7a65-edf5-4082-bb6b-f9595e0f36a9\" (UID: \"f5fd7a65-edf5-4082-bb6b-f9595e0f36a9\") "
Jul 20 16:54:38.964227 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:38.964074    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-dshm\") pod \"f5fd7a65-edf5-4082-bb6b-f9595e0f36a9\" (UID: \"f5fd7a65-edf5-4082-bb6b-f9595e0f36a9\") "
Jul 20 16:54:38.964227 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:38.964130    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-model-cache\") pod \"f5fd7a65-edf5-4082-bb6b-f9595e0f36a9\" (UID: \"f5fd7a65-edf5-4082-bb6b-f9595e0f36a9\") "
Jul 20 16:54:38.964227 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:38.964168    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-tmp-dir\") pod \"f5fd7a65-edf5-4082-bb6b-f9595e0f36a9\" (UID: \"f5fd7a65-edf5-4082-bb6b-f9595e0f36a9\") "
Jul 20 16:54:38.964540 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:38.964390    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-home" (OuterVolumeSpecName: "home") pod "f5fd7a65-edf5-4082-bb6b-f9595e0f36a9" (UID: "f5fd7a65-edf5-4082-bb6b-f9595e0f36a9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:54:38.964608 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:38.964578    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f5fd7a65-edf5-4082-bb6b-f9595e0f36a9" (UID: "f5fd7a65-edf5-4082-bb6b-f9595e0f36a9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:54:38.964669 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:38.964650    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-model-cache" (OuterVolumeSpecName: "model-cache") pod "f5fd7a65-edf5-4082-bb6b-f9595e0f36a9" (UID: "f5fd7a65-edf5-4082-bb6b-f9595e0f36a9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:54:38.966208 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:38.966184    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f5fd7a65-edf5-4082-bb6b-f9595e0f36a9" (UID: "f5fd7a65-edf5-4082-bb6b-f9595e0f36a9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:54:38.966208 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:38.966194    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-dshm" (OuterVolumeSpecName: "dshm") pod "f5fd7a65-edf5-4082-bb6b-f9595e0f36a9" (UID: "f5fd7a65-edf5-4082-bb6b-f9595e0f36a9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:54:38.966369 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:38.966262    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-kube-api-access-tv4nn" (OuterVolumeSpecName: "kube-api-access-tv4nn") pod "f5fd7a65-edf5-4082-bb6b-f9595e0f36a9" (UID: "f5fd7a65-edf5-4082-bb6b-f9595e0f36a9"). InnerVolumeSpecName "kube-api-access-tv4nn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:54:39.065440 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:39.065410    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-tls-certs\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:54:39.065440 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:39.065437    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tv4nn\" (UniqueName: \"kubernetes.io/projected/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-kube-api-access-tv4nn\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:54:39.065440 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:39.065447    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-home\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:54:39.065654 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:39.065455    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-dshm\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:54:39.065654 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:39.065465    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-model-cache\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:54:39.065654 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:39.065473    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9-tmp-dir\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:54:39.211448 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:39.211417    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05724bd3-4b2a-45a6-8e97-bf187352205d" path="/var/lib/kubelet/pods/05724bd3-4b2a-45a6-8e97-bf187352205d/volumes"
Jul 20 16:54:39.428571 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:39.428534    2577 generic.go:356] "Generic (PLEG): container finished" podID="f5fd7a65-edf5-4082-bb6b-f9595e0f36a9" containerID="def7459c4f5a537e69b6b861f0cc53e75d3ac41dbbcbe4f739802ec4d6a4963a" exitCode=0
Jul 20 16:54:39.428956 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:39.428578    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq"
Jul 20 16:54:39.428956 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:39.428615    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq" event={"ID":"f5fd7a65-edf5-4082-bb6b-f9595e0f36a9","Type":"ContainerDied","Data":"def7459c4f5a537e69b6b861f0cc53e75d3ac41dbbcbe4f739802ec4d6a4963a"}
Jul 20 16:54:39.428956 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:39.428659    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq" event={"ID":"f5fd7a65-edf5-4082-bb6b-f9595e0f36a9","Type":"ContainerDied","Data":"f67104c3942d1372993c7b4f50890110e0bc30f51399732aa320ee68b64829b1"}
Jul 20 16:54:39.428956 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:39.428680    2577 scope.go:117] "RemoveContainer" containerID="def7459c4f5a537e69b6b861f0cc53e75d3ac41dbbcbe4f739802ec4d6a4963a"
Jul 20 16:54:39.436796 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:39.436766    2577 scope.go:117] "RemoveContainer" containerID="def7459c4f5a537e69b6b861f0cc53e75d3ac41dbbcbe4f739802ec4d6a4963a"
Jul 20 16:54:39.437008 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:54:39.436991    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"def7459c4f5a537e69b6b861f0cc53e75d3ac41dbbcbe4f739802ec4d6a4963a\": container with ID starting with def7459c4f5a537e69b6b861f0cc53e75d3ac41dbbcbe4f739802ec4d6a4963a not found: ID does not exist" containerID="def7459c4f5a537e69b6b861f0cc53e75d3ac41dbbcbe4f739802ec4d6a4963a"
Jul 20 16:54:39.437053 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:39.437015    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"def7459c4f5a537e69b6b861f0cc53e75d3ac41dbbcbe4f739802ec4d6a4963a"} err="failed to get container status \"def7459c4f5a537e69b6b861f0cc53e75d3ac41dbbcbe4f739802ec4d6a4963a\": rpc error: code = NotFound desc = could not find container \"def7459c4f5a537e69b6b861f0cc53e75d3ac41dbbcbe4f739802ec4d6a4963a\": container with ID starting with def7459c4f5a537e69b6b861f0cc53e75d3ac41dbbcbe4f739802ec4d6a4963a not found: ID does not exist"
Jul 20 16:54:39.445966 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:39.445945    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq"]
Jul 20 16:54:39.449723 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:39.449703    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-9a8027cc-kserve-644c598c79-dz8xq"]
Jul 20 16:54:41.212094 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:54:41.212062    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5fd7a65-edf5-4082-bb6b-f9595e0f36a9" path="/var/lib/kubelet/pods/f5fd7a65-edf5-4082-bb6b-f9595e0f36a9/volumes"
Jul 20 16:57:39.109516 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:57:39.109480    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:57:47.083275 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:57:47.083244    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:57:55.348158 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:57:55.348110    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:58:00.720489 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:00.720415    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:58:03.647127 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:03.647089    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:58:03.748405 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:03.748363    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:58:04.100805 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:04.100763    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:58:09.771983 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:09.771951    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:58:09.841047 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:09.841017    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:58:10.096092 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:10.096061    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:58:10.351639 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:10.351541    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:58:10.650074 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:10.649996    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:58:12.238005 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:12.237968    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:58:12.346695 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:12.346666    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:58:12.424056 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:12.424023    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf"]
Jul 20 16:58:12.424441 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:12.424377    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05724bd3-4b2a-45a6-8e97-bf187352205d" containerName="main"
Jul 20 16:58:12.424441 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:12.424390    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="05724bd3-4b2a-45a6-8e97-bf187352205d" containerName="main"
Jul 20 16:58:12.424441 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:12.424398    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5fd7a65-edf5-4082-bb6b-f9595e0f36a9" containerName="main"
Jul 20 16:58:12.424441 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:12.424403    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5fd7a65-edf5-4082-bb6b-f9595e0f36a9" containerName="main"
Jul 20 16:58:12.424902 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:12.424472    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5fd7a65-edf5-4082-bb6b-f9595e0f36a9" containerName="main"
Jul 20 16:58:12.424902 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:12.424480    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="05724bd3-4b2a-45a6-8e97-bf187352205d" containerName="main"
Jul 20 16:58:12.427657 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:12.427635    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf"
Jul 20 16:58:12.430138 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:12.430117    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-d88d04e6-serving-cert\""
Jul 20 16:58:12.430232 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:12.430129    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-kxz6w\""
Jul 20 16:58:12.439239 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:12.439188    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf"]
Jul 20 16:58:12.476773 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:12.476732    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:58:12.592090 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:12.592056    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9299523e-826e-4c2a-bf08-09093f0d8dd8-openshift-service-ca\") pod \"maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf\" (UID: \"9299523e-826e-4c2a-bf08-09093f0d8dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf"
Jul 20 16:58:12.592255 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:12.592100    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwf6l\" (UniqueName: \"kubernetes.io/projected/9299523e-826e-4c2a-bf08-09093f0d8dd8-kube-api-access-wwf6l\") pod \"maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf\" (UID: \"9299523e-826e-4c2a-bf08-09093f0d8dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf"
Jul 20 16:58:12.592255 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:12.592184    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9299523e-826e-4c2a-bf08-09093f0d8dd8-maas-api-tls\") pod \"maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf\" (UID: \"9299523e-826e-4c2a-bf08-09093f0d8dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf"
Jul 20 16:58:12.660815 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:12.660786    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:58:12.693171 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:12.693142    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9299523e-826e-4c2a-bf08-09093f0d8dd8-maas-api-tls\") pod \"maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf\" (UID: \"9299523e-826e-4c2a-bf08-09093f0d8dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf"
Jul 20 16:58:12.693339 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:12.693190    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9299523e-826e-4c2a-bf08-09093f0d8dd8-openshift-service-ca\") pod \"maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf\" (UID: \"9299523e-826e-4c2a-bf08-09093f0d8dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf"
Jul 20 16:58:12.693339 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:12.693219    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwf6l\" (UniqueName: \"kubernetes.io/projected/9299523e-826e-4c2a-bf08-09093f0d8dd8-kube-api-access-wwf6l\") pod \"maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf\" (UID: \"9299523e-826e-4c2a-bf08-09093f0d8dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf"
Jul 20 16:58:12.693847 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:12.693827    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9299523e-826e-4c2a-bf08-09093f0d8dd8-openshift-service-ca\") pod \"maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf\" (UID: \"9299523e-826e-4c2a-bf08-09093f0d8dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf"
Jul 20 16:58:12.695484 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:12.695466    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9299523e-826e-4c2a-bf08-09093f0d8dd8-maas-api-tls\") pod \"maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf\" (UID: \"9299523e-826e-4c2a-bf08-09093f0d8dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf"
Jul 20 16:58:12.702620 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:12.702598    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwf6l\" (UniqueName: \"kubernetes.io/projected/9299523e-826e-4c2a-bf08-09093f0d8dd8-kube-api-access-wwf6l\") pod \"maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf\" (UID: \"9299523e-826e-4c2a-bf08-09093f0d8dd8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf"
Jul 20 16:58:12.740145 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:12.740121    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf"
Jul 20 16:58:12.874249 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:12.874199    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf"]
Jul 20 16:58:12.876546 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:58:12.876513    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9299523e_826e_4c2a_bf08_09093f0d8dd8.slice/crio-0ed20da72a310aa9796aedef65689e0e43682d60376890d9b22992bbc5d44d25 WatchSource:0}: Error finding container 0ed20da72a310aa9796aedef65689e0e43682d60376890d9b22992bbc5d44d25: Status 404 returned error can't find the container with id 0ed20da72a310aa9796aedef65689e0e43682d60376890d9b22992bbc5d44d25
Jul 20 16:58:12.877819 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:12.877804    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 16:58:12.945806 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:12.945775    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:58:13.053357 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:13.053327    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:58:13.327589 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:13.327555    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf" event={"ID":"9299523e-826e-4c2a-bf08-09093f0d8dd8","Type":"ContainerStarted","Data":"0ed20da72a310aa9796aedef65689e0e43682d60376890d9b22992bbc5d44d25"}
Jul 20 16:58:15.340385 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:15.340334    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf" event={"ID":"9299523e-826e-4c2a-bf08-09093f0d8dd8","Type":"ContainerStarted","Data":"0c668d4d0b8d30967ef3ad655ccb285a579c581ab5ae1c0b930905511a81f7e2"}
Jul 20 16:58:15.340886 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:15.340449    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf"
Jul 20 16:58:15.361672 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:15.361617    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf" podStartSLOduration=1.047579579 podStartE2EDuration="3.361600557s" totalImagesPullingTime="2.314020978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:58:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:58:15.360659449 +0000 UTC m=+1718.610153481" watchObservedRunningTime="2026-07-20 16:58:15.361600557 +0000 UTC m=+1718.611094575"
Jul 20 16:58:21.350395 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:21.350368    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf"
Jul 20 16:58:25.748273 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:25.748233    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:58:37.849407 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:37.849369    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d88d04e6-mt5ht"]
Jul 20 16:58:37.853109 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:37.853090    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d88d04e6-mt5ht"
Jul 20 16:58:37.855446 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:37.855426    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d9prm\""
Jul 20 16:58:37.859321 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:37.859280    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d88d04e6-mt5ht"]
Jul 20 16:58:37.918918 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:37.918887    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcce51e9-2038-472d-8221-06a666e14e2a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d88d04e6-mt5ht\" (UID: \"bcce51e9-2038-472d-8221-06a666e14e2a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d88d04e6-mt5ht"
Jul 20 16:58:38.020172 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:38.020132    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcce51e9-2038-472d-8221-06a666e14e2a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d88d04e6-mt5ht\" (UID: \"bcce51e9-2038-472d-8221-06a666e14e2a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d88d04e6-mt5ht"
Jul 20 16:58:38.020768 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:38.020746    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcce51e9-2038-472d-8221-06a666e14e2a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d88d04e6-mt5ht\" (UID: \"bcce51e9-2038-472d-8221-06a666e14e2a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d88d04e6-mt5ht"
Jul 20 16:58:38.164427 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:38.164333    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d88d04e6-mt5ht"
Jul 20 16:58:38.289932 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:38.289907    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d88d04e6-mt5ht"]
Jul 20 16:58:38.291753 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:58:38.291725    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcce51e9_2038_472d_8221_06a666e14e2a.slice/crio-567dbeb36c78be58dc1fd36fb8c3c5de27d405fbfc958c186ac9b6a8f85c3636 WatchSource:0}: Error finding container 567dbeb36c78be58dc1fd36fb8c3c5de27d405fbfc958c186ac9b6a8f85c3636: Status 404 returned error can't find the container with id 567dbeb36c78be58dc1fd36fb8c3c5de27d405fbfc958c186ac9b6a8f85c3636
Jul 20 16:58:38.442319 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:38.442236    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d88d04e6-mt5ht" event={"ID":"bcce51e9-2038-472d-8221-06a666e14e2a","Type":"ContainerStarted","Data":"567dbeb36c78be58dc1fd36fb8c3c5de27d405fbfc958c186ac9b6a8f85c3636"}
Jul 20 16:58:39.447531 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:39.447496    2577 generic.go:356] "Generic (PLEG): container finished" podID="bcce51e9-2038-472d-8221-06a666e14e2a" containerID="1ab2dc1806cd2dc3254b65f8f635e9057ddbed30e6b72525aba6b9627a99635d" exitCode=0
Jul 20 16:58:39.447914 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:39.447568    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d88d04e6-mt5ht" event={"ID":"bcce51e9-2038-472d-8221-06a666e14e2a","Type":"ContainerDied","Data":"1ab2dc1806cd2dc3254b65f8f635e9057ddbed30e6b72525aba6b9627a99635d"}
Jul 20 16:58:40.578726 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:40.578701    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d88d04e6-mt5ht"
Jul 20 16:58:40.746656 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:40.746567    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcce51e9-2038-472d-8221-06a666e14e2a-maas-api-service-ca\") pod \"bcce51e9-2038-472d-8221-06a666e14e2a\" (UID: \"bcce51e9-2038-472d-8221-06a666e14e2a\") "
Jul 20 16:58:40.746983 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:40.746956    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcce51e9-2038-472d-8221-06a666e14e2a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bcce51e9-2038-472d-8221-06a666e14e2a" (UID: "bcce51e9-2038-472d-8221-06a666e14e2a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:58:40.847680 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:40.847641    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcce51e9-2038-472d-8221-06a666e14e2a-maas-api-service-ca\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:58:41.458056 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:41.458026    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d88d04e6-mt5ht"
Jul 20 16:58:41.458056 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:41.458031    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d88d04e6-mt5ht" event={"ID":"bcce51e9-2038-472d-8221-06a666e14e2a","Type":"ContainerDied","Data":"567dbeb36c78be58dc1fd36fb8c3c5de27d405fbfc958c186ac9b6a8f85c3636"}
Jul 20 16:58:41.458341 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:41.458074    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="567dbeb36c78be58dc1fd36fb8c3c5de27d405fbfc958c186ac9b6a8f85c3636"
Jul 20 16:58:48.296565 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:48.296533    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf"]
Jul 20 16:58:48.297016 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:48.296795    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf" podUID="9299523e-826e-4c2a-bf08-09093f0d8dd8" containerName="maas-api" containerID="cri-o://0c668d4d0b8d30967ef3ad655ccb285a579c581ab5ae1c0b930905511a81f7e2" gracePeriod=30
Jul 20 16:58:48.324349 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:48.324318    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:58:48.489642 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:48.489611    2577 generic.go:356] "Generic (PLEG): container finished" podID="9299523e-826e-4c2a-bf08-09093f0d8dd8" containerID="0c668d4d0b8d30967ef3ad655ccb285a579c581ab5ae1c0b930905511a81f7e2" exitCode=0
Jul 20 16:58:48.489796 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:48.489690    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf" event={"ID":"9299523e-826e-4c2a-bf08-09093f0d8dd8","Type":"ContainerDied","Data":"0c668d4d0b8d30967ef3ad655ccb285a579c581ab5ae1c0b930905511a81f7e2"}
Jul 20 16:58:48.539689 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:48.539662    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf"
Jul 20 16:58:48.717429 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:48.717328    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9299523e-826e-4c2a-bf08-09093f0d8dd8-openshift-service-ca\") pod \"9299523e-826e-4c2a-bf08-09093f0d8dd8\" (UID: \"9299523e-826e-4c2a-bf08-09093f0d8dd8\") "
Jul 20 16:58:48.717612 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:48.717442    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wwf6l\" (UniqueName: \"kubernetes.io/projected/9299523e-826e-4c2a-bf08-09093f0d8dd8-kube-api-access-wwf6l\") pod \"9299523e-826e-4c2a-bf08-09093f0d8dd8\" (UID: \"9299523e-826e-4c2a-bf08-09093f0d8dd8\") "
Jul 20 16:58:48.717612 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:48.717506    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9299523e-826e-4c2a-bf08-09093f0d8dd8-maas-api-tls\") pod \"9299523e-826e-4c2a-bf08-09093f0d8dd8\" (UID: \"9299523e-826e-4c2a-bf08-09093f0d8dd8\") "
Jul 20 16:58:48.717762 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:48.717731    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9299523e-826e-4c2a-bf08-09093f0d8dd8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9299523e-826e-4c2a-bf08-09093f0d8dd8" (UID: "9299523e-826e-4c2a-bf08-09093f0d8dd8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:58:48.719608 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:48.719576    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9299523e-826e-4c2a-bf08-09093f0d8dd8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9299523e-826e-4c2a-bf08-09093f0d8dd8" (UID: "9299523e-826e-4c2a-bf08-09093f0d8dd8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:58:48.719729 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:48.719663    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9299523e-826e-4c2a-bf08-09093f0d8dd8-kube-api-access-wwf6l" (OuterVolumeSpecName: "kube-api-access-wwf6l") pod "9299523e-826e-4c2a-bf08-09093f0d8dd8" (UID: "9299523e-826e-4c2a-bf08-09093f0d8dd8"). InnerVolumeSpecName "kube-api-access-wwf6l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:58:48.818736 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:48.818704    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9299523e-826e-4c2a-bf08-09093f0d8dd8-openshift-service-ca\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:58:48.818736 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:48.818730    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwf6l\" (UniqueName: \"kubernetes.io/projected/9299523e-826e-4c2a-bf08-09093f0d8dd8-kube-api-access-wwf6l\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:58:48.818736 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:48.818741    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9299523e-826e-4c2a-bf08-09093f0d8dd8-maas-api-tls\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:58:49.495123 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:49.495048    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf"
Jul 20 16:58:49.495123 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:49.495065    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf" event={"ID":"9299523e-826e-4c2a-bf08-09093f0d8dd8","Type":"ContainerDied","Data":"0ed20da72a310aa9796aedef65689e0e43682d60376890d9b22992bbc5d44d25"}
Jul 20 16:58:49.495123 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:49.495118    2577 scope.go:117] "RemoveContainer" containerID="0c668d4d0b8d30967ef3ad655ccb285a579c581ab5ae1c0b930905511a81f7e2"
Jul 20 16:58:49.510025 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:49.510001    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf"]
Jul 20 16:58:49.514133 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:49.514107    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d88d04e6-679d8759c4-dg7vf"]
Jul 20 16:58:51.212821 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:51.212784    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9299523e-826e-4c2a-bf08-09093f0d8dd8" path="/var/lib/kubelet/pods/9299523e-826e-4c2a-bf08-09093f0d8dd8/volumes"
Jul 20 16:58:52.937756 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:52.937720    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d88d04e6-mt5ht"]
Jul 20 16:58:52.941090 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:52.941064    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d88d04e6-mt5ht"]
Jul 20 16:58:52.944025 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:52.944003    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:58:53.131026 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:53.130987    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:58:53.211986 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:53.211908    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcce51e9-2038-472d-8221-06a666e14e2a" path="/var/lib/kubelet/pods/bcce51e9-2038-472d-8221-06a666e14e2a/volumes"
Jul 20 16:58:53.353938 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:53.353905    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:58:53.611117 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:53.611084    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:58:53.710017 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:53.709984    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:58:53.812556 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:53.812506    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:58:54.183654 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.183616    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"]
Jul 20 16:58:54.184016 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.183942    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcce51e9-2038-472d-8221-06a666e14e2a" containerName="revoke-keys"
Jul 20 16:58:54.184016 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.183971    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcce51e9-2038-472d-8221-06a666e14e2a" containerName="revoke-keys"
Jul 20 16:58:54.184016 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.184002    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9299523e-826e-4c2a-bf08-09093f0d8dd8" containerName="maas-api"
Jul 20 16:58:54.184016 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.184009    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9299523e-826e-4c2a-bf08-09093f0d8dd8" containerName="maas-api"
Jul 20 16:58:54.184146 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.184058    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9299523e-826e-4c2a-bf08-09093f0d8dd8" containerName="maas-api"
Jul 20 16:58:54.184146 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.184073    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcce51e9-2038-472d-8221-06a666e14e2a" containerName="revoke-keys"
Jul 20 16:58:54.190747 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.190715    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:58:54.193428 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.193405    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-3e15ca90-gw-openshift-default-dockercfg-7qr4w\""
Jul 20 16:58:54.193822 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.193801    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 20 16:58:54.205194 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.205174    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"]
Jul 20 16:58:54.210280 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.210261    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:58:54.364229 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.364189    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/481f4335-a556-49a1-8ffb-b8bac474f08b-credential-socket\") pod \"e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") " pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:58:54.364229 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.364228    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/481f4335-a556-49a1-8ffb-b8bac474f08b-istio-data\") pod \"e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") " pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:58:54.364482 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.364267    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/481f4335-a556-49a1-8ffb-b8bac474f08b-istio-envoy\") pod \"e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") " pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:58:54.364482 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.364321    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/481f4335-a556-49a1-8ffb-b8bac474f08b-istiod-ca-cert\") pod \"e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") " pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:58:54.364482 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.364423    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/481f4335-a556-49a1-8ffb-b8bac474f08b-workload-socket\") pod \"e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") " pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:58:54.364580 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.364499    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/481f4335-a556-49a1-8ffb-b8bac474f08b-istio-token\") pod \"e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") " pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:58:54.364580 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.364528    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vr6f7\" (UniqueName: \"kubernetes.io/projected/481f4335-a556-49a1-8ffb-b8bac474f08b-kube-api-access-vr6f7\") pod \"e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") " pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:58:54.364653 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.364578    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/481f4335-a556-49a1-8ffb-b8bac474f08b-workload-certs\") pod \"e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") " pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:58:54.364653 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.364594    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/481f4335-a556-49a1-8ffb-b8bac474f08b-istio-podinfo\") pod \"e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") " pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:58:54.465683 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.465595    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/481f4335-a556-49a1-8ffb-b8bac474f08b-credential-socket\") pod \"e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") " pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:58:54.465683 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.465628    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/481f4335-a556-49a1-8ffb-b8bac474f08b-istio-data\") pod \"e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") " pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:58:54.465683 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.465649    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/481f4335-a556-49a1-8ffb-b8bac474f08b-istio-envoy\") pod \"e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") " pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:58:54.465683 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.465670    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/481f4335-a556-49a1-8ffb-b8bac474f08b-istiod-ca-cert\") pod \"e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") " pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:58:54.466005 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.465714    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/481f4335-a556-49a1-8ffb-b8bac474f08b-workload-socket\") pod \"e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") " pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:58:54.466005 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.465762    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/481f4335-a556-49a1-8ffb-b8bac474f08b-istio-token\") pod \"e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") " pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:58:54.466005 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.465783    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vr6f7\" (UniqueName: \"kubernetes.io/projected/481f4335-a556-49a1-8ffb-b8bac474f08b-kube-api-access-vr6f7\") pod \"e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") " pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:58:54.466005 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.465823    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/481f4335-a556-49a1-8ffb-b8bac474f08b-workload-certs\") pod \"e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") " pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:58:54.466005 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.465847    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/481f4335-a556-49a1-8ffb-b8bac474f08b-istio-podinfo\") pod \"e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") " pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:58:54.466286 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.466021    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/481f4335-a556-49a1-8ffb-b8bac474f08b-credential-socket\") pod \"e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") " pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:58:54.466286 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.466067    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/481f4335-a556-49a1-8ffb-b8bac474f08b-istio-data\") pod \"e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") " pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:58:54.466286 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.466159    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/481f4335-a556-49a1-8ffb-b8bac474f08b-workload-socket\") pod \"e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") " pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:58:54.466464 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.466341    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/481f4335-a556-49a1-8ffb-b8bac474f08b-workload-certs\") pod \"e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") " pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:58:54.466804 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.466778    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/481f4335-a556-49a1-8ffb-b8bac474f08b-istiod-ca-cert\") pod \"e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") " pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:58:54.468070 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.468040    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/481f4335-a556-49a1-8ffb-b8bac474f08b-istio-envoy\") pod \"e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") " pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:58:54.468382 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.468362    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/481f4335-a556-49a1-8ffb-b8bac474f08b-istio-podinfo\") pod \"e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") " pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:58:54.485525 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.485499    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/481f4335-a556-49a1-8ffb-b8bac474f08b-istio-token\") pod \"e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") " pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:58:54.496275 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.496252    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vr6f7\" (UniqueName: \"kubernetes.io/projected/481f4335-a556-49a1-8ffb-b8bac474f08b-kube-api-access-vr6f7\") pod \"e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") " pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:58:54.501128 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.501107    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:58:54.719621 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.719593    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"]
Jul 20 16:58:54.721977 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:58:54.721945    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod481f4335_a556_49a1_8ffb_b8bac474f08b.slice/crio-1fe95230d2b43dee4d9d78eea76c65d3e671600975aa66b377c534b5c08acbae WatchSource:0}: Error finding container 1fe95230d2b43dee4d9d78eea76c65d3e671600975aa66b377c534b5c08acbae: Status 404 returned error can't find the container with id 1fe95230d2b43dee4d9d78eea76c65d3e671600975aa66b377c534b5c08acbae
Jul 20 16:58:54.777392 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:54.777361    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:58:55.531218 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:55.531176    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258" event={"ID":"481f4335-a556-49a1-8ffb-b8bac474f08b","Type":"ContainerStarted","Data":"1fe95230d2b43dee4d9d78eea76c65d3e671600975aa66b377c534b5c08acbae"}
Jul 20 16:58:57.086697 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:57.086660    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 16:58:57.086977 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:57.086731    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 16:58:57.086977 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:57.086758    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 16:58:57.542487 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:57.542454    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258" event={"ID":"481f4335-a556-49a1-8ffb-b8bac474f08b","Type":"ContainerStarted","Data":"b6556cce0469ee9288b83329e8b5fdf16e3ca7442c78a613d764b66bb32401e1"}
Jul 20 16:58:57.572897 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:57.572845    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258" podStartSLOduration=1.210357828 podStartE2EDuration="3.572824528s" totalImagesPullingTime="2.3624667s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:58:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:58:57.570420324 +0000 UTC m=+1760.819914357" watchObservedRunningTime="2026-07-20 16:58:57.572824528 +0000 UTC m=+1760.822318549"
Jul 20 16:58:58.501558 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:58.501519    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:58:58.506511 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:58.506484    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:58:58.546725 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:58.546689    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:58:58.547582 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:58.547563    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:58:59.044582 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:59.044549    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:58:59.132630 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:59.132596    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w"]
Jul 20 16:58:59.135863 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:59.135847    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w"
Jul 20 16:58:59.138430 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:59.138410    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-kxz6w\""
Jul 20 16:58:59.138548 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:59.138448    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-3e15ca90-serving-cert\""
Jul 20 16:58:59.144269 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:59.144245    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w"]
Jul 20 16:58:59.212761 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:59.212731    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c48c6055-bcd9-4470-86fe-f18d07399098-openshift-service-ca\") pod \"maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w\" (UID: \"c48c6055-bcd9-4470-86fe-f18d07399098\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w"
Jul 20 16:58:59.212936 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:59.212825    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ttkn\" (UniqueName: \"kubernetes.io/projected/c48c6055-bcd9-4470-86fe-f18d07399098-kube-api-access-8ttkn\") pod \"maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w\" (UID: \"c48c6055-bcd9-4470-86fe-f18d07399098\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w"
Jul 20 16:58:59.212936 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:59.212870    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c48c6055-bcd9-4470-86fe-f18d07399098-maas-api-tls\") pod \"maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w\" (UID: \"c48c6055-bcd9-4470-86fe-f18d07399098\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w"
Jul 20 16:58:59.235092 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:59.235054    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:58:59.313590 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:59.313497    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c48c6055-bcd9-4470-86fe-f18d07399098-openshift-service-ca\") pod \"maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w\" (UID: \"c48c6055-bcd9-4470-86fe-f18d07399098\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w"
Jul 20 16:58:59.313590 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:59.313558    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ttkn\" (UniqueName: \"kubernetes.io/projected/c48c6055-bcd9-4470-86fe-f18d07399098-kube-api-access-8ttkn\") pod \"maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w\" (UID: \"c48c6055-bcd9-4470-86fe-f18d07399098\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w"
Jul 20 16:58:59.313783 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:59.313591    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c48c6055-bcd9-4470-86fe-f18d07399098-maas-api-tls\") pod \"maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w\" (UID: \"c48c6055-bcd9-4470-86fe-f18d07399098\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w"
Jul 20 16:58:59.313783 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:58:59.313753    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-3e15ca90-serving-cert: secret "maas-api-e2e-migrate-3e15ca90-serving-cert" not found
Jul 20 16:58:59.313861 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:58:59.313817    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c48c6055-bcd9-4470-86fe-f18d07399098-maas-api-tls podName:c48c6055-bcd9-4470-86fe-f18d07399098 nodeName:}" failed. No retries permitted until 2026-07-20 16:58:59.813796872 +0000 UTC m=+1763.063290884 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c48c6055-bcd9-4470-86fe-f18d07399098-maas-api-tls") pod "maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w" (UID: "c48c6055-bcd9-4470-86fe-f18d07399098") : secret "maas-api-e2e-migrate-3e15ca90-serving-cert" not found
Jul 20 16:58:59.314211 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:59.314193    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c48c6055-bcd9-4470-86fe-f18d07399098-openshift-service-ca\") pod \"maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w\" (UID: \"c48c6055-bcd9-4470-86fe-f18d07399098\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w"
Jul 20 16:58:59.322733 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:59.322700    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ttkn\" (UniqueName: \"kubernetes.io/projected/c48c6055-bcd9-4470-86fe-f18d07399098-kube-api-access-8ttkn\") pod \"maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w\" (UID: \"c48c6055-bcd9-4470-86fe-f18d07399098\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w"
Jul 20 16:58:59.493758 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:59.493722    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:58:59.697648 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:59.697568    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:58:59.801852 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:59.801824    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:58:59.818410 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:59.818376    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c48c6055-bcd9-4470-86fe-f18d07399098-maas-api-tls\") pod \"maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w\" (UID: \"c48c6055-bcd9-4470-86fe-f18d07399098\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w"
Jul 20 16:58:59.820718 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:58:59.820701    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c48c6055-bcd9-4470-86fe-f18d07399098-maas-api-tls\") pod \"maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w\" (UID: \"c48c6055-bcd9-4470-86fe-f18d07399098\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w"
Jul 20 16:59:00.046727 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:00.046702    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w"
Jul 20 16:59:00.183230 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:00.183206    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w"]
Jul 20 16:59:00.185381 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:59:00.185327    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc48c6055_bcd9_4470_86fe_f18d07399098.slice/crio-1f011d774c6059acf438d07098362cf70e27428a945d17e764c463bee67f223c WatchSource:0}: Error finding container 1f011d774c6059acf438d07098362cf70e27428a945d17e764c463bee67f223c: Status 404 returned error can't find the container with id 1f011d774c6059acf438d07098362cf70e27428a945d17e764c463bee67f223c
Jul 20 16:59:00.555968 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:00.555937    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w" event={"ID":"c48c6055-bcd9-4470-86fe-f18d07399098","Type":"ContainerStarted","Data":"1f011d774c6059acf438d07098362cf70e27428a945d17e764c463bee67f223c"}
Jul 20 16:59:01.561780 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:01.561743    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w" event={"ID":"c48c6055-bcd9-4470-86fe-f18d07399098","Type":"ContainerStarted","Data":"0de4087bc7fd854471f50a936269f1726dfd9213af33c730a8dac9ce7580a1b0"}
Jul 20 16:59:01.562179 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:01.561819    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w"
Jul 20 16:59:01.587580 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:01.587528    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w" podStartSLOduration=2.177339391 podStartE2EDuration="2.587512401s" totalImagesPullingTime="410.17301ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:58:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:59:01.587442164 +0000 UTC m=+1764.836936182" watchObservedRunningTime="2026-07-20 16:59:01.587512401 +0000 UTC m=+1764.837006420"
Jul 20 16:59:07.572277 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:07.572247    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w"
Jul 20 16:59:10.606436 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:10.606383    2577 generic.go:356] "Generic (PLEG): container finished" podID="481f4335-a556-49a1-8ffb-b8bac474f08b" containerID="b6556cce0469ee9288b83329e8b5fdf16e3ca7442c78a613d764b66bb32401e1" exitCode=0
Jul 20 16:59:10.606901 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:10.606455    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258" event={"ID":"481f4335-a556-49a1-8ffb-b8bac474f08b","Type":"ContainerDied","Data":"b6556cce0469ee9288b83329e8b5fdf16e3ca7442c78a613d764b66bb32401e1"}
Jul 20 16:59:10.606901 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:10.606841    2577 scope.go:117] "RemoveContainer" containerID="b6556cce0469ee9288b83329e8b5fdf16e3ca7442c78a613d764b66bb32401e1"
Jul 20 16:59:10.607856 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:10.607829    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 16:59:10.607909 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:10.607900    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 16:59:10.607951 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:10.607936    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 16:59:11.613092 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:11.613057    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258" event={"ID":"481f4335-a556-49a1-8ffb-b8bac474f08b","Type":"ContainerStarted","Data":"91806c57893993cce6d6831e1b91c9d2f128ecc01c6e691d596a5384efb92698"}
Jul 20 16:59:12.501345 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:12.501286    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:59:12.502540 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:12.502511    2577 patch_prober.go:28] interesting pod/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Jul 20 16:59:12.502660 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:12.502573    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258" podUID="481f4335-a556-49a1-8ffb-b8bac474f08b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Jul 20 16:59:13.501928 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:13.501890    2577 patch_prober.go:28] interesting pod/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Jul 20 16:59:13.502340 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:13.501944    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258" podUID="481f4335-a556-49a1-8ffb-b8bac474f08b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Jul 20 16:59:13.547218 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:13.547173    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:59:14.502113 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:14.502075    2577 patch_prober.go:28] interesting pod/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Jul 20 16:59:14.502536 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:14.502163    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258" podUID="481f4335-a556-49a1-8ffb-b8bac474f08b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Jul 20 16:59:15.502063 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:15.502028    2577 patch_prober.go:28] interesting pod/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body=
Jul 20 16:59:15.502236 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:15.502082    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258" podUID="481f4335-a556-49a1-8ffb-b8bac474f08b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused"
Jul 20 16:59:16.505334 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:16.505286    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:59:16.506285 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:16.506265    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:59:34.534262 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:34.534149    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w"]
Jul 20 16:59:34.534858 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:34.534819    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w" podUID="c48c6055-bcd9-4470-86fe-f18d07399098" containerName="maas-api" containerID="cri-o://0de4087bc7fd854471f50a936269f1726dfd9213af33c730a8dac9ce7580a1b0" gracePeriod=30
Jul 20 16:59:34.579966 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:34.579928    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:59:34.723025 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:34.722992    2577 generic.go:356] "Generic (PLEG): container finished" podID="c48c6055-bcd9-4470-86fe-f18d07399098" containerID="0de4087bc7fd854471f50a936269f1726dfd9213af33c730a8dac9ce7580a1b0" exitCode=0
Jul 20 16:59:34.723197 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:34.723039    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w" event={"ID":"c48c6055-bcd9-4470-86fe-f18d07399098","Type":"ContainerDied","Data":"0de4087bc7fd854471f50a936269f1726dfd9213af33c730a8dac9ce7580a1b0"}
Jul 20 16:59:34.781675 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:34.781645    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w"
Jul 20 16:59:34.840885 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:34.840855    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c48c6055-bcd9-4470-86fe-f18d07399098-openshift-service-ca\") pod \"c48c6055-bcd9-4470-86fe-f18d07399098\" (UID: \"c48c6055-bcd9-4470-86fe-f18d07399098\") "
Jul 20 16:59:34.841076 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:34.840892    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ttkn\" (UniqueName: \"kubernetes.io/projected/c48c6055-bcd9-4470-86fe-f18d07399098-kube-api-access-8ttkn\") pod \"c48c6055-bcd9-4470-86fe-f18d07399098\" (UID: \"c48c6055-bcd9-4470-86fe-f18d07399098\") "
Jul 20 16:59:34.841076 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:34.840928    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c48c6055-bcd9-4470-86fe-f18d07399098-maas-api-tls\") pod \"c48c6055-bcd9-4470-86fe-f18d07399098\" (UID: \"c48c6055-bcd9-4470-86fe-f18d07399098\") "
Jul 20 16:59:34.841413 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:34.841387    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c48c6055-bcd9-4470-86fe-f18d07399098-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c48c6055-bcd9-4470-86fe-f18d07399098" (UID: "c48c6055-bcd9-4470-86fe-f18d07399098"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:59:34.842918 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:34.842893    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c48c6055-bcd9-4470-86fe-f18d07399098-kube-api-access-8ttkn" (OuterVolumeSpecName: "kube-api-access-8ttkn") pod "c48c6055-bcd9-4470-86fe-f18d07399098" (UID: "c48c6055-bcd9-4470-86fe-f18d07399098"). InnerVolumeSpecName "kube-api-access-8ttkn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:59:34.843027 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:34.842976    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c48c6055-bcd9-4470-86fe-f18d07399098-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c48c6055-bcd9-4470-86fe-f18d07399098" (UID: "c48c6055-bcd9-4470-86fe-f18d07399098"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 16:59:34.941830 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:34.941790    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c48c6055-bcd9-4470-86fe-f18d07399098-openshift-service-ca\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:59:34.941830 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:34.941822    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ttkn\" (UniqueName: \"kubernetes.io/projected/c48c6055-bcd9-4470-86fe-f18d07399098-kube-api-access-8ttkn\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:59:34.941830 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:34.941834    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c48c6055-bcd9-4470-86fe-f18d07399098-maas-api-tls\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:59:35.729242 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:35.729216    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w"
Jul 20 16:59:35.729750 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:35.729211    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w" event={"ID":"c48c6055-bcd9-4470-86fe-f18d07399098","Type":"ContainerDied","Data":"1f011d774c6059acf438d07098362cf70e27428a945d17e764c463bee67f223c"}
Jul 20 16:59:35.729750 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:35.729351    2577 scope.go:117] "RemoveContainer" containerID="0de4087bc7fd854471f50a936269f1726dfd9213af33c730a8dac9ce7580a1b0"
Jul 20 16:59:35.746232 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:35.746208    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w"]
Jul 20 16:59:35.750504 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:35.750482    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-3e15ca90-797dd44987-mbd5w"]
Jul 20 16:59:37.212447 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:37.212415    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c48c6055-bcd9-4470-86fe-f18d07399098" path="/var/lib/kubelet/pods/c48c6055-bcd9-4470-86fe-f18d07399098/volumes"
Jul 20 16:59:37.214152 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:37.214119    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gd4p4_fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c/ovn-acl-logging/0.log"
Jul 20 16:59:37.217472 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:37.217451    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gd4p4_fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c/ovn-acl-logging/0.log"
Jul 20 16:59:39.110807 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:39.110742    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:59:39.241102 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:39.241069    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"]
Jul 20 16:59:39.241349 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:39.241321    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258" podUID="481f4335-a556-49a1-8ffb-b8bac474f08b" containerName="istio-proxy" containerID="cri-o://91806c57893993cce6d6831e1b91c9d2f128ecc01c6e691d596a5384efb92698" gracePeriod=30
Jul 20 16:59:39.264373 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:39.264344    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:59:39.576912 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:39.576878    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:59:39.735899 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:39.735867    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:59:40.016577 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.016500    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"]
Jul 20 16:59:40.017007 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.016992    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c48c6055-bcd9-4470-86fe-f18d07399098" containerName="maas-api"
Jul 20 16:59:40.017052 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.017010    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c48c6055-bcd9-4470-86fe-f18d07399098" containerName="maas-api"
Jul 20 16:59:40.017109 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.017098    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c48c6055-bcd9-4470-86fe-f18d07399098" containerName="maas-api"
Jul 20 16:59:40.020142 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.020123    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 16:59:40.022506 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.022485    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-4c8f9801-openshift-default-dockercfg-rbcnf\""
Jul 20 16:59:40.032686 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.032664    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"]
Jul 20 16:59:40.038139 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.038118    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:59:40.082476 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.082448    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50dcc47e-10dc-4f93-8508-ca2613add5a1-istiod-ca-cert\") pod \"e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") " pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 16:59:40.082587 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.082493    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50dcc47e-10dc-4f93-8508-ca2613add5a1-istio-data\") pod \"e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") " pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 16:59:40.082587 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.082532    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50dcc47e-10dc-4f93-8508-ca2613add5a1-workload-socket\") pod \"e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") " pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 16:59:40.082587 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.082556    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50dcc47e-10dc-4f93-8508-ca2613add5a1-istio-podinfo\") pod \"e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") " pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 16:59:40.082695 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.082586    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wdbw\" (UniqueName: \"kubernetes.io/projected/50dcc47e-10dc-4f93-8508-ca2613add5a1-kube-api-access-8wdbw\") pod \"e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") " pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 16:59:40.082695 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.082627    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50dcc47e-10dc-4f93-8508-ca2613add5a1-istio-envoy\") pod \"e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") " pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 16:59:40.082695 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.082658    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50dcc47e-10dc-4f93-8508-ca2613add5a1-workload-certs\") pod \"e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") " pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 16:59:40.082695 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.082676    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50dcc47e-10dc-4f93-8508-ca2613add5a1-istio-token\") pod \"e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") " pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 16:59:40.082820 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.082704    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50dcc47e-10dc-4f93-8508-ca2613add5a1-credential-socket\") pod \"e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") " pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 16:59:40.183851 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.183819    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50dcc47e-10dc-4f93-8508-ca2613add5a1-istio-data\") pod \"e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") " pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 16:59:40.184252 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.183865    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50dcc47e-10dc-4f93-8508-ca2613add5a1-workload-socket\") pod \"e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") " pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 16:59:40.184252 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.183892    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50dcc47e-10dc-4f93-8508-ca2613add5a1-istio-podinfo\") pod \"e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") " pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 16:59:40.184252 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.183923    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wdbw\" (UniqueName: \"kubernetes.io/projected/50dcc47e-10dc-4f93-8508-ca2613add5a1-kube-api-access-8wdbw\") pod \"e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") " pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 16:59:40.184252 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.183941    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50dcc47e-10dc-4f93-8508-ca2613add5a1-istio-envoy\") pod \"e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") " pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 16:59:40.184252 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.183973    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50dcc47e-10dc-4f93-8508-ca2613add5a1-workload-certs\") pod \"e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") " pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 16:59:40.184252 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.183996    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50dcc47e-10dc-4f93-8508-ca2613add5a1-istio-token\") pod \"e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") " pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 16:59:40.184252 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.184036    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50dcc47e-10dc-4f93-8508-ca2613add5a1-credential-socket\") pod \"e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") " pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 16:59:40.184252 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.184065    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50dcc47e-10dc-4f93-8508-ca2613add5a1-istiod-ca-cert\") pod \"e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") " pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 16:59:40.184590 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.184342    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50dcc47e-10dc-4f93-8508-ca2613add5a1-istio-data\") pod \"e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") " pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 16:59:40.184590 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.184399    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50dcc47e-10dc-4f93-8508-ca2613add5a1-workload-certs\") pod \"e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") " pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 16:59:40.184590 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.184487    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50dcc47e-10dc-4f93-8508-ca2613add5a1-credential-socket\") pod \"e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") " pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 16:59:40.184704 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.184606    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50dcc47e-10dc-4f93-8508-ca2613add5a1-workload-socket\") pod \"e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") " pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 16:59:40.184967 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.184950    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50dcc47e-10dc-4f93-8508-ca2613add5a1-istiod-ca-cert\") pod \"e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") " pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 16:59:40.186836 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.186804    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50dcc47e-10dc-4f93-8508-ca2613add5a1-istio-envoy\") pod \"e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") " pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 16:59:40.187002 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.186987    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50dcc47e-10dc-4f93-8508-ca2613add5a1-istio-podinfo\") pod \"e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") " pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 16:59:40.198836 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.198800    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wdbw\" (UniqueName: \"kubernetes.io/projected/50dcc47e-10dc-4f93-8508-ca2613add5a1-kube-api-access-8wdbw\") pod \"e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") " pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 16:59:40.198931 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.198806    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50dcc47e-10dc-4f93-8508-ca2613add5a1-istio-token\") pod \"e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") " pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 16:59:40.253668 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.253642    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:59:40.331488 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.331463    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 16:59:40.462693 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.462670    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"]
Jul 20 16:59:40.464374 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:59:40.464343    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50dcc47e_10dc_4f93_8508_ca2613add5a1.slice/crio-ea027c644de6391631a019b4234fe24e5cf939341c60cd88f248154e1f6f9541 WatchSource:0}: Error finding container ea027c644de6391631a019b4234fe24e5cf939341c60cd88f248154e1f6f9541: Status 404 returned error can't find the container with id ea027c644de6391631a019b4234fe24e5cf939341c60cd88f248154e1f6f9541
Jul 20 16:59:40.466580 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.466547    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 16:59:40.466663 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.466618    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 16:59:40.466663 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.466645    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 16:59:40.753592 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.753497    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j" event={"ID":"50dcc47e-10dc-4f93-8508-ca2613add5a1","Type":"ContainerStarted","Data":"650b24c3fe590b489e5b03ccdea51f73a204fcd81afde4e34b98301b7a27d787"}
Jul 20 16:59:40.753592 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:40.753563    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j" event={"ID":"50dcc47e-10dc-4f93-8508-ca2613add5a1","Type":"ContainerStarted","Data":"ea027c644de6391631a019b4234fe24e5cf939341c60cd88f248154e1f6f9541"}
Jul 20 16:59:41.332464 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:41.332431    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 16:59:41.337248 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:41.337227    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 16:59:41.360415 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:41.360368    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j" podStartSLOduration=2.36035302 podStartE2EDuration="2.36035302s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:59:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:59:40.775701818 +0000 UTC m=+1804.025195861" watchObservedRunningTime="2026-07-20 16:59:41.36035302 +0000 UTC m=+1804.609847039"
Jul 20 16:59:41.758738 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:41.758652    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 16:59:41.759745 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:41.759727    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 16:59:42.166432 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:42.166394    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:59:42.273615 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:42.273583    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:59:42.273811 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:42.273792    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6"]
Jul 20 16:59:42.277281 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:42.277260    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6"
Jul 20 16:59:42.280047 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:42.280028    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-4c8f9801-serving-cert\""
Jul 20 16:59:42.280164 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:42.280038    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-kxz6w\""
Jul 20 16:59:42.290870 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:42.290805    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6"]
Jul 20 16:59:42.302644 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:42.302619    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5jzb\" (UniqueName: \"kubernetes.io/projected/aace1e26-7a9d-473d-826e-7d5869a91546-kube-api-access-r5jzb\") pod \"maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6\" (UID: \"aace1e26-7a9d-473d-826e-7d5869a91546\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6"
Jul 20 16:59:42.302771 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:42.302661    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aace1e26-7a9d-473d-826e-7d5869a91546-openshift-service-ca\") pod \"maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6\" (UID: \"aace1e26-7a9d-473d-826e-7d5869a91546\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6"
Jul 20 16:59:42.302771 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:42.302706    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aace1e26-7a9d-473d-826e-7d5869a91546-maas-api-tls\") pod \"maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6\" (UID: \"aace1e26-7a9d-473d-826e-7d5869a91546\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6"
Jul 20 16:59:42.358851 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:42.358803    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:59:42.403460 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:42.403427    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5jzb\" (UniqueName: \"kubernetes.io/projected/aace1e26-7a9d-473d-826e-7d5869a91546-kube-api-access-r5jzb\") pod \"maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6\" (UID: \"aace1e26-7a9d-473d-826e-7d5869a91546\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6"
Jul 20 16:59:42.403620 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:42.403473    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aace1e26-7a9d-473d-826e-7d5869a91546-openshift-service-ca\") pod \"maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6\" (UID: \"aace1e26-7a9d-473d-826e-7d5869a91546\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6"
Jul 20 16:59:42.403620 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:42.403505    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aace1e26-7a9d-473d-826e-7d5869a91546-maas-api-tls\") pod \"maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6\" (UID: \"aace1e26-7a9d-473d-826e-7d5869a91546\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6"
Jul 20 16:59:42.403620 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:59:42.403603    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-4c8f9801-serving-cert: secret "maas-api-e2e-ait-4c8f9801-serving-cert" not found
Jul 20 16:59:42.403725 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:59:42.403661    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aace1e26-7a9d-473d-826e-7d5869a91546-maas-api-tls podName:aace1e26-7a9d-473d-826e-7d5869a91546 nodeName:}" failed. No retries permitted until 2026-07-20 16:59:42.903643809 +0000 UTC m=+1806.153137806 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/aace1e26-7a9d-473d-826e-7d5869a91546-maas-api-tls") pod "maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6" (UID: "aace1e26-7a9d-473d-826e-7d5869a91546") : secret "maas-api-e2e-ait-4c8f9801-serving-cert" not found
Jul 20 16:59:42.404293 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:42.404272    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aace1e26-7a9d-473d-826e-7d5869a91546-openshift-service-ca\") pod \"maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6\" (UID: \"aace1e26-7a9d-473d-826e-7d5869a91546\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6"
Jul 20 16:59:42.414025 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:42.413992    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5jzb\" (UniqueName: \"kubernetes.io/projected/aace1e26-7a9d-473d-826e-7d5869a91546-kube-api-access-r5jzb\") pod \"maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6\" (UID: \"aace1e26-7a9d-473d-826e-7d5869a91546\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6"
Jul 20 16:59:42.517613 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:42.517523    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:59:42.894627 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:42.894591    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:59:42.907876 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:42.907840    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aace1e26-7a9d-473d-826e-7d5869a91546-maas-api-tls\") pod \"maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6\" (UID: \"aace1e26-7a9d-473d-826e-7d5869a91546\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6"
Jul 20 16:59:42.910167 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:42.910142    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aace1e26-7a9d-473d-826e-7d5869a91546-maas-api-tls\") pod \"maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6\" (UID: \"aace1e26-7a9d-473d-826e-7d5869a91546\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6"
Jul 20 16:59:42.987454 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:42.987425    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 16:59:43.190931 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:43.190839    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6"
Jul 20 16:59:43.328420 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:43.328395    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6"]
Jul 20 16:59:43.330375 ip-10-0-130-85 kubenswrapper[2577]: W0720 16:59:43.330351    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaace1e26_7a9d_473d_826e_7d5869a91546.slice/crio-8ee08a73fa007fa2f5921a7ccf487f10e6c1edd88973842b0bd62030ba87c54b WatchSource:0}: Error finding container 8ee08a73fa007fa2f5921a7ccf487f10e6c1edd88973842b0bd62030ba87c54b: Status 404 returned error can't find the container with id 8ee08a73fa007fa2f5921a7ccf487f10e6c1edd88973842b0bd62030ba87c54b
Jul 20 16:59:43.767786 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:43.767757    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6" event={"ID":"aace1e26-7a9d-473d-826e-7d5869a91546","Type":"ContainerStarted","Data":"0949612d023b1ef2e675b09069d8d9d1166ba59d449689f9ebb9e853da6ad15a"}
Jul 20 16:59:43.768151 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:43.767808    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6" event={"ID":"aace1e26-7a9d-473d-826e-7d5869a91546","Type":"ContainerStarted","Data":"8ee08a73fa007fa2f5921a7ccf487f10e6c1edd88973842b0bd62030ba87c54b"}
Jul 20 16:59:43.786607 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:43.786557    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6" podStartSLOduration=1.420283489 podStartE2EDuration="1.786542532s" totalImagesPullingTime="366.259043ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 16:59:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 16:59:43.784703963 +0000 UTC m=+1807.034197975" watchObservedRunningTime="2026-07-20 16:59:43.786542532 +0000 UTC m=+1807.036036550"
Jul 20 16:59:44.494331 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.494285    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:59:44.523161 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.523081    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/481f4335-a556-49a1-8ffb-b8bac474f08b-workload-certs\") pod \"481f4335-a556-49a1-8ffb-b8bac474f08b\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") "
Jul 20 16:59:44.523161 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.523158    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/481f4335-a556-49a1-8ffb-b8bac474f08b-istio-data\") pod \"481f4335-a556-49a1-8ffb-b8bac474f08b\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") "
Jul 20 16:59:44.523356 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.523201    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/481f4335-a556-49a1-8ffb-b8bac474f08b-istio-podinfo\") pod \"481f4335-a556-49a1-8ffb-b8bac474f08b\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") "
Jul 20 16:59:44.523356 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.523237    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/481f4335-a556-49a1-8ffb-b8bac474f08b-workload-socket\") pod \"481f4335-a556-49a1-8ffb-b8bac474f08b\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") "
Jul 20 16:59:44.523356 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.523316    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/481f4335-a556-49a1-8ffb-b8bac474f08b-istiod-ca-cert\") pod \"481f4335-a556-49a1-8ffb-b8bac474f08b\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") "
Jul 20 16:59:44.523356 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.523350    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/481f4335-a556-49a1-8ffb-b8bac474f08b-istio-envoy\") pod \"481f4335-a556-49a1-8ffb-b8bac474f08b\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") "
Jul 20 16:59:44.523558 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.523403    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/481f4335-a556-49a1-8ffb-b8bac474f08b-credential-socket\") pod \"481f4335-a556-49a1-8ffb-b8bac474f08b\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") "
Jul 20 16:59:44.523558 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.523429    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/481f4335-a556-49a1-8ffb-b8bac474f08b-istio-token\") pod \"481f4335-a556-49a1-8ffb-b8bac474f08b\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") "
Jul 20 16:59:44.523558 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.523452    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vr6f7\" (UniqueName: \"kubernetes.io/projected/481f4335-a556-49a1-8ffb-b8bac474f08b-kube-api-access-vr6f7\") pod \"481f4335-a556-49a1-8ffb-b8bac474f08b\" (UID: \"481f4335-a556-49a1-8ffb-b8bac474f08b\") "
Jul 20 16:59:44.523824 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.523792    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/481f4335-a556-49a1-8ffb-b8bac474f08b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "481f4335-a556-49a1-8ffb-b8bac474f08b" (UID: "481f4335-a556-49a1-8ffb-b8bac474f08b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 16:59:44.523938 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.523890    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/481f4335-a556-49a1-8ffb-b8bac474f08b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "481f4335-a556-49a1-8ffb-b8bac474f08b" (UID: "481f4335-a556-49a1-8ffb-b8bac474f08b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:59:44.524002 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.523958    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/481f4335-a556-49a1-8ffb-b8bac474f08b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "481f4335-a556-49a1-8ffb-b8bac474f08b" (UID: "481f4335-a556-49a1-8ffb-b8bac474f08b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:59:44.524132 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.524107    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/481f4335-a556-49a1-8ffb-b8bac474f08b-istio-data" (OuterVolumeSpecName: "istio-data") pod "481f4335-a556-49a1-8ffb-b8bac474f08b" (UID: "481f4335-a556-49a1-8ffb-b8bac474f08b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:59:44.524220 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.524207    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/481f4335-a556-49a1-8ffb-b8bac474f08b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "481f4335-a556-49a1-8ffb-b8bac474f08b" (UID: "481f4335-a556-49a1-8ffb-b8bac474f08b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:59:44.525963 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.525930    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/481f4335-a556-49a1-8ffb-b8bac474f08b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "481f4335-a556-49a1-8ffb-b8bac474f08b" (UID: "481f4335-a556-49a1-8ffb-b8bac474f08b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 20 16:59:44.526042 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.525958    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/481f4335-a556-49a1-8ffb-b8bac474f08b-istio-token" (OuterVolumeSpecName: "istio-token") pod "481f4335-a556-49a1-8ffb-b8bac474f08b" (UID: "481f4335-a556-49a1-8ffb-b8bac474f08b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:59:44.526042 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.525963    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/481f4335-a556-49a1-8ffb-b8bac474f08b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "481f4335-a556-49a1-8ffb-b8bac474f08b" (UID: "481f4335-a556-49a1-8ffb-b8bac474f08b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 16:59:44.526167 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.526095    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/481f4335-a556-49a1-8ffb-b8bac474f08b-kube-api-access-vr6f7" (OuterVolumeSpecName: "kube-api-access-vr6f7") pod "481f4335-a556-49a1-8ffb-b8bac474f08b" (UID: "481f4335-a556-49a1-8ffb-b8bac474f08b"). InnerVolumeSpecName "kube-api-access-vr6f7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 16:59:44.624478 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.624453    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/481f4335-a556-49a1-8ffb-b8bac474f08b-istio-data\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:59:44.624478 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.624476    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/481f4335-a556-49a1-8ffb-b8bac474f08b-istio-podinfo\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:59:44.624622 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.624485    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/481f4335-a556-49a1-8ffb-b8bac474f08b-workload-socket\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:59:44.624622 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.624492    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/481f4335-a556-49a1-8ffb-b8bac474f08b-istiod-ca-cert\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:59:44.624622 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.624503    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/481f4335-a556-49a1-8ffb-b8bac474f08b-istio-envoy\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:59:44.624622 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.624511    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/481f4335-a556-49a1-8ffb-b8bac474f08b-credential-socket\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:59:44.624622 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.624518    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/481f4335-a556-49a1-8ffb-b8bac474f08b-istio-token\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:59:44.624622 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.624527    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vr6f7\" (UniqueName: \"kubernetes.io/projected/481f4335-a556-49a1-8ffb-b8bac474f08b-kube-api-access-vr6f7\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:59:44.624622 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.624535    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/481f4335-a556-49a1-8ffb-b8bac474f08b-workload-certs\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 16:59:44.774025 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.773936    2577 generic.go:356] "Generic (PLEG): container finished" podID="481f4335-a556-49a1-8ffb-b8bac474f08b" containerID="91806c57893993cce6d6831e1b91c9d2f128ecc01c6e691d596a5384efb92698" exitCode=0
Jul 20 16:59:44.774025 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.773985    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"
Jul 20 16:59:44.774558 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.774022    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258" event={"ID":"481f4335-a556-49a1-8ffb-b8bac474f08b","Type":"ContainerDied","Data":"91806c57893993cce6d6831e1b91c9d2f128ecc01c6e691d596a5384efb92698"}
Jul 20 16:59:44.774558 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.774069    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258" event={"ID":"481f4335-a556-49a1-8ffb-b8bac474f08b","Type":"ContainerDied","Data":"1fe95230d2b43dee4d9d78eea76c65d3e671600975aa66b377c534b5c08acbae"}
Jul 20 16:59:44.774558 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.774094    2577 scope.go:117] "RemoveContainer" containerID="91806c57893993cce6d6831e1b91c9d2f128ecc01c6e691d596a5384efb92698"
Jul 20 16:59:44.774558 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.774326    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6"
Jul 20 16:59:44.783395 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.783377    2577 scope.go:117] "RemoveContainer" containerID="b6556cce0469ee9288b83329e8b5fdf16e3ca7442c78a613d764b66bb32401e1"
Jul 20 16:59:44.790908 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.790891    2577 scope.go:117] "RemoveContainer" containerID="91806c57893993cce6d6831e1b91c9d2f128ecc01c6e691d596a5384efb92698"
Jul 20 16:59:44.791158 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:59:44.791138    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91806c57893993cce6d6831e1b91c9d2f128ecc01c6e691d596a5384efb92698\": container with ID starting with 91806c57893993cce6d6831e1b91c9d2f128ecc01c6e691d596a5384efb92698 not found: ID does not exist" containerID="91806c57893993cce6d6831e1b91c9d2f128ecc01c6e691d596a5384efb92698"
Jul 20 16:59:44.791229 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.791170    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91806c57893993cce6d6831e1b91c9d2f128ecc01c6e691d596a5384efb92698"} err="failed to get container status \"91806c57893993cce6d6831e1b91c9d2f128ecc01c6e691d596a5384efb92698\": rpc error: code = NotFound desc = could not find container \"91806c57893993cce6d6831e1b91c9d2f128ecc01c6e691d596a5384efb92698\": container with ID starting with 91806c57893993cce6d6831e1b91c9d2f128ecc01c6e691d596a5384efb92698 not found: ID does not exist"
Jul 20 16:59:44.791229 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.791195    2577 scope.go:117] "RemoveContainer" containerID="b6556cce0469ee9288b83329e8b5fdf16e3ca7442c78a613d764b66bb32401e1"
Jul 20 16:59:44.791450 ip-10-0-130-85 kubenswrapper[2577]: E0720 16:59:44.791433    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6556cce0469ee9288b83329e8b5fdf16e3ca7442c78a613d764b66bb32401e1\": container with ID starting with b6556cce0469ee9288b83329e8b5fdf16e3ca7442c78a613d764b66bb32401e1 not found: ID does not exist" containerID="b6556cce0469ee9288b83329e8b5fdf16e3ca7442c78a613d764b66bb32401e1"
Jul 20 16:59:44.791496 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.791456    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6556cce0469ee9288b83329e8b5fdf16e3ca7442c78a613d764b66bb32401e1"} err="failed to get container status \"b6556cce0469ee9288b83329e8b5fdf16e3ca7442c78a613d764b66bb32401e1\": rpc error: code = NotFound desc = could not find container \"b6556cce0469ee9288b83329e8b5fdf16e3ca7442c78a613d764b66bb32401e1\": container with ID starting with b6556cce0469ee9288b83329e8b5fdf16e3ca7442c78a613d764b66bb32401e1 not found: ID does not exist"
Jul 20 16:59:44.796906 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.796884    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"]
Jul 20 16:59:44.804172 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:44.804152    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-3e15ca90-gw-openshift-default-64c4d5559f-8k258"]
Jul 20 16:59:45.211420 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:45.211390    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="481f4335-a556-49a1-8ffb-b8bac474f08b" path="/var/lib/kubelet/pods/481f4335-a556-49a1-8ffb-b8bac474f08b/volumes"
Jul 20 16:59:50.785589 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:50.785557    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6"
Jul 20 16:59:56.760580 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:56.760545    2577 patch_prober.go:28] interesting pod/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Jul 20 16:59:56.760994 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:56.760603    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j" podUID="50dcc47e-10dc-4f93-8508-ca2613add5a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Jul 20 16:59:56.829469 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:56.829439    2577 generic.go:356] "Generic (PLEG): container finished" podID="50dcc47e-10dc-4f93-8508-ca2613add5a1" containerID="650b24c3fe590b489e5b03ccdea51f73a204fcd81afde4e34b98301b7a27d787" exitCode=0
Jul 20 16:59:56.829648 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:56.829516    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j" event={"ID":"50dcc47e-10dc-4f93-8508-ca2613add5a1","Type":"ContainerDied","Data":"650b24c3fe590b489e5b03ccdea51f73a204fcd81afde4e34b98301b7a27d787"}
Jul 20 16:59:56.829945 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:56.829931    2577 scope.go:117] "RemoveContainer" containerID="650b24c3fe590b489e5b03ccdea51f73a204fcd81afde4e34b98301b7a27d787"
Jul 20 16:59:56.830931 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:56.830905    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 16:59:56.830993 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:56.830968    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 16:59:56.831032 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:56.830997    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 16:59:57.835747 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:57.835707    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j" event={"ID":"50dcc47e-10dc-4f93-8508-ca2613add5a1","Type":"ContainerStarted","Data":"ca4f89a6c72fab7198df70d07c98091545cf85d3424abfa736ce75d7fd315528"}
Jul 20 16:59:58.332155 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:58.332120    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 16:59:58.332379 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:58.332360    2577 patch_prober.go:28] interesting pod/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Jul 20 16:59:58.332433 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:58.332402    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j" podUID="50dcc47e-10dc-4f93-8508-ca2613add5a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Jul 20 16:59:59.332395 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:59.332359    2577 patch_prober.go:28] interesting pod/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Jul 20 16:59:59.332859 ip-10-0-130-85 kubenswrapper[2577]: I0720 16:59:59.332419    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j" podUID="50dcc47e-10dc-4f93-8508-ca2613add5a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Jul 20 17:00:00.332184 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:00.332145    2577 patch_prober.go:28] interesting pod/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Jul 20 17:00:00.332396 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:00.332201    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j" podUID="50dcc47e-10dc-4f93-8508-ca2613add5a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Jul 20 17:00:01.332283 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:01.332240    2577 patch_prober.go:28] interesting pod/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Jul 20 17:00:01.332732 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:01.332324    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j" podUID="50dcc47e-10dc-4f93-8508-ca2613add5a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Jul 20 17:00:02.336092 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:02.336060    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 17:00:02.336508 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:02.336328    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 17:00:02.336972 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:02.336958    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 17:00:18.367997 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:18.367961    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6"]
Jul 20 17:00:18.368486 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:18.368163    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6" podUID="aace1e26-7a9d-473d-826e-7d5869a91546" containerName="maas-api" containerID="cri-o://0949612d023b1ef2e675b09069d8d9d1166ba59d449689f9ebb9e853da6ad15a" gracePeriod=30
Jul 20 17:00:18.391053 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:18.391019    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:00:18.607640 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:18.607617    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6"
Jul 20 17:00:18.632217 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:18.632142    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aace1e26-7a9d-473d-826e-7d5869a91546-openshift-service-ca\") pod \"aace1e26-7a9d-473d-826e-7d5869a91546\" (UID: \"aace1e26-7a9d-473d-826e-7d5869a91546\") "
Jul 20 17:00:18.632217 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:18.632183    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aace1e26-7a9d-473d-826e-7d5869a91546-maas-api-tls\") pod \"aace1e26-7a9d-473d-826e-7d5869a91546\" (UID: \"aace1e26-7a9d-473d-826e-7d5869a91546\") "
Jul 20 17:00:18.632499 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:18.632476    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aace1e26-7a9d-473d-826e-7d5869a91546-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aace1e26-7a9d-473d-826e-7d5869a91546" (UID: "aace1e26-7a9d-473d-826e-7d5869a91546"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 17:00:18.634284 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:18.634259    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aace1e26-7a9d-473d-826e-7d5869a91546-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aace1e26-7a9d-473d-826e-7d5869a91546" (UID: "aace1e26-7a9d-473d-826e-7d5869a91546"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:00:18.733339 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:18.733283    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5jzb\" (UniqueName: \"kubernetes.io/projected/aace1e26-7a9d-473d-826e-7d5869a91546-kube-api-access-r5jzb\") pod \"aace1e26-7a9d-473d-826e-7d5869a91546\" (UID: \"aace1e26-7a9d-473d-826e-7d5869a91546\") "
Jul 20 17:00:18.733521 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:18.733507    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aace1e26-7a9d-473d-826e-7d5869a91546-openshift-service-ca\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:00:18.733567 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:18.733524    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aace1e26-7a9d-473d-826e-7d5869a91546-maas-api-tls\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:00:18.735270 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:18.735248    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aace1e26-7a9d-473d-826e-7d5869a91546-kube-api-access-r5jzb" (OuterVolumeSpecName: "kube-api-access-r5jzb") pod "aace1e26-7a9d-473d-826e-7d5869a91546" (UID: "aace1e26-7a9d-473d-826e-7d5869a91546"). InnerVolumeSpecName "kube-api-access-r5jzb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:00:18.834803 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:18.834767    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5jzb\" (UniqueName: \"kubernetes.io/projected/aace1e26-7a9d-473d-826e-7d5869a91546-kube-api-access-r5jzb\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:00:18.934733 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:18.934657    2577 generic.go:356] "Generic (PLEG): container finished" podID="aace1e26-7a9d-473d-826e-7d5869a91546" containerID="0949612d023b1ef2e675b09069d8d9d1166ba59d449689f9ebb9e853da6ad15a" exitCode=0
Jul 20 17:00:18.934733 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:18.934708    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6"
Jul 20 17:00:18.934920 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:18.934711    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6" event={"ID":"aace1e26-7a9d-473d-826e-7d5869a91546","Type":"ContainerDied","Data":"0949612d023b1ef2e675b09069d8d9d1166ba59d449689f9ebb9e853da6ad15a"}
Jul 20 17:00:18.934920 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:18.934838    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6" event={"ID":"aace1e26-7a9d-473d-826e-7d5869a91546","Type":"ContainerDied","Data":"8ee08a73fa007fa2f5921a7ccf487f10e6c1edd88973842b0bd62030ba87c54b"}
Jul 20 17:00:18.934920 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:18.934869    2577 scope.go:117] "RemoveContainer" containerID="0949612d023b1ef2e675b09069d8d9d1166ba59d449689f9ebb9e853da6ad15a"
Jul 20 17:00:18.943019 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:18.942994    2577 scope.go:117] "RemoveContainer" containerID="0949612d023b1ef2e675b09069d8d9d1166ba59d449689f9ebb9e853da6ad15a"
Jul 20 17:00:18.943275 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:00:18.943255    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0949612d023b1ef2e675b09069d8d9d1166ba59d449689f9ebb9e853da6ad15a\": container with ID starting with 0949612d023b1ef2e675b09069d8d9d1166ba59d449689f9ebb9e853da6ad15a not found: ID does not exist" containerID="0949612d023b1ef2e675b09069d8d9d1166ba59d449689f9ebb9e853da6ad15a"
Jul 20 17:00:18.943376 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:18.943286    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0949612d023b1ef2e675b09069d8d9d1166ba59d449689f9ebb9e853da6ad15a"} err="failed to get container status \"0949612d023b1ef2e675b09069d8d9d1166ba59d449689f9ebb9e853da6ad15a\": rpc error: code = NotFound desc = could not find container \"0949612d023b1ef2e675b09069d8d9d1166ba59d449689f9ebb9e853da6ad15a\": container with ID starting with 0949612d023b1ef2e675b09069d8d9d1166ba59d449689f9ebb9e853da6ad15a not found: ID does not exist"
Jul 20 17:00:18.956239 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:18.956219    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6"]
Jul 20 17:00:18.962308 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:18.962275    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4c8f9801-64dd65db8-qn7p6"]
Jul 20 17:00:19.215376 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:19.213176    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aace1e26-7a9d-473d-826e-7d5869a91546" path="/var/lib/kubelet/pods/aace1e26-7a9d-473d-826e-7d5869a91546/volumes"
Jul 20 17:00:21.177235 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:21.177199    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:00:21.280741 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:21.280693    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:00:22.989221 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:22.989163    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:00:24.203692 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:24.203662    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"]
Jul 20 17:00:24.204081 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:24.203853    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j" podUID="50dcc47e-10dc-4f93-8508-ca2613add5a1" containerName="istio-proxy" containerID="cri-o://ca4f89a6c72fab7198df70d07c98091545cf85d3424abfa736ce75d7fd315528" gracePeriod=30
Jul 20 17:00:24.224937 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:24.224907    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:00:24.465561 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:24.465445    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:00:24.679189 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:24.679137    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:00:24.956405 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:24.956374    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"]
Jul 20 17:00:24.956734 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:24.956719    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="481f4335-a556-49a1-8ffb-b8bac474f08b" containerName="istio-proxy"
Jul 20 17:00:24.956781 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:24.956737    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="481f4335-a556-49a1-8ffb-b8bac474f08b" containerName="istio-proxy"
Jul 20 17:00:24.956781 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:24.956753    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aace1e26-7a9d-473d-826e-7d5869a91546" containerName="maas-api"
Jul 20 17:00:24.956781 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:24.956758    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="aace1e26-7a9d-473d-826e-7d5869a91546" containerName="maas-api"
Jul 20 17:00:24.956875 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:24.956810    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="481f4335-a556-49a1-8ffb-b8bac474f08b" containerName="istio-proxy"
Jul 20 17:00:24.956875 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:24.956817    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="aace1e26-7a9d-473d-826e-7d5869a91546" containerName="maas-api"
Jul 20 17:00:24.956875 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:24.956825    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="481f4335-a556-49a1-8ffb-b8bac474f08b" containerName="istio-proxy"
Jul 20 17:00:24.961084 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:24.961060    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:24.963636 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:24.963611    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-c86df9a5-openshift-default-dockercfg-5bdk9\""
Jul 20 17:00:24.972463 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:24.972440    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"]
Jul 20 17:00:24.988649 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:24.988624    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab56ac26-9a40-462f-8f54-5332a3aae155-istio-token\") pod \"e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") " pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:24.988773 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:24.988664    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab56ac26-9a40-462f-8f54-5332a3aae155-credential-socket\") pod \"e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") " pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:24.988773 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:24.988694    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab56ac26-9a40-462f-8f54-5332a3aae155-workload-certs\") pod \"e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") " pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:24.988854 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:24.988795    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab56ac26-9a40-462f-8f54-5332a3aae155-workload-socket\") pod \"e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") " pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:24.988854 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:24.988841    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trtbv\" (UniqueName: \"kubernetes.io/projected/ab56ac26-9a40-462f-8f54-5332a3aae155-kube-api-access-trtbv\") pod \"e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") " pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:24.988918 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:24.988893    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab56ac26-9a40-462f-8f54-5332a3aae155-istio-data\") pod \"e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") " pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:24.988990 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:24.988973    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab56ac26-9a40-462f-8f54-5332a3aae155-istio-podinfo\") pod \"e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") " pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:24.989027 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:24.989012    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab56ac26-9a40-462f-8f54-5332a3aae155-istiod-ca-cert\") pod \"e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") " pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:24.989068 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:24.989057    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab56ac26-9a40-462f-8f54-5332a3aae155-istio-envoy\") pod \"e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") " pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:24.995275 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:24.995248    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:00:25.089585 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:25.089545    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab56ac26-9a40-462f-8f54-5332a3aae155-istio-envoy\") pod \"e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") " pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:25.089779 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:25.089607    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab56ac26-9a40-462f-8f54-5332a3aae155-istio-token\") pod \"e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") " pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:25.089779 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:25.089636    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab56ac26-9a40-462f-8f54-5332a3aae155-credential-socket\") pod \"e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") " pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:25.089779 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:25.089666    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab56ac26-9a40-462f-8f54-5332a3aae155-workload-certs\") pod \"e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") " pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:25.089779 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:25.089710    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab56ac26-9a40-462f-8f54-5332a3aae155-workload-socket\") pod \"e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") " pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:25.089779 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:25.089740    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trtbv\" (UniqueName: \"kubernetes.io/projected/ab56ac26-9a40-462f-8f54-5332a3aae155-kube-api-access-trtbv\") pod \"e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") " pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:25.089779 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:25.089771    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab56ac26-9a40-462f-8f54-5332a3aae155-istio-data\") pod \"e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") " pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:25.090117 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:25.089826    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab56ac26-9a40-462f-8f54-5332a3aae155-istio-podinfo\") pod \"e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") " pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:25.090117 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:25.089856    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab56ac26-9a40-462f-8f54-5332a3aae155-istiod-ca-cert\") pod \"e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") " pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:25.090117 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:25.090075    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab56ac26-9a40-462f-8f54-5332a3aae155-workload-certs\") pod \"e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") " pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:25.090326 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:25.090133    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab56ac26-9a40-462f-8f54-5332a3aae155-credential-socket\") pod \"e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") " pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:25.090326 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:25.090318    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab56ac26-9a40-462f-8f54-5332a3aae155-workload-socket\") pod \"e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") " pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:25.090497 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:25.090480    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab56ac26-9a40-462f-8f54-5332a3aae155-istio-data\") pod \"e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") " pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:25.090789 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:25.090770    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab56ac26-9a40-462f-8f54-5332a3aae155-istiod-ca-cert\") pod \"e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") " pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:25.091940 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:25.091920    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab56ac26-9a40-462f-8f54-5332a3aae155-istio-envoy\") pod \"e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") " pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:25.092408 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:25.092391    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab56ac26-9a40-462f-8f54-5332a3aae155-istio-podinfo\") pod \"e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") " pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:25.098462 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:25.098437    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab56ac26-9a40-462f-8f54-5332a3aae155-istio-token\") pod \"e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") " pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:25.098838 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:25.098809    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trtbv\" (UniqueName: \"kubernetes.io/projected/ab56ac26-9a40-462f-8f54-5332a3aae155-kube-api-access-trtbv\") pod \"e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") " pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:25.271599 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:25.271504    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:25.419562 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:25.419529    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"]
Jul 20 17:00:25.422256 ip-10-0-130-85 kubenswrapper[2577]: W0720 17:00:25.422220    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab56ac26_9a40_462f_8f54_5332a3aae155.slice/crio-f8baefccbe3d0529980e72ff9d2f07c12bb53dafcd9ce43f41acdec8326de7a9 WatchSource:0}: Error finding container f8baefccbe3d0529980e72ff9d2f07c12bb53dafcd9ce43f41acdec8326de7a9: Status 404 returned error can't find the container with id f8baefccbe3d0529980e72ff9d2f07c12bb53dafcd9ce43f41acdec8326de7a9
Jul 20 17:00:25.424439 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:25.424404    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:00:25.424523 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:25.424466    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:00:25.424523 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:25.424503    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:00:25.525066 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:25.524990    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:00:25.969206 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:25.969170    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs" event={"ID":"ab56ac26-9a40-462f-8f54-5332a3aae155","Type":"ContainerStarted","Data":"261faaafca6d8fdd54b4cccc5fa339dde1e367612e4751b37e299b27cfa92813"}
Jul 20 17:00:25.969441 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:25.969262    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs" event={"ID":"ab56ac26-9a40-462f-8f54-5332a3aae155","Type":"ContainerStarted","Data":"f8baefccbe3d0529980e72ff9d2f07c12bb53dafcd9ce43f41acdec8326de7a9"}
Jul 20 17:00:25.999264 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:25.999205    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs" podStartSLOduration=1.9991856270000001 podStartE2EDuration="1.999185627s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:00:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:00:25.998221409 +0000 UTC m=+1849.247715440" watchObservedRunningTime="2026-07-20 17:00:25.999185627 +0000 UTC m=+1849.248679647"
Jul 20 17:00:26.272253 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:26.272158    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:26.273663 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:26.273639    2577 patch_prober.go:28] interesting pod/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Jul 20 17:00:26.273751 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:26.273690    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs" podUID="ab56ac26-9a40-462f-8f54-5332a3aae155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Jul 20 17:00:27.275887 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:27.275854    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:27.340093 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:27.340061    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:00:27.438017 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:27.437983    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:00:27.440608 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:27.440572    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr"]
Jul 20 17:00:27.441124 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:27.441103    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="481f4335-a556-49a1-8ffb-b8bac474f08b" containerName="istio-proxy"
Jul 20 17:00:27.441209 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:27.441129    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="481f4335-a556-49a1-8ffb-b8bac474f08b" containerName="istio-proxy"
Jul 20 17:00:27.444418 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:27.444398    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr"
Jul 20 17:00:27.450486 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:27.450465    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-kxz6w\""
Jul 20 17:00:27.450486 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:27.450465    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-c86df9a5-serving-cert\""
Jul 20 17:00:27.460692 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:27.460668    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr"]
Jul 20 17:00:27.511464 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:27.511431    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppvrm\" (UniqueName: \"kubernetes.io/projected/cd2ebfce-8bdb-44c0-9993-2e2a43badaf0-kube-api-access-ppvrm\") pod \"maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr\" (UID: \"cd2ebfce-8bdb-44c0-9993-2e2a43badaf0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr"
Jul 20 17:00:27.511627 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:27.511483    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd2ebfce-8bdb-44c0-9993-2e2a43badaf0-openshift-service-ca\") pod \"maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr\" (UID: \"cd2ebfce-8bdb-44c0-9993-2e2a43badaf0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr"
Jul 20 17:00:27.511627 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:27.511602    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd2ebfce-8bdb-44c0-9993-2e2a43badaf0-maas-api-tls\") pod \"maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr\" (UID: \"cd2ebfce-8bdb-44c0-9993-2e2a43badaf0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr"
Jul 20 17:00:27.561046 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:27.561016    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:00:27.612038 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:27.612002    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd2ebfce-8bdb-44c0-9993-2e2a43badaf0-maas-api-tls\") pod \"maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr\" (UID: \"cd2ebfce-8bdb-44c0-9993-2e2a43badaf0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr"
Jul 20 17:00:27.612242 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:27.612079    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppvrm\" (UniqueName: \"kubernetes.io/projected/cd2ebfce-8bdb-44c0-9993-2e2a43badaf0-kube-api-access-ppvrm\") pod \"maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr\" (UID: \"cd2ebfce-8bdb-44c0-9993-2e2a43badaf0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr"
Jul 20 17:00:27.612242 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:27.612133    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd2ebfce-8bdb-44c0-9993-2e2a43badaf0-openshift-service-ca\") pod \"maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr\" (UID: \"cd2ebfce-8bdb-44c0-9993-2e2a43badaf0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr"
Jul 20 17:00:27.612769 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:27.612738    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd2ebfce-8bdb-44c0-9993-2e2a43badaf0-openshift-service-ca\") pod \"maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr\" (UID: \"cd2ebfce-8bdb-44c0-9993-2e2a43badaf0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr"
Jul 20 17:00:27.614545 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:27.614522    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd2ebfce-8bdb-44c0-9993-2e2a43badaf0-maas-api-tls\") pod \"maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr\" (UID: \"cd2ebfce-8bdb-44c0-9993-2e2a43badaf0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr"
Jul 20 17:00:27.625379 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:27.625357    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppvrm\" (UniqueName: \"kubernetes.io/projected/cd2ebfce-8bdb-44c0-9993-2e2a43badaf0-kube-api-access-ppvrm\") pod \"maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr\" (UID: \"cd2ebfce-8bdb-44c0-9993-2e2a43badaf0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr"
Jul 20 17:00:27.661146 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:27.661119    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:00:27.755160 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:27.755132    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr"
Jul 20 17:00:27.890470 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:27.890445    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr"]
Jul 20 17:00:27.892460 ip-10-0-130-85 kubenswrapper[2577]: W0720 17:00:27.892429    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd2ebfce_8bdb_44c0_9993_2e2a43badaf0.slice/crio-ae5d732fc68eedab7e90562d4f680217bd35ec26f905ab458541a3d2e1fd37c4 WatchSource:0}: Error finding container ae5d732fc68eedab7e90562d4f680217bd35ec26f905ab458541a3d2e1fd37c4: Status 404 returned error can't find the container with id ae5d732fc68eedab7e90562d4f680217bd35ec26f905ab458541a3d2e1fd37c4
Jul 20 17:00:27.980048 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:27.980011    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr" event={"ID":"cd2ebfce-8bdb-44c0-9993-2e2a43badaf0","Type":"ContainerStarted","Data":"ae5d732fc68eedab7e90562d4f680217bd35ec26f905ab458541a3d2e1fd37c4"}
Jul 20 17:00:27.980330 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:27.980310    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:27.981311 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:27.981279    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:28.082474 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:28.082391    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:00:28.181174 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:28.181079    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:00:29.460105 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.460082    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 17:00:29.530236 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.530209    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50dcc47e-10dc-4f93-8508-ca2613add5a1-istio-podinfo\") pod \"50dcc47e-10dc-4f93-8508-ca2613add5a1\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") "
Jul 20 17:00:29.530409 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.530252    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50dcc47e-10dc-4f93-8508-ca2613add5a1-workload-certs\") pod \"50dcc47e-10dc-4f93-8508-ca2613add5a1\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") "
Jul 20 17:00:29.530409 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.530314    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wdbw\" (UniqueName: \"kubernetes.io/projected/50dcc47e-10dc-4f93-8508-ca2613add5a1-kube-api-access-8wdbw\") pod \"50dcc47e-10dc-4f93-8508-ca2613add5a1\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") "
Jul 20 17:00:29.530409 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.530332    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50dcc47e-10dc-4f93-8508-ca2613add5a1-istio-envoy\") pod \"50dcc47e-10dc-4f93-8508-ca2613add5a1\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") "
Jul 20 17:00:29.530409 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.530361    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50dcc47e-10dc-4f93-8508-ca2613add5a1-istio-data\") pod \"50dcc47e-10dc-4f93-8508-ca2613add5a1\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") "
Jul 20 17:00:29.530409 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.530386    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50dcc47e-10dc-4f93-8508-ca2613add5a1-workload-socket\") pod \"50dcc47e-10dc-4f93-8508-ca2613add5a1\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") "
Jul 20 17:00:29.530709 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.530410    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50dcc47e-10dc-4f93-8508-ca2613add5a1-istiod-ca-cert\") pod \"50dcc47e-10dc-4f93-8508-ca2613add5a1\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") "
Jul 20 17:00:29.530709 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.530437    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50dcc47e-10dc-4f93-8508-ca2613add5a1-credential-socket\") pod \"50dcc47e-10dc-4f93-8508-ca2613add5a1\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") "
Jul 20 17:00:29.530709 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.530485    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50dcc47e-10dc-4f93-8508-ca2613add5a1-istio-token\") pod \"50dcc47e-10dc-4f93-8508-ca2613add5a1\" (UID: \"50dcc47e-10dc-4f93-8508-ca2613add5a1\") "
Jul 20 17:00:29.530709 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.530573    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50dcc47e-10dc-4f93-8508-ca2613add5a1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "50dcc47e-10dc-4f93-8508-ca2613add5a1" (UID: "50dcc47e-10dc-4f93-8508-ca2613add5a1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:00:29.530926 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.530832    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50dcc47e-10dc-4f93-8508-ca2613add5a1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "50dcc47e-10dc-4f93-8508-ca2613add5a1" (UID: "50dcc47e-10dc-4f93-8508-ca2613add5a1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:00:29.530926 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.530857    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50dcc47e-10dc-4f93-8508-ca2613add5a1-workload-certs\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:00:29.530926 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.530861    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50dcc47e-10dc-4f93-8508-ca2613add5a1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "50dcc47e-10dc-4f93-8508-ca2613add5a1" (UID: "50dcc47e-10dc-4f93-8508-ca2613add5a1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 17:00:29.530926 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.530901    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50dcc47e-10dc-4f93-8508-ca2613add5a1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "50dcc47e-10dc-4f93-8508-ca2613add5a1" (UID: "50dcc47e-10dc-4f93-8508-ca2613add5a1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:00:29.531168 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.531143    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50dcc47e-10dc-4f93-8508-ca2613add5a1-istio-data" (OuterVolumeSpecName: "istio-data") pod "50dcc47e-10dc-4f93-8508-ca2613add5a1" (UID: "50dcc47e-10dc-4f93-8508-ca2613add5a1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:00:29.532539 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.532517    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50dcc47e-10dc-4f93-8508-ca2613add5a1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "50dcc47e-10dc-4f93-8508-ca2613add5a1" (UID: "50dcc47e-10dc-4f93-8508-ca2613add5a1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:00:29.532646 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.532572    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/50dcc47e-10dc-4f93-8508-ca2613add5a1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "50dcc47e-10dc-4f93-8508-ca2613add5a1" (UID: "50dcc47e-10dc-4f93-8508-ca2613add5a1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 20 17:00:29.532646 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.532572    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50dcc47e-10dc-4f93-8508-ca2613add5a1-kube-api-access-8wdbw" (OuterVolumeSpecName: "kube-api-access-8wdbw") pod "50dcc47e-10dc-4f93-8508-ca2613add5a1" (UID: "50dcc47e-10dc-4f93-8508-ca2613add5a1"). InnerVolumeSpecName "kube-api-access-8wdbw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:00:29.532747 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.532731    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50dcc47e-10dc-4f93-8508-ca2613add5a1-istio-token" (OuterVolumeSpecName: "istio-token") pod "50dcc47e-10dc-4f93-8508-ca2613add5a1" (UID: "50dcc47e-10dc-4f93-8508-ca2613add5a1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:00:29.632194 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.632118    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wdbw\" (UniqueName: \"kubernetes.io/projected/50dcc47e-10dc-4f93-8508-ca2613add5a1-kube-api-access-8wdbw\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:00:29.632194 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.632145    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50dcc47e-10dc-4f93-8508-ca2613add5a1-istio-envoy\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:00:29.632194 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.632155    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50dcc47e-10dc-4f93-8508-ca2613add5a1-istio-data\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:00:29.632194 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.632164    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50dcc47e-10dc-4f93-8508-ca2613add5a1-workload-socket\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:00:29.632194 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.632173    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50dcc47e-10dc-4f93-8508-ca2613add5a1-istiod-ca-cert\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:00:29.632194 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.632183    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50dcc47e-10dc-4f93-8508-ca2613add5a1-credential-socket\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:00:29.632194 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.632192    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50dcc47e-10dc-4f93-8508-ca2613add5a1-istio-token\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:00:29.632194 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.632201    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50dcc47e-10dc-4f93-8508-ca2613add5a1-istio-podinfo\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:00:29.991195 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.991108    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr" event={"ID":"cd2ebfce-8bdb-44c0-9993-2e2a43badaf0","Type":"ContainerStarted","Data":"9548524383fd58234ee9409ffdc04432aa2b22598733cdc3858e186600ead926"}
Jul 20 17:00:29.991370 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.991220    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr"
Jul 20 17:00:29.992806 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.992781    2577 generic.go:356] "Generic (PLEG): container finished" podID="50dcc47e-10dc-4f93-8508-ca2613add5a1" containerID="ca4f89a6c72fab7198df70d07c98091545cf85d3424abfa736ce75d7fd315528" exitCode=0
Jul 20 17:00:29.992900 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.992827    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"
Jul 20 17:00:29.992900 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.992865    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j" event={"ID":"50dcc47e-10dc-4f93-8508-ca2613add5a1","Type":"ContainerDied","Data":"ca4f89a6c72fab7198df70d07c98091545cf85d3424abfa736ce75d7fd315528"}
Jul 20 17:00:29.992971 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.992909    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j" event={"ID":"50dcc47e-10dc-4f93-8508-ca2613add5a1","Type":"ContainerDied","Data":"ea027c644de6391631a019b4234fe24e5cf939341c60cd88f248154e1f6f9541"}
Jul 20 17:00:29.992971 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:29.992931    2577 scope.go:117] "RemoveContainer" containerID="ca4f89a6c72fab7198df70d07c98091545cf85d3424abfa736ce75d7fd315528"
Jul 20 17:00:30.001918 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:30.001898    2577 scope.go:117] "RemoveContainer" containerID="650b24c3fe590b489e5b03ccdea51f73a204fcd81afde4e34b98301b7a27d787"
Jul 20 17:00:30.009347 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:30.009330    2577 scope.go:117] "RemoveContainer" containerID="ca4f89a6c72fab7198df70d07c98091545cf85d3424abfa736ce75d7fd315528"
Jul 20 17:00:30.009602 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:00:30.009583    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca4f89a6c72fab7198df70d07c98091545cf85d3424abfa736ce75d7fd315528\": container with ID starting with ca4f89a6c72fab7198df70d07c98091545cf85d3424abfa736ce75d7fd315528 not found: ID does not exist" containerID="ca4f89a6c72fab7198df70d07c98091545cf85d3424abfa736ce75d7fd315528"
Jul 20 17:00:30.009654 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:30.009609    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca4f89a6c72fab7198df70d07c98091545cf85d3424abfa736ce75d7fd315528"} err="failed to get container status \"ca4f89a6c72fab7198df70d07c98091545cf85d3424abfa736ce75d7fd315528\": rpc error: code = NotFound desc = could not find container \"ca4f89a6c72fab7198df70d07c98091545cf85d3424abfa736ce75d7fd315528\": container with ID starting with ca4f89a6c72fab7198df70d07c98091545cf85d3424abfa736ce75d7fd315528 not found: ID does not exist"
Jul 20 17:00:30.009654 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:30.009626    2577 scope.go:117] "RemoveContainer" containerID="650b24c3fe590b489e5b03ccdea51f73a204fcd81afde4e34b98301b7a27d787"
Jul 20 17:00:30.009849 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:00:30.009832    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"650b24c3fe590b489e5b03ccdea51f73a204fcd81afde4e34b98301b7a27d787\": container with ID starting with 650b24c3fe590b489e5b03ccdea51f73a204fcd81afde4e34b98301b7a27d787 not found: ID does not exist" containerID="650b24c3fe590b489e5b03ccdea51f73a204fcd81afde4e34b98301b7a27d787"
Jul 20 17:00:30.009902 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:30.009858    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"650b24c3fe590b489e5b03ccdea51f73a204fcd81afde4e34b98301b7a27d787"} err="failed to get container status \"650b24c3fe590b489e5b03ccdea51f73a204fcd81afde4e34b98301b7a27d787\": rpc error: code = NotFound desc = could not find container \"650b24c3fe590b489e5b03ccdea51f73a204fcd81afde4e34b98301b7a27d787\": container with ID starting with 650b24c3fe590b489e5b03ccdea51f73a204fcd81afde4e34b98301b7a27d787 not found: ID does not exist"
Jul 20 17:00:30.016477 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:30.016428    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr" podStartSLOduration=1.8362403029999999 podStartE2EDuration="3.016410965s" totalImagesPullingTime="1.180170662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:00:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:00:30.014134914 +0000 UTC m=+1853.263628931" watchObservedRunningTime="2026-07-20 17:00:30.016410965 +0000 UTC m=+1853.265904985"
Jul 20 17:00:30.028743 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:30.028719    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"]
Jul 20 17:00:30.033331 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:30.033292    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-4c8f9801-openshift-default-558b97d597-4q99j"]
Jul 20 17:00:31.211757 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:31.211723    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50dcc47e-10dc-4f93-8508-ca2613add5a1" path="/var/lib/kubelet/pods/50dcc47e-10dc-4f93-8508-ca2613add5a1/volumes"
Jul 20 17:00:36.005307 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:36.005268    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr"
Jul 20 17:00:36.693263 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:36.693229    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:00:41.046034 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:41.046004    2577 generic.go:356] "Generic (PLEG): container finished" podID="ab56ac26-9a40-462f-8f54-5332a3aae155" containerID="261faaafca6d8fdd54b4cccc5fa339dde1e367612e4751b37e299b27cfa92813" exitCode=0
Jul 20 17:00:41.046469 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:41.046064    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs" event={"ID":"ab56ac26-9a40-462f-8f54-5332a3aae155","Type":"ContainerDied","Data":"261faaafca6d8fdd54b4cccc5fa339dde1e367612e4751b37e299b27cfa92813"}
Jul 20 17:00:41.046469 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:41.046435    2577 scope.go:117] "RemoveContainer" containerID="261faaafca6d8fdd54b4cccc5fa339dde1e367612e4751b37e299b27cfa92813"
Jul 20 17:00:41.047532 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:41.047496    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:00:41.047604 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:41.047584    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:00:41.047648 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:41.047629    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:00:42.055316 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:42.055256    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs" event={"ID":"ab56ac26-9a40-462f-8f54-5332a3aae155","Type":"ContainerStarted","Data":"13f1ff88f9fe1cacb349b68b2cfada3e7540928abe90e8118c28b273b8659ffa"}
Jul 20 17:00:42.272120 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:42.272080    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:42.273324 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:42.273281    2577 patch_prober.go:28] interesting pod/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Jul 20 17:00:42.273433 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:42.273352    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs" podUID="ab56ac26-9a40-462f-8f54-5332a3aae155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Jul 20 17:00:42.980313 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:42.980249    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:43.272415 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:43.272292    2577 patch_prober.go:28] interesting pod/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Jul 20 17:00:43.272415 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:43.272384    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs" podUID="ab56ac26-9a40-462f-8f54-5332a3aae155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Jul 20 17:00:44.272823 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:44.272789    2577 patch_prober.go:28] interesting pod/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Jul 20 17:00:44.273216 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:44.272862    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs" podUID="ab56ac26-9a40-462f-8f54-5332a3aae155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Jul 20 17:00:45.272180 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:45.272137    2577 patch_prober.go:28] interesting pod/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Jul 20 17:00:45.272399 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:45.272198    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs" podUID="ab56ac26-9a40-462f-8f54-5332a3aae155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Jul 20 17:00:46.275899 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:46.275863    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:46.276839 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:46.276822    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:00:50.923357 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:00:50.923323    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:01:02.841071 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:02.840986    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr"]
Jul 20 17:01:02.841626 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:02.841217    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr" podUID="cd2ebfce-8bdb-44c0-9993-2e2a43badaf0" containerName="maas-api" containerID="cri-o://9548524383fd58234ee9409ffdc04432aa2b22598733cdc3858e186600ead926" gracePeriod=30
Jul 20 17:01:02.877727 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:02.877700    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:01:03.087320 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:03.087276    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr"
Jul 20 17:01:03.150308 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:03.150215    2577 generic.go:356] "Generic (PLEG): container finished" podID="cd2ebfce-8bdb-44c0-9993-2e2a43badaf0" containerID="9548524383fd58234ee9409ffdc04432aa2b22598733cdc3858e186600ead926" exitCode=0
Jul 20 17:01:03.150308 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:03.150261    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr"
Jul 20 17:01:03.150493 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:03.150316    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr" event={"ID":"cd2ebfce-8bdb-44c0-9993-2e2a43badaf0","Type":"ContainerDied","Data":"9548524383fd58234ee9409ffdc04432aa2b22598733cdc3858e186600ead926"}
Jul 20 17:01:03.150493 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:03.150373    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr" event={"ID":"cd2ebfce-8bdb-44c0-9993-2e2a43badaf0","Type":"ContainerDied","Data":"ae5d732fc68eedab7e90562d4f680217bd35ec26f905ab458541a3d2e1fd37c4"}
Jul 20 17:01:03.150493 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:03.150397    2577 scope.go:117] "RemoveContainer" containerID="9548524383fd58234ee9409ffdc04432aa2b22598733cdc3858e186600ead926"
Jul 20 17:01:03.158320 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:03.158290    2577 scope.go:117] "RemoveContainer" containerID="9548524383fd58234ee9409ffdc04432aa2b22598733cdc3858e186600ead926"
Jul 20 17:01:03.158559 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:01:03.158541    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9548524383fd58234ee9409ffdc04432aa2b22598733cdc3858e186600ead926\": container with ID starting with 9548524383fd58234ee9409ffdc04432aa2b22598733cdc3858e186600ead926 not found: ID does not exist" containerID="9548524383fd58234ee9409ffdc04432aa2b22598733cdc3858e186600ead926"
Jul 20 17:01:03.158610 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:03.158567    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9548524383fd58234ee9409ffdc04432aa2b22598733cdc3858e186600ead926"} err="failed to get container status \"9548524383fd58234ee9409ffdc04432aa2b22598733cdc3858e186600ead926\": rpc error: code = NotFound desc = could not find container \"9548524383fd58234ee9409ffdc04432aa2b22598733cdc3858e186600ead926\": container with ID starting with 9548524383fd58234ee9409ffdc04432aa2b22598733cdc3858e186600ead926 not found: ID does not exist"
Jul 20 17:01:03.236881 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:03.236846    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ppvrm\" (UniqueName: \"kubernetes.io/projected/cd2ebfce-8bdb-44c0-9993-2e2a43badaf0-kube-api-access-ppvrm\") pod \"cd2ebfce-8bdb-44c0-9993-2e2a43badaf0\" (UID: \"cd2ebfce-8bdb-44c0-9993-2e2a43badaf0\") "
Jul 20 17:01:03.237040 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:03.236890    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd2ebfce-8bdb-44c0-9993-2e2a43badaf0-maas-api-tls\") pod \"cd2ebfce-8bdb-44c0-9993-2e2a43badaf0\" (UID: \"cd2ebfce-8bdb-44c0-9993-2e2a43badaf0\") "
Jul 20 17:01:03.237040 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:03.236941    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd2ebfce-8bdb-44c0-9993-2e2a43badaf0-openshift-service-ca\") pod \"cd2ebfce-8bdb-44c0-9993-2e2a43badaf0\" (UID: \"cd2ebfce-8bdb-44c0-9993-2e2a43badaf0\") "
Jul 20 17:01:03.237379 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:03.237346    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd2ebfce-8bdb-44c0-9993-2e2a43badaf0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cd2ebfce-8bdb-44c0-9993-2e2a43badaf0" (UID: "cd2ebfce-8bdb-44c0-9993-2e2a43badaf0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 17:01:03.238982 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:03.238955    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd2ebfce-8bdb-44c0-9993-2e2a43badaf0-kube-api-access-ppvrm" (OuterVolumeSpecName: "kube-api-access-ppvrm") pod "cd2ebfce-8bdb-44c0-9993-2e2a43badaf0" (UID: "cd2ebfce-8bdb-44c0-9993-2e2a43badaf0"). InnerVolumeSpecName "kube-api-access-ppvrm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:01:03.239067 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:03.238984    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd2ebfce-8bdb-44c0-9993-2e2a43badaf0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cd2ebfce-8bdb-44c0-9993-2e2a43badaf0" (UID: "cd2ebfce-8bdb-44c0-9993-2e2a43badaf0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:01:03.338156 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:03.338118    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd2ebfce-8bdb-44c0-9993-2e2a43badaf0-openshift-service-ca\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:01:03.338156 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:03.338148    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppvrm\" (UniqueName: \"kubernetes.io/projected/cd2ebfce-8bdb-44c0-9993-2e2a43badaf0-kube-api-access-ppvrm\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:01:03.338398 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:03.338239    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd2ebfce-8bdb-44c0-9993-2e2a43badaf0-maas-api-tls\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:01:03.472031 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:03.472001    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr"]
Jul 20 17:01:03.475772 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:03.475748    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c86df9a5-5bdfc86c68-6m8fr"]
Jul 20 17:01:05.212399 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:05.212365    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd2ebfce-8bdb-44c0-9993-2e2a43badaf0" path="/var/lib/kubelet/pods/cd2ebfce-8bdb-44c0-9993-2e2a43badaf0/volumes"
Jul 20 17:01:07.297340 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:07.297291    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:01:07.451824 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:07.451788    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"]
Jul 20 17:01:07.452095 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:07.452067    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs" podUID="ab56ac26-9a40-462f-8f54-5332a3aae155" containerName="istio-proxy" containerID="cri-o://13f1ff88f9fe1cacb349b68b2cfada3e7540928abe90e8118c28b273b8659ffa" gracePeriod=30
Jul 20 17:01:07.467887 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:07.467861    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:01:07.716483 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:07.716394    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:01:08.208546 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:08.208513    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:01:08.303465 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:08.303424    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:01:08.403079 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:08.403043    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:01:09.833773 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:09.833734    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:01:12.700278 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:12.700253    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:01:12.712850 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:12.712774    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab56ac26-9a40-462f-8f54-5332a3aae155-workload-socket\") pod \"ab56ac26-9a40-462f-8f54-5332a3aae155\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") "
Jul 20 17:01:12.712850 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:12.712813    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab56ac26-9a40-462f-8f54-5332a3aae155-istiod-ca-cert\") pod \"ab56ac26-9a40-462f-8f54-5332a3aae155\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") "
Jul 20 17:01:12.713053 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:12.712869    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab56ac26-9a40-462f-8f54-5332a3aae155-istio-envoy\") pod \"ab56ac26-9a40-462f-8f54-5332a3aae155\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") "
Jul 20 17:01:12.713053 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:12.712900    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab56ac26-9a40-462f-8f54-5332a3aae155-istio-data\") pod \"ab56ac26-9a40-462f-8f54-5332a3aae155\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") "
Jul 20 17:01:12.713053 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:12.712924    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab56ac26-9a40-462f-8f54-5332a3aae155-istio-token\") pod \"ab56ac26-9a40-462f-8f54-5332a3aae155\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") "
Jul 20 17:01:12.713053 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:12.712961    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab56ac26-9a40-462f-8f54-5332a3aae155-istio-podinfo\") pod \"ab56ac26-9a40-462f-8f54-5332a3aae155\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") "
Jul 20 17:01:12.713053 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:12.712984    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab56ac26-9a40-462f-8f54-5332a3aae155-credential-socket\") pod \"ab56ac26-9a40-462f-8f54-5332a3aae155\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") "
Jul 20 17:01:12.713053 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:12.713008    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trtbv\" (UniqueName: \"kubernetes.io/projected/ab56ac26-9a40-462f-8f54-5332a3aae155-kube-api-access-trtbv\") pod \"ab56ac26-9a40-462f-8f54-5332a3aae155\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") "
Jul 20 17:01:12.713053 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:12.713041    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab56ac26-9a40-462f-8f54-5332a3aae155-workload-certs\") pod \"ab56ac26-9a40-462f-8f54-5332a3aae155\" (UID: \"ab56ac26-9a40-462f-8f54-5332a3aae155\") "
Jul 20 17:01:12.713431 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:12.713057    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab56ac26-9a40-462f-8f54-5332a3aae155-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ab56ac26-9a40-462f-8f54-5332a3aae155" (UID: "ab56ac26-9a40-462f-8f54-5332a3aae155"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:01:12.713431 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:12.713272    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab56ac26-9a40-462f-8f54-5332a3aae155-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ab56ac26-9a40-462f-8f54-5332a3aae155" (UID: "ab56ac26-9a40-462f-8f54-5332a3aae155"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:01:12.713431 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:12.713281    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab56ac26-9a40-462f-8f54-5332a3aae155-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ab56ac26-9a40-462f-8f54-5332a3aae155" (UID: "ab56ac26-9a40-462f-8f54-5332a3aae155"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 17:01:12.713431 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:12.713291    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab56ac26-9a40-462f-8f54-5332a3aae155-workload-socket\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:01:12.713625 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:12.713536    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab56ac26-9a40-462f-8f54-5332a3aae155-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ab56ac26-9a40-462f-8f54-5332a3aae155" (UID: "ab56ac26-9a40-462f-8f54-5332a3aae155"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:01:12.713752 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:12.713725    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab56ac26-9a40-462f-8f54-5332a3aae155-istio-data" (OuterVolumeSpecName: "istio-data") pod "ab56ac26-9a40-462f-8f54-5332a3aae155" (UID: "ab56ac26-9a40-462f-8f54-5332a3aae155"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:01:12.715190 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:12.715162    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab56ac26-9a40-462f-8f54-5332a3aae155-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ab56ac26-9a40-462f-8f54-5332a3aae155" (UID: "ab56ac26-9a40-462f-8f54-5332a3aae155"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:01:12.715382 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:12.715360    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab56ac26-9a40-462f-8f54-5332a3aae155-kube-api-access-trtbv" (OuterVolumeSpecName: "kube-api-access-trtbv") pod "ab56ac26-9a40-462f-8f54-5332a3aae155" (UID: "ab56ac26-9a40-462f-8f54-5332a3aae155"). InnerVolumeSpecName "kube-api-access-trtbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:01:12.715452 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:12.715437    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab56ac26-9a40-462f-8f54-5332a3aae155-istio-token" (OuterVolumeSpecName: "istio-token") pod "ab56ac26-9a40-462f-8f54-5332a3aae155" (UID: "ab56ac26-9a40-462f-8f54-5332a3aae155"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:01:12.715605 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:12.715591    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ab56ac26-9a40-462f-8f54-5332a3aae155-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ab56ac26-9a40-462f-8f54-5332a3aae155" (UID: "ab56ac26-9a40-462f-8f54-5332a3aae155"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 20 17:01:12.814399 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:12.814363    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab56ac26-9a40-462f-8f54-5332a3aae155-istio-token\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:01:12.814399 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:12.814396    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab56ac26-9a40-462f-8f54-5332a3aae155-istio-podinfo\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:01:12.814399 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:12.814406    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab56ac26-9a40-462f-8f54-5332a3aae155-credential-socket\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:01:12.814660 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:12.814416    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trtbv\" (UniqueName: \"kubernetes.io/projected/ab56ac26-9a40-462f-8f54-5332a3aae155-kube-api-access-trtbv\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:01:12.814660 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:12.814426    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab56ac26-9a40-462f-8f54-5332a3aae155-workload-certs\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:01:12.814660 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:12.814434    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab56ac26-9a40-462f-8f54-5332a3aae155-istiod-ca-cert\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:01:12.814660 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:12.814441    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab56ac26-9a40-462f-8f54-5332a3aae155-istio-envoy\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:01:12.814660 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:12.814448    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab56ac26-9a40-462f-8f54-5332a3aae155-istio-data\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:01:13.195900 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:13.195864    2577 generic.go:356] "Generic (PLEG): container finished" podID="ab56ac26-9a40-462f-8f54-5332a3aae155" containerID="13f1ff88f9fe1cacb349b68b2cfada3e7540928abe90e8118c28b273b8659ffa" exitCode=0
Jul 20 17:01:13.196081 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:13.195911    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs" event={"ID":"ab56ac26-9a40-462f-8f54-5332a3aae155","Type":"ContainerDied","Data":"13f1ff88f9fe1cacb349b68b2cfada3e7540928abe90e8118c28b273b8659ffa"}
Jul 20 17:01:13.196081 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:13.195928    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"
Jul 20 17:01:13.196081 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:13.195999    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs" event={"ID":"ab56ac26-9a40-462f-8f54-5332a3aae155","Type":"ContainerDied","Data":"f8baefccbe3d0529980e72ff9d2f07c12bb53dafcd9ce43f41acdec8326de7a9"}
Jul 20 17:01:13.196081 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:13.196030    2577 scope.go:117] "RemoveContainer" containerID="13f1ff88f9fe1cacb349b68b2cfada3e7540928abe90e8118c28b273b8659ffa"
Jul 20 17:01:13.207502 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:13.207482    2577 scope.go:117] "RemoveContainer" containerID="261faaafca6d8fdd54b4cccc5fa339dde1e367612e4751b37e299b27cfa92813"
Jul 20 17:01:13.216232 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:13.216211    2577 scope.go:117] "RemoveContainer" containerID="13f1ff88f9fe1cacb349b68b2cfada3e7540928abe90e8118c28b273b8659ffa"
Jul 20 17:01:13.216508 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:01:13.216487    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13f1ff88f9fe1cacb349b68b2cfada3e7540928abe90e8118c28b273b8659ffa\": container with ID starting with 13f1ff88f9fe1cacb349b68b2cfada3e7540928abe90e8118c28b273b8659ffa not found: ID does not exist" containerID="13f1ff88f9fe1cacb349b68b2cfada3e7540928abe90e8118c28b273b8659ffa"
Jul 20 17:01:13.216593 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:13.216522    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13f1ff88f9fe1cacb349b68b2cfada3e7540928abe90e8118c28b273b8659ffa"} err="failed to get container status \"13f1ff88f9fe1cacb349b68b2cfada3e7540928abe90e8118c28b273b8659ffa\": rpc error: code = NotFound desc = could not find container \"13f1ff88f9fe1cacb349b68b2cfada3e7540928abe90e8118c28b273b8659ffa\": container with ID starting with 13f1ff88f9fe1cacb349b68b2cfada3e7540928abe90e8118c28b273b8659ffa not found: ID does not exist"
Jul 20 17:01:13.216593 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:13.216548    2577 scope.go:117] "RemoveContainer" containerID="261faaafca6d8fdd54b4cccc5fa339dde1e367612e4751b37e299b27cfa92813"
Jul 20 17:01:13.216821 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:01:13.216802    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"261faaafca6d8fdd54b4cccc5fa339dde1e367612e4751b37e299b27cfa92813\": container with ID starting with 261faaafca6d8fdd54b4cccc5fa339dde1e367612e4751b37e299b27cfa92813 not found: ID does not exist" containerID="261faaafca6d8fdd54b4cccc5fa339dde1e367612e4751b37e299b27cfa92813"
Jul 20 17:01:13.216862 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:13.216827    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"261faaafca6d8fdd54b4cccc5fa339dde1e367612e4751b37e299b27cfa92813"} err="failed to get container status \"261faaafca6d8fdd54b4cccc5fa339dde1e367612e4751b37e299b27cfa92813\": rpc error: code = NotFound desc = could not find container \"261faaafca6d8fdd54b4cccc5fa339dde1e367612e4751b37e299b27cfa92813\": container with ID starting with 261faaafca6d8fdd54b4cccc5fa339dde1e367612e4751b37e299b27cfa92813 not found: ID does not exist"
Jul 20 17:01:13.222166 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:13.222140    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"]
Jul 20 17:01:13.226007 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:13.225979    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-c86df9a5-openshift-default-659d66f67-nd7gs"]
Jul 20 17:01:15.212107 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:15.212070    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab56ac26-9a40-462f-8f54-5332a3aae155" path="/var/lib/kubelet/pods/ab56ac26-9a40-462f-8f54-5332a3aae155/volumes"
Jul 20 17:01:15.456086 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:15.456054    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:01:15.907621 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:15.907591    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:01:16.011746 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:16.011701    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:01:16.212014 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:16.211926    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:01:42.585775 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:42.585723    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:01:48.507878 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:48.507837    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:01:48.617815 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:48.617780    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:01:48.771583 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:01:48.771500    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:02:09.611649 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:09.611578    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:02:13.461519 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:13.461470    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:02:15.460080 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:15.460039    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:02:15.561186 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:15.561142    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:02:15.664579 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:15.664539    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:02:26.167932 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:26.167872    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:02:26.968220 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:26.968180    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:02:34.789362 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:34.789248    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:02:34.917980 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:34.917946    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:02:35.049360 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:35.049332    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:02:35.152060 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:35.152028    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:02:35.938016 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:35.937987    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:02:36.076224 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:36.076192    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:02:37.695622 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:37.695588    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:02:43.797328 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:43.797282    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:02:45.878413 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:45.878367    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:02:51.973978 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:51.973944    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:02:52.073980 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.073950    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:02:52.226347 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.226237    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:02:52.562376 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.562345    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"]
Jul 20 17:02:52.562843 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.562814    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab56ac26-9a40-462f-8f54-5332a3aae155" containerName="istio-proxy"
Jul 20 17:02:52.562843 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.562842    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab56ac26-9a40-462f-8f54-5332a3aae155" containerName="istio-proxy"
Jul 20 17:02:52.562940 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.562857    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd2ebfce-8bdb-44c0-9993-2e2a43badaf0" containerName="maas-api"
Jul 20 17:02:52.562940 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.562867    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd2ebfce-8bdb-44c0-9993-2e2a43badaf0" containerName="maas-api"
Jul 20 17:02:52.562940 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.562890    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50dcc47e-10dc-4f93-8508-ca2613add5a1" containerName="istio-proxy"
Jul 20 17:02:52.562940 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.562900    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="50dcc47e-10dc-4f93-8508-ca2613add5a1" containerName="istio-proxy"
Jul 20 17:02:52.563068 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.562979    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd2ebfce-8bdb-44c0-9993-2e2a43badaf0" containerName="maas-api"
Jul 20 17:02:52.563068 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.562990    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab56ac26-9a40-462f-8f54-5332a3aae155" containerName="istio-proxy"
Jul 20 17:02:52.563068 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.563000    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="50dcc47e-10dc-4f93-8508-ca2613add5a1" containerName="istio-proxy"
Jul 20 17:02:52.563068 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.563011    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="50dcc47e-10dc-4f93-8508-ca2613add5a1" containerName="istio-proxy"
Jul 20 17:02:52.565967 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.565950    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:02:52.570580 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.570558    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 20 17:02:52.570889 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.570867    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-0dc9df57-openshift-default-dockercfg-ksltl\""
Jul 20 17:02:52.571425 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.571385    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f0abe6b-a976-4993-83bd-527d9b4dbaba-workload-certs\") pod \"e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") " pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:02:52.571515 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.571433    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f0abe6b-a976-4993-83bd-527d9b4dbaba-istiod-ca-cert\") pod \"e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") " pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:02:52.571515 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.571501    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f0abe6b-a976-4993-83bd-527d9b4dbaba-credential-socket\") pod \"e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") " pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:02:52.571638 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.571542    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f0abe6b-a976-4993-83bd-527d9b4dbaba-istio-data\") pod \"e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") " pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:02:52.571638 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.571582    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f0abe6b-a976-4993-83bd-527d9b4dbaba-workload-socket\") pod \"e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") " pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:02:52.571638 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.571622    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f0abe6b-a976-4993-83bd-527d9b4dbaba-istio-podinfo\") pod \"e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") " pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:02:52.571796 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.571659    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f0abe6b-a976-4993-83bd-527d9b4dbaba-istio-envoy\") pod \"e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") " pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:02:52.571796 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.571700    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bzh5\" (UniqueName: \"kubernetes.io/projected/8f0abe6b-a976-4993-83bd-527d9b4dbaba-kube-api-access-5bzh5\") pod \"e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") " pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:02:52.571796 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.571738    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f0abe6b-a976-4993-83bd-527d9b4dbaba-istio-token\") pod \"e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") " pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:02:52.597448 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.597425    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"]
Jul 20 17:02:52.606412 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.606383    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:02:52.672077 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.672049    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f0abe6b-a976-4993-83bd-527d9b4dbaba-istio-token\") pod \"e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") " pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:02:52.672232 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.672107    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f0abe6b-a976-4993-83bd-527d9b4dbaba-workload-certs\") pod \"e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") " pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:02:52.672232 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.672124    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f0abe6b-a976-4993-83bd-527d9b4dbaba-istiod-ca-cert\") pod \"e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") " pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:02:52.672232 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.672141    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f0abe6b-a976-4993-83bd-527d9b4dbaba-credential-socket\") pod \"e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") " pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:02:52.672232 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.672157    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f0abe6b-a976-4993-83bd-527d9b4dbaba-istio-data\") pod \"e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") " pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:02:52.672232 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.672177    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f0abe6b-a976-4993-83bd-527d9b4dbaba-workload-socket\") pod \"e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") " pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:02:52.672232 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.672207    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f0abe6b-a976-4993-83bd-527d9b4dbaba-istio-podinfo\") pod \"e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") " pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:02:52.672618 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.672239    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f0abe6b-a976-4993-83bd-527d9b4dbaba-istio-envoy\") pod \"e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") " pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:02:52.672618 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.672274    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bzh5\" (UniqueName: \"kubernetes.io/projected/8f0abe6b-a976-4993-83bd-527d9b4dbaba-kube-api-access-5bzh5\") pod \"e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") " pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:02:52.672618 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.672515    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f0abe6b-a976-4993-83bd-527d9b4dbaba-workload-certs\") pod \"e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") " pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:02:52.672618 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.672591    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f0abe6b-a976-4993-83bd-527d9b4dbaba-istio-data\") pod \"e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") " pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:02:52.672839 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.672774    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f0abe6b-a976-4993-83bd-527d9b4dbaba-workload-socket\") pod \"e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") " pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:02:52.672898 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.672875    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f0abe6b-a976-4993-83bd-527d9b4dbaba-credential-socket\") pod \"e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") " pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:02:52.672993 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.672967    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f0abe6b-a976-4993-83bd-527d9b4dbaba-istiod-ca-cert\") pod \"e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") " pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:02:52.674544 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.674525    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f0abe6b-a976-4993-83bd-527d9b4dbaba-istio-envoy\") pod \"e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") " pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:02:52.674720 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.674703    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f0abe6b-a976-4993-83bd-527d9b4dbaba-istio-podinfo\") pod \"e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") " pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:02:52.684037 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.684015    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f0abe6b-a976-4993-83bd-527d9b4dbaba-istio-token\") pod \"e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") " pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:02:52.684276 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.684258    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bzh5\" (UniqueName: \"kubernetes.io/projected/8f0abe6b-a976-4993-83bd-527d9b4dbaba-kube-api-access-5bzh5\") pod \"e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") " pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:02:52.824330 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.824223    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:02:52.876497 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:52.876463    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:02:53.029286 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:53.029250    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"]
Jul 20 17:02:53.029608 ip-10-0-130-85 kubenswrapper[2577]: W0720 17:02:53.029459    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f0abe6b_a976_4993_83bd_527d9b4dbaba.slice/crio-b1fa31f48fe36c5c6752cb21d59d4d18d68404fd83676400ac991de02dfeefd2 WatchSource:0}: Error finding container b1fa31f48fe36c5c6752cb21d59d4d18d68404fd83676400ac991de02dfeefd2: Status 404 returned error can't find the container with id b1fa31f48fe36c5c6752cb21d59d4d18d68404fd83676400ac991de02dfeefd2
Jul 20 17:02:53.031566 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:53.031531    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:02:53.031646 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:53.031599    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:02:53.031646 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:53.031625    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:02:53.619850 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:53.619813    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc" event={"ID":"8f0abe6b-a976-4993-83bd-527d9b4dbaba","Type":"ContainerStarted","Data":"6bff7674e4f02354adb2a684416d659e1931dde4b473f591e4d41dfce9e4c380"}
Jul 20 17:02:53.620041 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:53.619878    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc" event={"ID":"8f0abe6b-a976-4993-83bd-527d9b4dbaba","Type":"ContainerStarted","Data":"b1fa31f48fe36c5c6752cb21d59d4d18d68404fd83676400ac991de02dfeefd2"}
Jul 20 17:02:53.651446 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:53.651399    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc" podStartSLOduration=1.651382463 podStartE2EDuration="1.651382463s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:02:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:02:53.650890244 +0000 UTC m=+1996.900384264" watchObservedRunningTime="2026-07-20 17:02:53.651382463 +0000 UTC m=+1996.900876482"
Jul 20 17:02:53.877571 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:53.877482    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:02:54.881544 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:54.881510    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:02:55.629253 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:55.629222    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:02:55.630272 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:55.630253    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:02:55.707503 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:55.707476    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:02:55.806554 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:55.806526    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr"]
Jul 20 17:02:55.806844 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:55.806831    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab56ac26-9a40-462f-8f54-5332a3aae155" containerName="istio-proxy"
Jul 20 17:02:55.806924 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:55.806845    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab56ac26-9a40-462f-8f54-5332a3aae155" containerName="istio-proxy"
Jul 20 17:02:55.806924 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:55.806862    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50dcc47e-10dc-4f93-8508-ca2613add5a1" containerName="istio-proxy"
Jul 20 17:02:55.806924 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:55.806868    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="50dcc47e-10dc-4f93-8508-ca2613add5a1" containerName="istio-proxy"
Jul 20 17:02:55.806924 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:55.806919    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab56ac26-9a40-462f-8f54-5332a3aae155" containerName="istio-proxy"
Jul 20 17:02:55.809864 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:55.809846    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr"
Jul 20 17:02:55.814957 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:55.814938    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-0dc9df57-serving-cert\""
Jul 20 17:02:55.815095 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:55.814942    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-kxz6w\""
Jul 20 17:02:55.827539 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:55.827516    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr"]
Jul 20 17:02:55.878069 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:55.878038    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:02:55.896342 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:55.896254    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f-maas-api-tls\") pod \"maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr\" (UID: \"3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr"
Jul 20 17:02:55.896735 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:55.896391    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f-openshift-service-ca\") pod \"maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr\" (UID: \"3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr"
Jul 20 17:02:55.896735 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:55.896435    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8x4q\" (UniqueName: \"kubernetes.io/projected/3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f-kube-api-access-k8x4q\") pod \"maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr\" (UID: \"3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr"
Jul 20 17:02:55.997642 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:55.997607    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f-maas-api-tls\") pod \"maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr\" (UID: \"3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr"
Jul 20 17:02:55.997796 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:55.997685    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f-openshift-service-ca\") pod \"maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr\" (UID: \"3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr"
Jul 20 17:02:55.997796 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:55.997720    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8x4q\" (UniqueName: \"kubernetes.io/projected/3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f-kube-api-access-k8x4q\") pod \"maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr\" (UID: \"3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr"
Jul 20 17:02:55.997796 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:02:55.997753    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-0dc9df57-serving-cert: secret "maas-api-e2e-mt-0dc9df57-serving-cert" not found
Jul 20 17:02:55.997998 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:02:55.997816    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f-maas-api-tls podName:3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f nodeName:}" failed. No retries permitted until 2026-07-20 17:02:56.497799022 +0000 UTC m=+1999.747293019 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f-maas-api-tls") pod "maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr" (UID: "3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f") : secret "maas-api-e2e-mt-0dc9df57-serving-cert" not found
Jul 20 17:02:55.998491 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:55.998469    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f-openshift-service-ca\") pod \"maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr\" (UID: \"3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr"
Jul 20 17:02:56.017042 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:56.017011    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8x4q\" (UniqueName: \"kubernetes.io/projected/3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f-kube-api-access-k8x4q\") pod \"maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr\" (UID: \"3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr"
Jul 20 17:02:56.198903 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:56.198775    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:02:56.407740 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:56.407703    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:02:56.502740 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:56.502660    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f-maas-api-tls\") pod \"maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr\" (UID: \"3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr"
Jul 20 17:02:56.504855 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:56.504831    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f-maas-api-tls\") pod \"maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr\" (UID: \"3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr"
Jul 20 17:02:56.512644 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:56.512624    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:02:56.720375 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:56.720343    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr"
Jul 20 17:02:56.860692 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:56.860530    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr"]
Jul 20 17:02:56.863360 ip-10-0-130-85 kubenswrapper[2577]: W0720 17:02:56.863337    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ffc5de3_fa8f_4fe1_a4ba_c41cde9f335f.slice/crio-08fb5ae8a2cf59f90b298de6fd377421386fb0bf38db8aa1e55ef414d0fb4130 WatchSource:0}: Error finding container 08fb5ae8a2cf59f90b298de6fd377421386fb0bf38db8aa1e55ef414d0fb4130: Status 404 returned error can't find the container with id 08fb5ae8a2cf59f90b298de6fd377421386fb0bf38db8aa1e55ef414d0fb4130
Jul 20 17:02:57.639202 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:57.639143    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr" event={"ID":"3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f","Type":"ContainerStarted","Data":"47b4d1a3263e0c6d76b5988cc7cb3e5b72c867e5636b5a14b1858c98358fffba"}
Jul 20 17:02:57.639692 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:57.639213    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr" event={"ID":"3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f","Type":"ContainerStarted","Data":"08fb5ae8a2cf59f90b298de6fd377421386fb0bf38db8aa1e55ef414d0fb4130"}
Jul 20 17:02:57.639692 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:57.639405    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr"
Jul 20 17:02:57.657908 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:02:57.657825    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr" podStartSLOduration=2.124596701 podStartE2EDuration="2.657810675s" totalImagesPullingTime="533.213974ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:02:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:02:57.656240461 +0000 UTC m=+2000.905734480" watchObservedRunningTime="2026-07-20 17:02:57.657810675 +0000 UTC m=+2000.907304695"
Jul 20 17:03:00.095289 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:00.095254    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:03:00.850062 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:00.850027    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:03:00.952446 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:00.952413    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:03:01.230251 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:01.230165    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:03:03.657177 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:03.657133    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr"
Jul 20 17:03:07.171486 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:07.171455    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:03:07.230489 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:07.230458    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:03:10.630766 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:10.630722    2577 patch_prober.go:28] interesting pod/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Jul 20 17:03:10.631174 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:10.630788    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc" podUID="8f0abe6b-a976-4993-83bd-527d9b4dbaba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Jul 20 17:03:10.696318 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:10.696270    2577 generic.go:356] "Generic (PLEG): container finished" podID="8f0abe6b-a976-4993-83bd-527d9b4dbaba" containerID="6bff7674e4f02354adb2a684416d659e1931dde4b473f591e4d41dfce9e4c380" exitCode=0
Jul 20 17:03:10.696455 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:10.696343    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc" event={"ID":"8f0abe6b-a976-4993-83bd-527d9b4dbaba","Type":"ContainerDied","Data":"6bff7674e4f02354adb2a684416d659e1931dde4b473f591e4d41dfce9e4c380"}
Jul 20 17:03:10.696734 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:10.696720    2577 scope.go:117] "RemoveContainer" containerID="6bff7674e4f02354adb2a684416d659e1931dde4b473f591e4d41dfce9e4c380"
Jul 20 17:03:10.697693 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:10.697662    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:03:10.697799 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:10.697745    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:03:10.697877 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:10.697795    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:03:11.702199 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:11.702159    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc" event={"ID":"8f0abe6b-a976-4993-83bd-527d9b4dbaba","Type":"ContainerStarted","Data":"7af66ff97cd607b7017c5f7254fb40ed88887c074f60e49dbb0cff82fbcdaf62"}
Jul 20 17:03:11.877279 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:11.877241    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:03:11.877495 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:11.877471    2577 patch_prober.go:28] interesting pod/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Jul 20 17:03:11.877538 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:11.877521    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc" podUID="8f0abe6b-a976-4993-83bd-527d9b4dbaba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Jul 20 17:03:12.877392 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:12.877350    2577 patch_prober.go:28] interesting pod/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Jul 20 17:03:12.877857 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:12.877411    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc" podUID="8f0abe6b-a976-4993-83bd-527d9b4dbaba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Jul 20 17:03:13.876985 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:13.876946    2577 patch_prober.go:28] interesting pod/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Jul 20 17:03:13.877193 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:13.876999    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc" podUID="8f0abe6b-a976-4993-83bd-527d9b4dbaba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Jul 20 17:03:14.877314 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:14.877259    2577 patch_prober.go:28] interesting pod/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Jul 20 17:03:14.877707 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:14.877337    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc" podUID="8f0abe6b-a976-4993-83bd-527d9b4dbaba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Jul 20 17:03:15.877586 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:15.877547    2577 patch_prober.go:28] interesting pod/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Jul 20 17:03:15.877973 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:15.877606    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc" podUID="8f0abe6b-a976-4993-83bd-527d9b4dbaba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Jul 20 17:03:16.881393 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:16.881361    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:03:16.881784 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:16.881621    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:03:16.882288 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:16.882270    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:03:18.020050 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:18.020012    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr"]
Jul 20 17:03:18.020542 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:18.020240    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr" podUID="3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f" containerName="maas-api" containerID="cri-o://47b4d1a3263e0c6d76b5988cc7cb3e5b72c867e5636b5a14b1858c98358fffba" gracePeriod=30
Jul 20 17:03:18.047977 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:18.047883    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:03:18.261979 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:18.261957    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr"
Jul 20 17:03:18.399706 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:18.399671    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f-openshift-service-ca\") pod \"3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f\" (UID: \"3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f\") "
Jul 20 17:03:18.399706 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:18.399707    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8x4q\" (UniqueName: \"kubernetes.io/projected/3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f-kube-api-access-k8x4q\") pod \"3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f\" (UID: \"3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f\") "
Jul 20 17:03:18.399972 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:18.399764    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f-maas-api-tls\") pod \"3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f\" (UID: \"3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f\") "
Jul 20 17:03:18.400095 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:18.400061    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f" (UID: "3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 17:03:18.401866 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:18.401841    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f-kube-api-access-k8x4q" (OuterVolumeSpecName: "kube-api-access-k8x4q") pod "3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f" (UID: "3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f"). InnerVolumeSpecName "kube-api-access-k8x4q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:03:18.401965 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:18.401925    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f" (UID: "3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:03:18.500861 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:18.500828    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f-maas-api-tls\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:03:18.500861 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:18.500857    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f-openshift-service-ca\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:03:18.500861 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:18.500867    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8x4q\" (UniqueName: \"kubernetes.io/projected/3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f-kube-api-access-k8x4q\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:03:18.733646 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:18.733562    2577 generic.go:356] "Generic (PLEG): container finished" podID="3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f" containerID="47b4d1a3263e0c6d76b5988cc7cb3e5b72c867e5636b5a14b1858c98358fffba" exitCode=0
Jul 20 17:03:18.733646 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:18.733616    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr"
Jul 20 17:03:18.733868 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:18.733645    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr" event={"ID":"3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f","Type":"ContainerDied","Data":"47b4d1a3263e0c6d76b5988cc7cb3e5b72c867e5636b5a14b1858c98358fffba"}
Jul 20 17:03:18.733868 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:18.733697    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr" event={"ID":"3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f","Type":"ContainerDied","Data":"08fb5ae8a2cf59f90b298de6fd377421386fb0bf38db8aa1e55ef414d0fb4130"}
Jul 20 17:03:18.733868 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:18.733718    2577 scope.go:117] "RemoveContainer" containerID="47b4d1a3263e0c6d76b5988cc7cb3e5b72c867e5636b5a14b1858c98358fffba"
Jul 20 17:03:18.744443 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:18.744420    2577 scope.go:117] "RemoveContainer" containerID="47b4d1a3263e0c6d76b5988cc7cb3e5b72c867e5636b5a14b1858c98358fffba"
Jul 20 17:03:18.744730 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:03:18.744713    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47b4d1a3263e0c6d76b5988cc7cb3e5b72c867e5636b5a14b1858c98358fffba\": container with ID starting with 47b4d1a3263e0c6d76b5988cc7cb3e5b72c867e5636b5a14b1858c98358fffba not found: ID does not exist" containerID="47b4d1a3263e0c6d76b5988cc7cb3e5b72c867e5636b5a14b1858c98358fffba"
Jul 20 17:03:18.744786 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:18.744738    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47b4d1a3263e0c6d76b5988cc7cb3e5b72c867e5636b5a14b1858c98358fffba"} err="failed to get container status \"47b4d1a3263e0c6d76b5988cc7cb3e5b72c867e5636b5a14b1858c98358fffba\": rpc error: code = NotFound desc = could not find container \"47b4d1a3263e0c6d76b5988cc7cb3e5b72c867e5636b5a14b1858c98358fffba\": container with ID starting with 47b4d1a3263e0c6d76b5988cc7cb3e5b72c867e5636b5a14b1858c98358fffba not found: ID does not exist"
Jul 20 17:03:18.757741 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:18.757716    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr"]
Jul 20 17:03:18.761914 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:18.761894    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0dc9df57-685455cc5d-89lkr"]
Jul 20 17:03:19.212526 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:19.212496    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f" path="/var/lib/kubelet/pods/3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f/volumes"
Jul 20 17:03:22.618553 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:22.618518    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:03:23.058470 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:23.058431    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:03:28.638683 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:28.638647    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:03:28.738615 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:28.738580    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:03:28.905680 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:28.905591    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"]
Jul 20 17:03:28.905854 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:28.905793    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc" podUID="8f0abe6b-a976-4993-83bd-527d9b4dbaba" containerName="istio-proxy" containerID="cri-o://7af66ff97cd607b7017c5f7254fb40ed88887c074f60e49dbb0cff82fbcdaf62" gracePeriod=30
Jul 20 17:03:28.935589 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:28.935552    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:03:29.177134 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:29.177051    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:03:29.333479 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:29.333419    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:03:29.428769 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:29.428687    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:03:29.636625 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:29.636570    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:03:31.884164 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:31.884122    2577 patch_prober.go:28] interesting pod/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 20 17:03:31.884573 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:31.884186    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc" podUID="8f0abe6b-a976-4993-83bd-527d9b4dbaba" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 20 17:03:34.155737 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.155713    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:03:34.344243 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.344209    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f0abe6b-a976-4993-83bd-527d9b4dbaba-istio-token\") pod \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") "
Jul 20 17:03:34.344454 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.344254    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f0abe6b-a976-4993-83bd-527d9b4dbaba-istio-data\") pod \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") "
Jul 20 17:03:34.344454 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.344292    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f0abe6b-a976-4993-83bd-527d9b4dbaba-workload-socket\") pod \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") "
Jul 20 17:03:34.344454 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.344353    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f0abe6b-a976-4993-83bd-527d9b4dbaba-credential-socket\") pod \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") "
Jul 20 17:03:34.344454 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.344402    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5bzh5\" (UniqueName: \"kubernetes.io/projected/8f0abe6b-a976-4993-83bd-527d9b4dbaba-kube-api-access-5bzh5\") pod \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") "
Jul 20 17:03:34.344454 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.344430    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f0abe6b-a976-4993-83bd-527d9b4dbaba-istio-podinfo\") pod \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") "
Jul 20 17:03:34.344735 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.344481    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f0abe6b-a976-4993-83bd-527d9b4dbaba-workload-certs\") pod \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") "
Jul 20 17:03:34.344735 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.344508    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f0abe6b-a976-4993-83bd-527d9b4dbaba-istio-envoy\") pod \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") "
Jul 20 17:03:34.344735 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.344541    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f0abe6b-a976-4993-83bd-527d9b4dbaba-istiod-ca-cert\") pod \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\" (UID: \"8f0abe6b-a976-4993-83bd-527d9b4dbaba\") "
Jul 20 17:03:34.344735 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.344566    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f0abe6b-a976-4993-83bd-527d9b4dbaba-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8f0abe6b-a976-4993-83bd-527d9b4dbaba" (UID: "8f0abe6b-a976-4993-83bd-527d9b4dbaba"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:03:34.344987 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.344728    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f0abe6b-a976-4993-83bd-527d9b4dbaba-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8f0abe6b-a976-4993-83bd-527d9b4dbaba" (UID: "8f0abe6b-a976-4993-83bd-527d9b4dbaba"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:03:34.344987 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.344752    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f0abe6b-a976-4993-83bd-527d9b4dbaba-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8f0abe6b-a976-4993-83bd-527d9b4dbaba" (UID: "8f0abe6b-a976-4993-83bd-527d9b4dbaba"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:03:34.344987 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.344905    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f0abe6b-a976-4993-83bd-527d9b4dbaba-workload-socket\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:03:34.344987 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.344926    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f0abe6b-a976-4993-83bd-527d9b4dbaba-credential-socket\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:03:34.344987 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.344942    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f0abe6b-a976-4993-83bd-527d9b4dbaba-workload-certs\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:03:34.345233 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.345020    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f0abe6b-a976-4993-83bd-527d9b4dbaba-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8f0abe6b-a976-4993-83bd-527d9b4dbaba" (UID: "8f0abe6b-a976-4993-83bd-527d9b4dbaba"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 17:03:34.345233 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.345182    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f0abe6b-a976-4993-83bd-527d9b4dbaba-istio-data" (OuterVolumeSpecName: "istio-data") pod "8f0abe6b-a976-4993-83bd-527d9b4dbaba" (UID: "8f0abe6b-a976-4993-83bd-527d9b4dbaba"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:03:34.346907 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.346878    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f0abe6b-a976-4993-83bd-527d9b4dbaba-kube-api-access-5bzh5" (OuterVolumeSpecName: "kube-api-access-5bzh5") pod "8f0abe6b-a976-4993-83bd-527d9b4dbaba" (UID: "8f0abe6b-a976-4993-83bd-527d9b4dbaba"). InnerVolumeSpecName "kube-api-access-5bzh5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:03:34.346907 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.346884    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8f0abe6b-a976-4993-83bd-527d9b4dbaba-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8f0abe6b-a976-4993-83bd-527d9b4dbaba" (UID: "8f0abe6b-a976-4993-83bd-527d9b4dbaba"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 20 17:03:34.347078 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.346954    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f0abe6b-a976-4993-83bd-527d9b4dbaba-istio-token" (OuterVolumeSpecName: "istio-token") pod "8f0abe6b-a976-4993-83bd-527d9b4dbaba" (UID: "8f0abe6b-a976-4993-83bd-527d9b4dbaba"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:03:34.347078 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.346979    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f0abe6b-a976-4993-83bd-527d9b4dbaba-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8f0abe6b-a976-4993-83bd-527d9b4dbaba" (UID: "8f0abe6b-a976-4993-83bd-527d9b4dbaba"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:03:34.445813 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.445773    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f0abe6b-a976-4993-83bd-527d9b4dbaba-istio-podinfo\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:03:34.445813 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.445807    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f0abe6b-a976-4993-83bd-527d9b4dbaba-istio-envoy\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:03:34.445813 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.445819    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f0abe6b-a976-4993-83bd-527d9b4dbaba-istiod-ca-cert\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:03:34.446056 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.445830    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f0abe6b-a976-4993-83bd-527d9b4dbaba-istio-token\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:03:34.446056 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.445840    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f0abe6b-a976-4993-83bd-527d9b4dbaba-istio-data\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:03:34.446056 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.445853    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5bzh5\" (UniqueName: \"kubernetes.io/projected/8f0abe6b-a976-4993-83bd-527d9b4dbaba-kube-api-access-5bzh5\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:03:34.806435 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.806398    2577 generic.go:356] "Generic (PLEG): container finished" podID="8f0abe6b-a976-4993-83bd-527d9b4dbaba" containerID="7af66ff97cd607b7017c5f7254fb40ed88887c074f60e49dbb0cff82fbcdaf62" exitCode=0
Jul 20 17:03:34.806600 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.806443    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"
Jul 20 17:03:34.806600 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.806486    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc" event={"ID":"8f0abe6b-a976-4993-83bd-527d9b4dbaba","Type":"ContainerDied","Data":"7af66ff97cd607b7017c5f7254fb40ed88887c074f60e49dbb0cff82fbcdaf62"}
Jul 20 17:03:34.806600 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.806535    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc" event={"ID":"8f0abe6b-a976-4993-83bd-527d9b4dbaba","Type":"ContainerDied","Data":"b1fa31f48fe36c5c6752cb21d59d4d18d68404fd83676400ac991de02dfeefd2"}
Jul 20 17:03:34.806600 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.806556    2577 scope.go:117] "RemoveContainer" containerID="7af66ff97cd607b7017c5f7254fb40ed88887c074f60e49dbb0cff82fbcdaf62"
Jul 20 17:03:34.815215 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.815196    2577 scope.go:117] "RemoveContainer" containerID="6bff7674e4f02354adb2a684416d659e1931dde4b473f591e4d41dfce9e4c380"
Jul 20 17:03:34.822476 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.822457    2577 scope.go:117] "RemoveContainer" containerID="7af66ff97cd607b7017c5f7254fb40ed88887c074f60e49dbb0cff82fbcdaf62"
Jul 20 17:03:34.822719 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:03:34.822699    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7af66ff97cd607b7017c5f7254fb40ed88887c074f60e49dbb0cff82fbcdaf62\": container with ID starting with 7af66ff97cd607b7017c5f7254fb40ed88887c074f60e49dbb0cff82fbcdaf62 not found: ID does not exist" containerID="7af66ff97cd607b7017c5f7254fb40ed88887c074f60e49dbb0cff82fbcdaf62"
Jul 20 17:03:34.822781 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.822727    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7af66ff97cd607b7017c5f7254fb40ed88887c074f60e49dbb0cff82fbcdaf62"} err="failed to get container status \"7af66ff97cd607b7017c5f7254fb40ed88887c074f60e49dbb0cff82fbcdaf62\": rpc error: code = NotFound desc = could not find container \"7af66ff97cd607b7017c5f7254fb40ed88887c074f60e49dbb0cff82fbcdaf62\": container with ID starting with 7af66ff97cd607b7017c5f7254fb40ed88887c074f60e49dbb0cff82fbcdaf62 not found: ID does not exist"
Jul 20 17:03:34.822781 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.822745    2577 scope.go:117] "RemoveContainer" containerID="6bff7674e4f02354adb2a684416d659e1931dde4b473f591e4d41dfce9e4c380"
Jul 20 17:03:34.822986 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:03:34.822967    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bff7674e4f02354adb2a684416d659e1931dde4b473f591e4d41dfce9e4c380\": container with ID starting with 6bff7674e4f02354adb2a684416d659e1931dde4b473f591e4d41dfce9e4c380 not found: ID does not exist" containerID="6bff7674e4f02354adb2a684416d659e1931dde4b473f591e4d41dfce9e4c380"
Jul 20 17:03:34.823027 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.822992    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bff7674e4f02354adb2a684416d659e1931dde4b473f591e4d41dfce9e4c380"} err="failed to get container status \"6bff7674e4f02354adb2a684416d659e1931dde4b473f591e4d41dfce9e4c380\": rpc error: code = NotFound desc = could not find container \"6bff7674e4f02354adb2a684416d659e1931dde4b473f591e4d41dfce9e4c380\": container with ID starting with 6bff7674e4f02354adb2a684416d659e1931dde4b473f591e4d41dfce9e4c380 not found: ID does not exist"
Jul 20 17:03:34.828523 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.828500    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"]
Jul 20 17:03:34.832017 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:34.831997    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-0dc9df57-openshift-default-778d56dc7f-9pxrc"]
Jul 20 17:03:35.212319 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:35.212198    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f0abe6b-a976-4993-83bd-527d9b4dbaba" path="/var/lib/kubelet/pods/8f0abe6b-a976-4993-83bd-527d9b4dbaba/volumes"
Jul 20 17:03:35.416213 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:35.416175    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:03:35.489369 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:35.489272    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:03:35.641459 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:35.641429    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:03:35.742083 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:35.741994    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:03:35.843486 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:35.843451    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:03:41.587813 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:41.587780    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:03:41.660474 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:41.660440    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:03:48.381985 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.381950    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:03:48.660979 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.660893    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"]
Jul 20 17:03:48.661403 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.661384    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f0abe6b-a976-4993-83bd-527d9b4dbaba" containerName="istio-proxy"
Jul 20 17:03:48.661493 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.661404    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f0abe6b-a976-4993-83bd-527d9b4dbaba" containerName="istio-proxy"
Jul 20 17:03:48.661493 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.661419    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f0abe6b-a976-4993-83bd-527d9b4dbaba" containerName="istio-proxy"
Jul 20 17:03:48.661493 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.661427    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f0abe6b-a976-4993-83bd-527d9b4dbaba" containerName="istio-proxy"
Jul 20 17:03:48.661493 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.661439    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f" containerName="maas-api"
Jul 20 17:03:48.661493 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.661447    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f" containerName="maas-api"
Jul 20 17:03:48.661643 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.661527    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ffc5de3-fa8f-4fe1-a4ba-c41cde9f335f" containerName="maas-api"
Jul 20 17:03:48.661643 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.661542    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f0abe6b-a976-4993-83bd-527d9b4dbaba" containerName="istio-proxy"
Jul 20 17:03:48.661643 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.661553    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f0abe6b-a976-4993-83bd-527d9b4dbaba" containerName="istio-proxy"
Jul 20 17:03:48.663628 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.663610    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:03:48.666467 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.666444    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-06ad55ec-openshift-default-dockercfg-grbbn\""
Jul 20 17:03:48.666573 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.666444    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 20 17:03:48.676260 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.676238    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"]
Jul 20 17:03:48.714610 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.714554    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:03:48.777575 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.777535    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-istio-podinfo\") pod \"e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") " pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:03:48.777575 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.777573    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-istio-token\") pod \"e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") " pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:03:48.777789 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.777601    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-istio-envoy\") pod \"e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") " pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:03:48.777789 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.777656    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-credential-socket\") pod \"e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") " pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:03:48.777789 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.777738    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-istio-data\") pod \"e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") " pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:03:48.777908 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.777789    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-workload-socket\") pod \"e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") " pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:03:48.777908 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.777870    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-workload-certs\") pod \"e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") " pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:03:48.777978 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.777916    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-istiod-ca-cert\") pod \"e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") " pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:03:48.777978 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.777945    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsth9\" (UniqueName: \"kubernetes.io/projected/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-kube-api-access-hsth9\") pod \"e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") " pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:03:48.878527 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.878490    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-istio-podinfo\") pod \"e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") " pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:03:48.878527 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.878529    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-istio-token\") pod \"e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") " pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:03:48.878758 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.878555    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-istio-envoy\") pod \"e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") " pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:03:48.878758 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.878682    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-credential-socket\") pod \"e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") " pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:03:48.878758 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.878734    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-istio-data\") pod \"e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") " pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:03:48.878937 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.878774    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-workload-socket\") pod \"e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") " pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:03:48.878937 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.878801    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-workload-certs\") pod \"e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") " pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:03:48.878937 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.878840    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-istiod-ca-cert\") pod \"e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") " pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:03:48.878937 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.878865    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsth9\" (UniqueName: \"kubernetes.io/projected/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-kube-api-access-hsth9\") pod \"e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") " pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:03:48.879147 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.879111    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-credential-socket\") pod \"e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") " pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:03:48.879205 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.879162    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-istio-data\") pod \"e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") " pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:03:48.879257 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.879208    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-workload-certs\") pod \"e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") " pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:03:48.879396 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.879377    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-workload-socket\") pod \"e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") " pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:03:48.879724 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.879702    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-istiod-ca-cert\") pod \"e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") " pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:03:48.881403 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.881378    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-istio-envoy\") pod \"e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") " pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:03:48.881633 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.881604    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:03:48.881701 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.881657    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-istio-podinfo\") pod \"e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") " pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:03:48.886774 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.886746    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-istio-token\") pod \"e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") " pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:03:48.887481 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.887466    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsth9\" (UniqueName: \"kubernetes.io/projected/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-kube-api-access-hsth9\") pod \"e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") " pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:03:48.974596 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:48.974526    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:03:49.107522 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:49.107496    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"]
Jul 20 17:03:49.109256 ip-10-0-130-85 kubenswrapper[2577]: W0720 17:03:49.109228    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff7a5a88_925e_488e_b93e_fe9a40e3d8e4.slice/crio-7d785c87cb2fb639580d07b211007b6582660c7512d6d25ee2ce21e46e86ce5b WatchSource:0}: Error finding container 7d785c87cb2fb639580d07b211007b6582660c7512d6d25ee2ce21e46e86ce5b: Status 404 returned error can't find the container with id 7d785c87cb2fb639580d07b211007b6582660c7512d6d25ee2ce21e46e86ce5b
Jul 20 17:03:49.111249 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:49.111214    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:03:49.111339 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:49.111283    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:03:49.111384 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:49.111352    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:03:49.869717 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:49.869677    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb" event={"ID":"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4","Type":"ContainerStarted","Data":"5b0d68027fd82af7b670257959c8f022729342272bcc45dfabfb63b694732096"}
Jul 20 17:03:49.870088 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:49.869736    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb" event={"ID":"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4","Type":"ContainerStarted","Data":"7d785c87cb2fb639580d07b211007b6582660c7512d6d25ee2ce21e46e86ce5b"}
Jul 20 17:03:49.895446 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:49.895394    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb" podStartSLOduration=1.8953793669999999 podStartE2EDuration="1.895379367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:03:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:03:49.894288685 +0000 UTC m=+2053.143782717" watchObservedRunningTime="2026-07-20 17:03:49.895379367 +0000 UTC m=+2053.144873386"
Jul 20 17:03:49.975013 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:49.974976    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:03:50.980628 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:50.980599    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:03:51.805319 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:51.805254    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:03:51.878600 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:51.878568    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:03:51.879577 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:51.879561    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:03:51.898270 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:51.898238    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc"]
Jul 20 17:03:51.901073 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:51.901056    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc"
Jul 20 17:03:51.903519 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:51.903498    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-kxz6w\""
Jul 20 17:03:51.903741 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:51.903726    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-06ad55ec-serving-cert\""
Jul 20 17:03:51.904737 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:51.904718    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:03:51.916240 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:51.916215    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc"]
Jul 20 17:03:52.000458 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:52.000428    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:03:52.009042 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:52.009019    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bfea49cf-4f39-4bce-a403-c37c5b02018a-maas-api-tls\") pod \"maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc\" (UID: \"bfea49cf-4f39-4bce-a403-c37c5b02018a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc"
Jul 20 17:03:52.009121 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:52.009052    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfea49cf-4f39-4bce-a403-c37c5b02018a-openshift-service-ca\") pod \"maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc\" (UID: \"bfea49cf-4f39-4bce-a403-c37c5b02018a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc"
Jul 20 17:03:52.009121 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:52.009084    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxnq6\" (UniqueName: \"kubernetes.io/projected/bfea49cf-4f39-4bce-a403-c37c5b02018a-kube-api-access-mxnq6\") pod \"maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc\" (UID: \"bfea49cf-4f39-4bce-a403-c37c5b02018a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc"
Jul 20 17:03:52.108158 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:52.108065    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:03:52.110316 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:52.110272    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bfea49cf-4f39-4bce-a403-c37c5b02018a-maas-api-tls\") pod \"maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc\" (UID: \"bfea49cf-4f39-4bce-a403-c37c5b02018a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc"
Jul 20 17:03:52.110451 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:52.110335    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfea49cf-4f39-4bce-a403-c37c5b02018a-openshift-service-ca\") pod \"maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc\" (UID: \"bfea49cf-4f39-4bce-a403-c37c5b02018a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc"
Jul 20 17:03:52.110451 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:52.110378    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxnq6\" (UniqueName: \"kubernetes.io/projected/bfea49cf-4f39-4bce-a403-c37c5b02018a-kube-api-access-mxnq6\") pod \"maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc\" (UID: \"bfea49cf-4f39-4bce-a403-c37c5b02018a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc"
Jul 20 17:03:52.111240 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:52.111205    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfea49cf-4f39-4bce-a403-c37c5b02018a-openshift-service-ca\") pod \"maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc\" (UID: \"bfea49cf-4f39-4bce-a403-c37c5b02018a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc"
Jul 20 17:03:52.112981 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:52.112957    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bfea49cf-4f39-4bce-a403-c37c5b02018a-maas-api-tls\") pod \"maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc\" (UID: \"bfea49cf-4f39-4bce-a403-c37c5b02018a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc"
Jul 20 17:03:52.124193 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:52.124168    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxnq6\" (UniqueName: \"kubernetes.io/projected/bfea49cf-4f39-4bce-a403-c37c5b02018a-kube-api-access-mxnq6\") pod \"maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc\" (UID: \"bfea49cf-4f39-4bce-a403-c37c5b02018a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc"
Jul 20 17:03:52.212440 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:52.212403    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc"
Jul 20 17:03:52.344223 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:52.344188    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc"]
Jul 20 17:03:52.347113 ip-10-0-130-85 kubenswrapper[2577]: W0720 17:03:52.347081    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfea49cf_4f39_4bce_a403_c37c5b02018a.slice/crio-0e2088e31f97c88c0e523a4605b21d7efa936aa18fff199dd90b97988a50d840 WatchSource:0}: Error finding container 0e2088e31f97c88c0e523a4605b21d7efa936aa18fff199dd90b97988a50d840: Status 404 returned error can't find the container with id 0e2088e31f97c88c0e523a4605b21d7efa936aa18fff199dd90b97988a50d840
Jul 20 17:03:52.348572 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:52.348552    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 17:03:52.594149 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:52.594122    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:03:52.696152 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:52.696103    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:03:52.884703 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:52.884604    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc" event={"ID":"bfea49cf-4f39-4bce-a403-c37c5b02018a","Type":"ContainerStarted","Data":"0e2088e31f97c88c0e523a4605b21d7efa936aa18fff199dd90b97988a50d840"}
Jul 20 17:03:53.890220 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:53.890178    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc" event={"ID":"bfea49cf-4f39-4bce-a403-c37c5b02018a","Type":"ContainerStarted","Data":"52d28d2def40245994b1a96921bc2ff2d05ff0eec4db574b3f6ad4775c01c620"}
Jul 20 17:03:53.890703 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:53.890365    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc"
Jul 20 17:03:53.926622 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:53.926565    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc" podStartSLOduration=2.364181749 podStartE2EDuration="2.926546443s" totalImagesPullingTime="562.364694ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:03:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:03:53.925499756 +0000 UTC m=+2057.174993777" watchObservedRunningTime="2026-07-20 17:03:53.926546443 +0000 UTC m=+2057.176040533"
Jul 20 17:03:56.297293 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:56.297256    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:03:59.901264 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:03:59.901176    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc"
Jul 20 17:04:06.880077 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:06.880038    2577 patch_prober.go:28] interesting pod/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 20 17:04:06.880536 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:06.880109    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb" podUID="ff7a5a88-925e-488e-b93e-fe9a40e3d8e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 20 17:04:06.949050 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:06.949016    2577 generic.go:356] "Generic (PLEG): container finished" podID="ff7a5a88-925e-488e-b93e-fe9a40e3d8e4" containerID="5b0d68027fd82af7b670257959c8f022729342272bcc45dfabfb63b694732096" exitCode=0
Jul 20 17:04:06.949211 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:06.949056    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb" event={"ID":"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4","Type":"ContainerDied","Data":"5b0d68027fd82af7b670257959c8f022729342272bcc45dfabfb63b694732096"}
Jul 20 17:04:06.949513 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:06.949493    2577 scope.go:117] "RemoveContainer" containerID="5b0d68027fd82af7b670257959c8f022729342272bcc45dfabfb63b694732096"
Jul 20 17:04:06.950491 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:06.950462    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:04:06.950554 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:06.950528    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:04:06.950591 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:06.950558    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:04:07.961807 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:07.961772    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb" event={"ID":"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4","Type":"ContainerStarted","Data":"02ef08375fdc073c61f8ec492f0c203b9d71d19fd6f9766a542897603329d186"}
Jul 20 17:04:07.974962 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:07.974933    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:04:08.975716 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:08.975678    2577 patch_prober.go:28] interesting pod/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 20 17:04:08.976149 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:08.975734    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb" podUID="ff7a5a88-925e-488e-b93e-fe9a40e3d8e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 20 17:04:09.975475 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:09.975443    2577 patch_prober.go:28] interesting pod/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 20 17:04:09.975654 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:09.975495    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb" podUID="ff7a5a88-925e-488e-b93e-fe9a40e3d8e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 20 17:04:10.295233 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:10.295198    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt"]
Jul 20 17:04:10.297546 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:10.297526    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt"
Jul 20 17:04:10.300256 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:10.300233    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-06ad55ec\"/\"openshift-service-ca.crt\""
Jul 20 17:04:10.301211 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:10.301188    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-06ad55ec\"/\"default-dockercfg-vrzs7\""
Jul 20 17:04:10.301353 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:10.301188    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-06ad55ec\"/\"kube-root-ca.crt\""
Jul 20 17:04:10.301353 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:10.301253    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-06ad55ec\"/\"e2e-lifecycle-model-06ad55ec-kserve-self-signed-certs\""
Jul 20 17:04:10.308460 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:10.308436    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt"]
Jul 20 17:04:10.353335 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:10.352941    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:04:10.370891 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:10.370860    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f21cdcce-1126-450e-9a9b-754d67fbeb99-dshm\") pod \"e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt\" (UID: \"f21cdcce-1126-450e-9a9b-754d67fbeb99\") " pod="ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt"
Jul 20 17:04:10.370891 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:10.370892    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f21cdcce-1126-450e-9a9b-754d67fbeb99-tmp-dir\") pod \"e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt\" (UID: \"f21cdcce-1126-450e-9a9b-754d67fbeb99\") " pod="ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt"
Jul 20 17:04:10.371083 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:10.371003    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f21cdcce-1126-450e-9a9b-754d67fbeb99-model-cache\") pod \"e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt\" (UID: \"f21cdcce-1126-450e-9a9b-754d67fbeb99\") " pod="ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt"
Jul 20 17:04:10.371083 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:10.371043    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9ckw\" (UniqueName: \"kubernetes.io/projected/f21cdcce-1126-450e-9a9b-754d67fbeb99-kube-api-access-h9ckw\") pod \"e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt\" (UID: \"f21cdcce-1126-450e-9a9b-754d67fbeb99\") " pod="ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt"
Jul 20 17:04:10.371160 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:10.371090    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f21cdcce-1126-450e-9a9b-754d67fbeb99-tls-certs\") pod \"e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt\" (UID: \"f21cdcce-1126-450e-9a9b-754d67fbeb99\") " pod="ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt"
Jul 20 17:04:10.371160 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:10.371134    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f21cdcce-1126-450e-9a9b-754d67fbeb99-home\") pod \"e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt\" (UID: \"f21cdcce-1126-450e-9a9b-754d67fbeb99\") " pod="ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt"
Jul 20 17:04:10.471956 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:10.471917    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f21cdcce-1126-450e-9a9b-754d67fbeb99-model-cache\") pod \"e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt\" (UID: \"f21cdcce-1126-450e-9a9b-754d67fbeb99\") " pod="ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt"
Jul 20 17:04:10.471956 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:10.471955    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9ckw\" (UniqueName: \"kubernetes.io/projected/f21cdcce-1126-450e-9a9b-754d67fbeb99-kube-api-access-h9ckw\") pod \"e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt\" (UID: \"f21cdcce-1126-450e-9a9b-754d67fbeb99\") " pod="ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt"
Jul 20 17:04:10.472232 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:10.471981    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f21cdcce-1126-450e-9a9b-754d67fbeb99-tls-certs\") pod \"e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt\" (UID: \"f21cdcce-1126-450e-9a9b-754d67fbeb99\") " pod="ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt"
Jul 20 17:04:10.472232 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:10.472018    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f21cdcce-1126-450e-9a9b-754d67fbeb99-home\") pod \"e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt\" (UID: \"f21cdcce-1126-450e-9a9b-754d67fbeb99\") " pod="ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt"
Jul 20 17:04:10.472232 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:10.472054    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f21cdcce-1126-450e-9a9b-754d67fbeb99-dshm\") pod \"e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt\" (UID: \"f21cdcce-1126-450e-9a9b-754d67fbeb99\") " pod="ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt"
Jul 20 17:04:10.472232 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:10.472071    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f21cdcce-1126-450e-9a9b-754d67fbeb99-tmp-dir\") pod \"e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt\" (UID: \"f21cdcce-1126-450e-9a9b-754d67fbeb99\") " pod="ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt"
Jul 20 17:04:10.472486 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:10.472440    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f21cdcce-1126-450e-9a9b-754d67fbeb99-model-cache\") pod \"e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt\" (UID: \"f21cdcce-1126-450e-9a9b-754d67fbeb99\") " pod="ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt"
Jul 20 17:04:10.472486 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:10.472455    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f21cdcce-1126-450e-9a9b-754d67fbeb99-home\") pod \"e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt\" (UID: \"f21cdcce-1126-450e-9a9b-754d67fbeb99\") " pod="ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt"
Jul 20 17:04:10.472559 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:10.472519    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f21cdcce-1126-450e-9a9b-754d67fbeb99-tmp-dir\") pod \"e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt\" (UID: \"f21cdcce-1126-450e-9a9b-754d67fbeb99\") " pod="ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt"
Jul 20 17:04:10.474421 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:10.474402    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f21cdcce-1126-450e-9a9b-754d67fbeb99-dshm\") pod \"e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt\" (UID: \"f21cdcce-1126-450e-9a9b-754d67fbeb99\") " pod="ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt"
Jul 20 17:04:10.474529 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:10.474512    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f21cdcce-1126-450e-9a9b-754d67fbeb99-tls-certs\") pod \"e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt\" (UID: \"f21cdcce-1126-450e-9a9b-754d67fbeb99\") " pod="ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt"
Jul 20 17:04:10.481456 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:10.481436    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9ckw\" (UniqueName: \"kubernetes.io/projected/f21cdcce-1126-450e-9a9b-754d67fbeb99-kube-api-access-h9ckw\") pod \"e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt\" (UID: \"f21cdcce-1126-450e-9a9b-754d67fbeb99\") " pod="ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt"
Jul 20 17:04:10.608870 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:10.608789    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt"
Jul 20 17:04:10.788367 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:10.787355    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt"]
Jul 20 17:04:10.975035 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:10.974939    2577 patch_prober.go:28] interesting pod/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 20 17:04:10.975035 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:10.975003    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb" podUID="ff7a5a88-925e-488e-b93e-fe9a40e3d8e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 20 17:04:10.977445 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:10.977414    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt" event={"ID":"f21cdcce-1126-450e-9a9b-754d67fbeb99","Type":"ContainerStarted","Data":"6aaead9be34e7f14e27c4ae22394729b12798769698b6fd7363e4837416d5537"}
Jul 20 17:04:10.977591 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:10.977456    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt" event={"ID":"f21cdcce-1126-450e-9a9b-754d67fbeb99","Type":"ContainerStarted","Data":"729e807aadbb3c7ff12963b6c59a860b42761b20280b9ef699e4ffd298fb7b5d"}
Jul 20 17:04:10.999233 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:10.999186    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt" podStartSLOduration=0.999169625 podStartE2EDuration="999.169625ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:04:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:04:10.998854817 +0000 UTC m=+2074.248348838" watchObservedRunningTime="2026-07-20 17:04:10.999169625 +0000 UTC m=+2074.248663643"
Jul 20 17:04:11.975216 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:11.975174    2577 patch_prober.go:28] interesting pod/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 20 17:04:11.975647 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:11.975233    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb" podUID="ff7a5a88-925e-488e-b93e-fe9a40e3d8e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 20 17:04:12.979521 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:12.979486    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:04:12.979942 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:12.979751    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:04:12.980383 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:12.980366    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:04:20.608905 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:20.608870    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt"
Jul 20 17:04:20.609327 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:20.608923    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt"
Jul 20 17:04:20.621406 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:20.621379    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt"
Jul 20 17:04:21.033407 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:21.033325    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt"
Jul 20 17:04:37.239771 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:37.239740    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gd4p4_fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c/ovn-acl-logging/0.log"
Jul 20 17:04:37.244844 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:37.244820    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gd4p4_fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c/ovn-acl-logging/0.log"
Jul 20 17:04:45.167579 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:45.167541    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc"]
Jul 20 17:04:45.168064 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:45.167736    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc" podUID="bfea49cf-4f39-4bce-a403-c37c5b02018a" containerName="maas-api" containerID="cri-o://52d28d2def40245994b1a96921bc2ff2d05ff0eec4db574b3f6ad4775c01c620" gracePeriod=30
Jul 20 17:04:45.195470 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:45.195446    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:04:45.413949 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:45.413921    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc"
Jul 20 17:04:45.471494 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:45.471433    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bfea49cf-4f39-4bce-a403-c37c5b02018a-maas-api-tls\") pod \"bfea49cf-4f39-4bce-a403-c37c5b02018a\" (UID: \"bfea49cf-4f39-4bce-a403-c37c5b02018a\") "
Jul 20 17:04:45.471612 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:45.471545    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfea49cf-4f39-4bce-a403-c37c5b02018a-openshift-service-ca\") pod \"bfea49cf-4f39-4bce-a403-c37c5b02018a\" (UID: \"bfea49cf-4f39-4bce-a403-c37c5b02018a\") "
Jul 20 17:04:45.471612 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:45.471573    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxnq6\" (UniqueName: \"kubernetes.io/projected/bfea49cf-4f39-4bce-a403-c37c5b02018a-kube-api-access-mxnq6\") pod \"bfea49cf-4f39-4bce-a403-c37c5b02018a\" (UID: \"bfea49cf-4f39-4bce-a403-c37c5b02018a\") "
Jul 20 17:04:45.471888 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:45.471862    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfea49cf-4f39-4bce-a403-c37c5b02018a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bfea49cf-4f39-4bce-a403-c37c5b02018a" (UID: "bfea49cf-4f39-4bce-a403-c37c5b02018a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 17:04:45.473222 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:45.473200    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bfea49cf-4f39-4bce-a403-c37c5b02018a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bfea49cf-4f39-4bce-a403-c37c5b02018a" (UID: "bfea49cf-4f39-4bce-a403-c37c5b02018a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:04:45.473389 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:45.473370    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfea49cf-4f39-4bce-a403-c37c5b02018a-kube-api-access-mxnq6" (OuterVolumeSpecName: "kube-api-access-mxnq6") pod "bfea49cf-4f39-4bce-a403-c37c5b02018a" (UID: "bfea49cf-4f39-4bce-a403-c37c5b02018a"). InnerVolumeSpecName "kube-api-access-mxnq6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:04:45.572256 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:45.572232    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxnq6\" (UniqueName: \"kubernetes.io/projected/bfea49cf-4f39-4bce-a403-c37c5b02018a-kube-api-access-mxnq6\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:04:45.572256 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:45.572255    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bfea49cf-4f39-4bce-a403-c37c5b02018a-maas-api-tls\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:04:45.572424 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:45.572265    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfea49cf-4f39-4bce-a403-c37c5b02018a-openshift-service-ca\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:04:46.142481 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:46.142449    2577 generic.go:356] "Generic (PLEG): container finished" podID="bfea49cf-4f39-4bce-a403-c37c5b02018a" containerID="52d28d2def40245994b1a96921bc2ff2d05ff0eec4db574b3f6ad4775c01c620" exitCode=0
Jul 20 17:04:46.142644 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:46.142489    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc"
Jul 20 17:04:46.142644 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:46.142528    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc" event={"ID":"bfea49cf-4f39-4bce-a403-c37c5b02018a","Type":"ContainerDied","Data":"52d28d2def40245994b1a96921bc2ff2d05ff0eec4db574b3f6ad4775c01c620"}
Jul 20 17:04:46.142644 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:46.142577    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc" event={"ID":"bfea49cf-4f39-4bce-a403-c37c5b02018a","Type":"ContainerDied","Data":"0e2088e31f97c88c0e523a4605b21d7efa936aa18fff199dd90b97988a50d840"}
Jul 20 17:04:46.142644 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:46.142600    2577 scope.go:117] "RemoveContainer" containerID="52d28d2def40245994b1a96921bc2ff2d05ff0eec4db574b3f6ad4775c01c620"
Jul 20 17:04:46.154132 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:46.154118    2577 scope.go:117] "RemoveContainer" containerID="52d28d2def40245994b1a96921bc2ff2d05ff0eec4db574b3f6ad4775c01c620"
Jul 20 17:04:46.154366 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:04:46.154347    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52d28d2def40245994b1a96921bc2ff2d05ff0eec4db574b3f6ad4775c01c620\": container with ID starting with 52d28d2def40245994b1a96921bc2ff2d05ff0eec4db574b3f6ad4775c01c620 not found: ID does not exist" containerID="52d28d2def40245994b1a96921bc2ff2d05ff0eec4db574b3f6ad4775c01c620"
Jul 20 17:04:46.154469 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:46.154374    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52d28d2def40245994b1a96921bc2ff2d05ff0eec4db574b3f6ad4775c01c620"} err="failed to get container status \"52d28d2def40245994b1a96921bc2ff2d05ff0eec4db574b3f6ad4775c01c620\": rpc error: code = NotFound desc = could not find container \"52d28d2def40245994b1a96921bc2ff2d05ff0eec4db574b3f6ad4775c01c620\": container with ID starting with 52d28d2def40245994b1a96921bc2ff2d05ff0eec4db574b3f6ad4775c01c620 not found: ID does not exist"
Jul 20 17:04:46.165421 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:46.165399    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc"]
Jul 20 17:04:46.168829 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:46.168809    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-06ad55ec-6dfbc77449-nzhmc"]
Jul 20 17:04:47.211367 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:47.211333    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfea49cf-4f39-4bce-a403-c37c5b02018a" path="/var/lib/kubelet/pods/bfea49cf-4f39-4bce-a403-c37c5b02018a/volumes"
Jul 20 17:04:49.778560 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:49.778528    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:04:52.123668 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:52.123636    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:04:57.103208 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:57.103172    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt"]
Jul 20 17:04:57.148479 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:04:57.148450    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:05:04.111260 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:04.111218    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:05:04.190017 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:04.189991    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:05:12.123283 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:12.123238    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt" podUID="f21cdcce-1126-450e-9a9b-754d67fbeb99" containerName="main" containerID="cri-o://6aaead9be34e7f14e27c4ae22394729b12798769698b6fd7363e4837416d5537" gracePeriod=45
Jul 20 17:05:12.259824 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:12.259799    2577 generic.go:356] "Generic (PLEG): container finished" podID="f21cdcce-1126-450e-9a9b-754d67fbeb99" containerID="6aaead9be34e7f14e27c4ae22394729b12798769698b6fd7363e4837416d5537" exitCode=0
Jul 20 17:05:12.259944 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:12.259871    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt" event={"ID":"f21cdcce-1126-450e-9a9b-754d67fbeb99","Type":"ContainerDied","Data":"6aaead9be34e7f14e27c4ae22394729b12798769698b6fd7363e4837416d5537"}
Jul 20 17:05:12.366550 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:12.366527    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt"
Jul 20 17:05:12.481275 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:12.481198    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f21cdcce-1126-450e-9a9b-754d67fbeb99-tmp-dir\") pod \"f21cdcce-1126-450e-9a9b-754d67fbeb99\" (UID: \"f21cdcce-1126-450e-9a9b-754d67fbeb99\") "
Jul 20 17:05:12.481275 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:12.481251    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f21cdcce-1126-450e-9a9b-754d67fbeb99-home\") pod \"f21cdcce-1126-450e-9a9b-754d67fbeb99\" (UID: \"f21cdcce-1126-450e-9a9b-754d67fbeb99\") "
Jul 20 17:05:12.481275 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:12.481273    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9ckw\" (UniqueName: \"kubernetes.io/projected/f21cdcce-1126-450e-9a9b-754d67fbeb99-kube-api-access-h9ckw\") pod \"f21cdcce-1126-450e-9a9b-754d67fbeb99\" (UID: \"f21cdcce-1126-450e-9a9b-754d67fbeb99\") "
Jul 20 17:05:12.481525 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:12.481310    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f21cdcce-1126-450e-9a9b-754d67fbeb99-dshm\") pod \"f21cdcce-1126-450e-9a9b-754d67fbeb99\" (UID: \"f21cdcce-1126-450e-9a9b-754d67fbeb99\") "
Jul 20 17:05:12.481525 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:12.481357    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f21cdcce-1126-450e-9a9b-754d67fbeb99-model-cache\") pod \"f21cdcce-1126-450e-9a9b-754d67fbeb99\" (UID: \"f21cdcce-1126-450e-9a9b-754d67fbeb99\") "
Jul 20 17:05:12.481525 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:12.481384    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f21cdcce-1126-450e-9a9b-754d67fbeb99-tls-certs\") pod \"f21cdcce-1126-450e-9a9b-754d67fbeb99\" (UID: \"f21cdcce-1126-450e-9a9b-754d67fbeb99\") "
Jul 20 17:05:12.481525 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:12.481473    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f21cdcce-1126-450e-9a9b-754d67fbeb99-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f21cdcce-1126-450e-9a9b-754d67fbeb99" (UID: "f21cdcce-1126-450e-9a9b-754d67fbeb99"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:05:12.481728 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:12.481558    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f21cdcce-1126-450e-9a9b-754d67fbeb99-home" (OuterVolumeSpecName: "home") pod "f21cdcce-1126-450e-9a9b-754d67fbeb99" (UID: "f21cdcce-1126-450e-9a9b-754d67fbeb99"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:05:12.481728 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:12.481676    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f21cdcce-1126-450e-9a9b-754d67fbeb99-model-cache" (OuterVolumeSpecName: "model-cache") pod "f21cdcce-1126-450e-9a9b-754d67fbeb99" (UID: "f21cdcce-1126-450e-9a9b-754d67fbeb99"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:05:12.481728 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:12.481705    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f21cdcce-1126-450e-9a9b-754d67fbeb99-tmp-dir\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:05:12.481728 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:12.481718    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f21cdcce-1126-450e-9a9b-754d67fbeb99-home\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:05:12.483554 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:12.483525    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f21cdcce-1126-450e-9a9b-754d67fbeb99-kube-api-access-h9ckw" (OuterVolumeSpecName: "kube-api-access-h9ckw") pod "f21cdcce-1126-450e-9a9b-754d67fbeb99" (UID: "f21cdcce-1126-450e-9a9b-754d67fbeb99"). InnerVolumeSpecName "kube-api-access-h9ckw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:05:12.483661 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:12.483560    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f21cdcce-1126-450e-9a9b-754d67fbeb99-dshm" (OuterVolumeSpecName: "dshm") pod "f21cdcce-1126-450e-9a9b-754d67fbeb99" (UID: "f21cdcce-1126-450e-9a9b-754d67fbeb99"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:05:12.483661 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:12.483587    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f21cdcce-1126-450e-9a9b-754d67fbeb99-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f21cdcce-1126-450e-9a9b-754d67fbeb99" (UID: "f21cdcce-1126-450e-9a9b-754d67fbeb99"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:05:12.582626 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:12.582598    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f21cdcce-1126-450e-9a9b-754d67fbeb99-model-cache\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:05:12.582626 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:12.582622    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f21cdcce-1126-450e-9a9b-754d67fbeb99-tls-certs\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:05:12.582626 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:12.582631    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9ckw\" (UniqueName: \"kubernetes.io/projected/f21cdcce-1126-450e-9a9b-754d67fbeb99-kube-api-access-h9ckw\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:05:12.582837 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:12.582641    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f21cdcce-1126-450e-9a9b-754d67fbeb99-dshm\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:05:13.266037 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:13.266005    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt" event={"ID":"f21cdcce-1126-450e-9a9b-754d67fbeb99","Type":"ContainerDied","Data":"729e807aadbb3c7ff12963b6c59a860b42761b20280b9ef699e4ffd298fb7b5d"}
Jul 20 17:05:13.266469 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:13.266049    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt"
Jul 20 17:05:13.266469 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:13.266055    2577 scope.go:117] "RemoveContainer" containerID="6aaead9be34e7f14e27c4ae22394729b12798769698b6fd7363e4837416d5537"
Jul 20 17:05:13.283801 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:13.283778    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt"]
Jul 20 17:05:13.290089 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:13.290066    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-06ad55ec/e2e-lifecycle-model-06ad55ec-kserve-7ccd7b4df5-r5zjt"]
Jul 20 17:05:15.211552 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:15.211517    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f21cdcce-1126-450e-9a9b-754d67fbeb99" path="/var/lib/kubelet/pods/f21cdcce-1126-450e-9a9b-754d67fbeb99/volumes"
Jul 20 17:05:22.122089 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:22.122060    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:05:22.223632 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:22.223582    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:05:33.511024 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:33.510941    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:05:34.391975 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:34.391931    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:05:37.329256 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:37.329226    2577 scope.go:117] "RemoveContainer" containerID="1ab2dc1806cd2dc3254b65f8f635e9057ddbed30e6b72525aba6b9627a99635d"
Jul 20 17:05:40.034381 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:40.034349    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:05:40.093180 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:40.093142    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:05:40.294732 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:40.294696    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"]
Jul 20 17:05:40.295283 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:40.295248    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb" podUID="ff7a5a88-925e-488e-b93e-fe9a40e3d8e4" containerName="istio-proxy" containerID="cri-o://02ef08375fdc073c61f8ec492f0c203b9d71d19fd6f9766a542897603329d186" gracePeriod=30
Jul 20 17:05:40.313677 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:40.313654    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:05:40.574614 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:40.574534    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:05:41.607572 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:41.607525    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:05:41.705721 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:41.705684    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:05:41.804406 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:41.804378    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:05:42.523035 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:42.523002    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:05:42.622361 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:42.622325    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:05:42.721747 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:42.721714    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:05:42.982611 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:42.982576    2577 patch_prober.go:28] interesting pod/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 20 17:05:42.982776 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:42.982638    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb" podUID="ff7a5a88-925e-488e-b93e-fe9a40e3d8e4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 20 17:05:44.099933 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:44.099875    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:05:45.541755 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:45.541732    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:05:45.660752 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:45.660657    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-istio-data\") pod \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") "
Jul 20 17:05:45.660752 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:45.660715    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-credential-socket\") pod \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") "
Jul 20 17:05:45.660752 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:45.660732    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-workload-certs\") pod \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") "
Jul 20 17:05:45.660752 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:45.660757    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-istio-token\") pod \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") "
Jul 20 17:05:45.661103 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:45.660785    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-workload-socket\") pod \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") "
Jul 20 17:05:45.661103 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:45.660805    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hsth9\" (UniqueName: \"kubernetes.io/projected/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-kube-api-access-hsth9\") pod \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") "
Jul 20 17:05:45.661103 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:45.660840    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-istio-podinfo\") pod \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") "
Jul 20 17:05:45.661103 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:45.660873    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-istio-envoy\") pod \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") "
Jul 20 17:05:45.661103 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:45.660941    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-istiod-ca-cert\") pod \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\" (UID: \"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4\") "
Jul 20 17:05:45.661103 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:45.660965    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ff7a5a88-925e-488e-b93e-fe9a40e3d8e4" (UID: "ff7a5a88-925e-488e-b93e-fe9a40e3d8e4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:05:45.661446 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:45.661139    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ff7a5a88-925e-488e-b93e-fe9a40e3d8e4" (UID: "ff7a5a88-925e-488e-b93e-fe9a40e3d8e4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:05:45.661446 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:45.661155    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ff7a5a88-925e-488e-b93e-fe9a40e3d8e4" (UID: "ff7a5a88-925e-488e-b93e-fe9a40e3d8e4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:05:45.661446 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:45.661226    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-workload-socket\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:05:45.661446 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:45.661244    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-credential-socket\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:05:45.661446 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:45.661260    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-workload-certs\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:05:45.661688 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:45.661449    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ff7a5a88-925e-488e-b93e-fe9a40e3d8e4" (UID: "ff7a5a88-925e-488e-b93e-fe9a40e3d8e4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 17:05:45.661688 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:45.661586    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-istio-data" (OuterVolumeSpecName: "istio-data") pod "ff7a5a88-925e-488e-b93e-fe9a40e3d8e4" (UID: "ff7a5a88-925e-488e-b93e-fe9a40e3d8e4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:05:45.663119 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:45.663096    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-istio-token" (OuterVolumeSpecName: "istio-token") pod "ff7a5a88-925e-488e-b93e-fe9a40e3d8e4" (UID: "ff7a5a88-925e-488e-b93e-fe9a40e3d8e4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:05:45.663119 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:45.663097    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ff7a5a88-925e-488e-b93e-fe9a40e3d8e4" (UID: "ff7a5a88-925e-488e-b93e-fe9a40e3d8e4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:05:45.663584 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:45.663558    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-kube-api-access-hsth9" (OuterVolumeSpecName: "kube-api-access-hsth9") pod "ff7a5a88-925e-488e-b93e-fe9a40e3d8e4" (UID: "ff7a5a88-925e-488e-b93e-fe9a40e3d8e4"). InnerVolumeSpecName "kube-api-access-hsth9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:05:45.663675 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:45.663566    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ff7a5a88-925e-488e-b93e-fe9a40e3d8e4" (UID: "ff7a5a88-925e-488e-b93e-fe9a40e3d8e4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 20 17:05:45.761862 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:45.761827    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-istiod-ca-cert\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:05:45.761862 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:45.761854    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-istio-data\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:05:45.761862 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:45.761863    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-istio-token\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:05:45.762088 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:45.761872    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hsth9\" (UniqueName: \"kubernetes.io/projected/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-kube-api-access-hsth9\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:05:45.762088 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:45.761880    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-istio-podinfo\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:05:45.762088 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:45.761889    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4-istio-envoy\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:05:46.413332 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:46.413275    2577 generic.go:356] "Generic (PLEG): container finished" podID="ff7a5a88-925e-488e-b93e-fe9a40e3d8e4" containerID="02ef08375fdc073c61f8ec492f0c203b9d71d19fd6f9766a542897603329d186" exitCode=0
Jul 20 17:05:46.413512 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:46.413349    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"
Jul 20 17:05:46.413512 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:46.413358    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb" event={"ID":"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4","Type":"ContainerDied","Data":"02ef08375fdc073c61f8ec492f0c203b9d71d19fd6f9766a542897603329d186"}
Jul 20 17:05:46.413512 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:46.413416    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb" event={"ID":"ff7a5a88-925e-488e-b93e-fe9a40e3d8e4","Type":"ContainerDied","Data":"7d785c87cb2fb639580d07b211007b6582660c7512d6d25ee2ce21e46e86ce5b"}
Jul 20 17:05:46.413512 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:46.413438    2577 scope.go:117] "RemoveContainer" containerID="02ef08375fdc073c61f8ec492f0c203b9d71d19fd6f9766a542897603329d186"
Jul 20 17:05:46.421958 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:46.421857    2577 scope.go:117] "RemoveContainer" containerID="5b0d68027fd82af7b670257959c8f022729342272bcc45dfabfb63b694732096"
Jul 20 17:05:46.429292 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:46.429274    2577 scope.go:117] "RemoveContainer" containerID="02ef08375fdc073c61f8ec492f0c203b9d71d19fd6f9766a542897603329d186"
Jul 20 17:05:46.429536 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:05:46.429517    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02ef08375fdc073c61f8ec492f0c203b9d71d19fd6f9766a542897603329d186\": container with ID starting with 02ef08375fdc073c61f8ec492f0c203b9d71d19fd6f9766a542897603329d186 not found: ID does not exist" containerID="02ef08375fdc073c61f8ec492f0c203b9d71d19fd6f9766a542897603329d186"
Jul 20 17:05:46.429580 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:46.429551    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02ef08375fdc073c61f8ec492f0c203b9d71d19fd6f9766a542897603329d186"} err="failed to get container status \"02ef08375fdc073c61f8ec492f0c203b9d71d19fd6f9766a542897603329d186\": rpc error: code = NotFound desc = could not find container \"02ef08375fdc073c61f8ec492f0c203b9d71d19fd6f9766a542897603329d186\": container with ID starting with 02ef08375fdc073c61f8ec492f0c203b9d71d19fd6f9766a542897603329d186 not found: ID does not exist"
Jul 20 17:05:46.429580 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:46.429572    2577 scope.go:117] "RemoveContainer" containerID="5b0d68027fd82af7b670257959c8f022729342272bcc45dfabfb63b694732096"
Jul 20 17:05:46.429794 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:05:46.429776    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b0d68027fd82af7b670257959c8f022729342272bcc45dfabfb63b694732096\": container with ID starting with 5b0d68027fd82af7b670257959c8f022729342272bcc45dfabfb63b694732096 not found: ID does not exist" containerID="5b0d68027fd82af7b670257959c8f022729342272bcc45dfabfb63b694732096"
Jul 20 17:05:46.429831 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:46.429801    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b0d68027fd82af7b670257959c8f022729342272bcc45dfabfb63b694732096"} err="failed to get container status \"5b0d68027fd82af7b670257959c8f022729342272bcc45dfabfb63b694732096\": rpc error: code = NotFound desc = could not find container \"5b0d68027fd82af7b670257959c8f022729342272bcc45dfabfb63b694732096\": container with ID starting with 5b0d68027fd82af7b670257959c8f022729342272bcc45dfabfb63b694732096 not found: ID does not exist"
Jul 20 17:05:46.436153 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:46.436130    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"]
Jul 20 17:05:46.442956 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:46.442932    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-06ad55ec-openshift-default-88b45c967-6bzzb"]
Jul 20 17:05:47.211527 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:47.211493    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff7a5a88-925e-488e-b93e-fe9a40e3d8e4" path="/var/lib/kubelet/pods/ff7a5a88-925e-488e-b93e-fe9a40e3d8e4/volumes"
Jul 20 17:05:50.075586 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:50.075544    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:05:50.136651 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:50.136619    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:05:50.745877 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:50.745835    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:05:57.008344 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:57.008292    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:05:57.131782 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:57.131748    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:05:57.224749 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:05:57.224714    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:06:07.738939 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:07.738909    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:06:08.077895 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:08.077849    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:06:29.042749 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:29.042716    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:06:35.949519 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:35.949481    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:06:36.048119 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.048089    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:06:36.292996 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.292969    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:06:36.620152 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.620047    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"]
Jul 20 17:06:36.620392 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.620379    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff7a5a88-925e-488e-b93e-fe9a40e3d8e4" containerName="istio-proxy"
Jul 20 17:06:36.620452 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.620394    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff7a5a88-925e-488e-b93e-fe9a40e3d8e4" containerName="istio-proxy"
Jul 20 17:06:36.620452 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.620402    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f21cdcce-1126-450e-9a9b-754d67fbeb99" containerName="main"
Jul 20 17:06:36.620452 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.620409    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f21cdcce-1126-450e-9a9b-754d67fbeb99" containerName="main"
Jul 20 17:06:36.620452 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.620421    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfea49cf-4f39-4bce-a403-c37c5b02018a" containerName="maas-api"
Jul 20 17:06:36.620452 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.620427    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfea49cf-4f39-4bce-a403-c37c5b02018a" containerName="maas-api"
Jul 20 17:06:36.620452 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.620438    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff7a5a88-925e-488e-b93e-fe9a40e3d8e4" containerName="istio-proxy"
Jul 20 17:06:36.620452 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.620443    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff7a5a88-925e-488e-b93e-fe9a40e3d8e4" containerName="istio-proxy"
Jul 20 17:06:36.620689 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.620492    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff7a5a88-925e-488e-b93e-fe9a40e3d8e4" containerName="istio-proxy"
Jul 20 17:06:36.620689 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.620498    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfea49cf-4f39-4bce-a403-c37c5b02018a" containerName="maas-api"
Jul 20 17:06:36.620689 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.620505    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f21cdcce-1126-450e-9a9b-754d67fbeb99" containerName="main"
Jul 20 17:06:36.620689 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.620512    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff7a5a88-925e-488e-b93e-fe9a40e3d8e4" containerName="istio-proxy"
Jul 20 17:06:36.623605 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.623587    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:36.626962 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.626939    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-be281d-openshift-default-dockercfg-d9s8x\""
Jul 20 17:06:36.627411 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.627393    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 20 17:06:36.639240 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.639220    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"]
Jul 20 17:06:36.658641 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.658620    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:06:36.692067 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.692036    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-istiod-ca-cert\") pod \"e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") " pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:36.692161 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.692089    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-workload-socket\") pod \"e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") " pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:36.692209 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.692177    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-workload-certs\") pod \"e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") " pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:36.692209 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.692201    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-istio-envoy\") pod \"e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") " pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:36.692287 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.692219    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-istio-podinfo\") pod \"e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") " pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:36.692287 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.692233    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-istio-token\") pod \"e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") " pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:36.692287 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.692254    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-istio-data\") pod \"e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") " pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:36.692422 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.692354    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-credential-socket\") pod \"e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") " pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:36.692422 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.692375    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv4nj\" (UniqueName: \"kubernetes.io/projected/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-kube-api-access-fv4nj\") pod \"e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") " pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:36.793223 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.793195    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-workload-certs\") pod \"e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") " pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:36.793384 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.793231    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-istio-envoy\") pod \"e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") " pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:36.793384 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.793257    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-istio-podinfo\") pod \"e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") " pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:36.793384 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.793336    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-istio-token\") pod \"e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") " pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:36.793384 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.793371    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-istio-data\") pod \"e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") " pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:36.793587 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.793427    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-credential-socket\") pod \"e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") " pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:36.793587 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.793455    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fv4nj\" (UniqueName: \"kubernetes.io/projected/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-kube-api-access-fv4nj\") pod \"e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") " pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:36.793587 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.793504    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-istiod-ca-cert\") pod \"e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") " pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:36.793587 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.793543    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-workload-socket\") pod \"e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") " pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:36.793786 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.793619    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-workload-certs\") pod \"e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") " pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:36.793786 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.793686    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-istio-data\") pod \"e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") " pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:36.793898 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.793870    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-workload-socket\") pod \"e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") " pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:36.793955 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.793940    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-credential-socket\") pod \"e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") " pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:36.794259 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.794237    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-istiod-ca-cert\") pod \"e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") " pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:36.795503 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.795478    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-istio-envoy\") pod \"e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") " pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:36.795784 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.795766    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-istio-podinfo\") pod \"e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") " pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:36.801652 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.801617    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-istio-token\") pod \"e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") " pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:36.801839 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.801819    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv4nj\" (UniqueName: \"kubernetes.io/projected/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-kube-api-access-fv4nj\") pod \"e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") " pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:36.935113 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:36.935032    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:37.069547 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:37.069519    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"]
Jul 20 17:06:37.071598 ip-10-0-130-85 kubenswrapper[2577]: W0720 17:06:37.071573    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode60b68f6_7c5c_4343_a5af_7fba55a4bab7.slice/crio-232a9db7075cdeeda92aa9984714f6e5f969877c2a8c66c9308ce2cfce265154 WatchSource:0}: Error finding container 232a9db7075cdeeda92aa9984714f6e5f969877c2a8c66c9308ce2cfce265154: Status 404 returned error can't find the container with id 232a9db7075cdeeda92aa9984714f6e5f969877c2a8c66c9308ce2cfce265154
Jul 20 17:06:37.074022 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:37.073987    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:06:37.074122 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:37.074102    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:06:37.074187 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:37.074149    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:06:37.374587 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:37.374559    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:06:37.637588 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:37.637507    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5" event={"ID":"e60b68f6-7c5c-4343-a5af-7fba55a4bab7","Type":"ContainerStarted","Data":"5660ad0944e1eb757011272e60438c594d0b4f07b2c7d9f74b710362b97fd7dc"}
Jul 20 17:06:37.637588 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:37.637562    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5" event={"ID":"e60b68f6-7c5c-4343-a5af-7fba55a4bab7","Type":"ContainerStarted","Data":"232a9db7075cdeeda92aa9984714f6e5f969877c2a8c66c9308ce2cfce265154"}
Jul 20 17:06:37.660548 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:37.660503    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5" podStartSLOduration=1.660487812 podStartE2EDuration="1.660487812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:06:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:06:37.657636052 +0000 UTC m=+2220.907130071" watchObservedRunningTime="2026-07-20 17:06:37.660487812 +0000 UTC m=+2220.909981830"
Jul 20 17:06:37.935678 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:37.935589    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:38.939779 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:38.939751    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:39.589692 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:39.589664    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:06:39.646029 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:39.645998    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:39.646858 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:39.646833    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:39.697257 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:39.697233    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:06:39.873142 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:39.873068    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd"]
Jul 20 17:06:39.877638 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:39.877618    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd"
Jul 20 17:06:39.880523 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:39.880500    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-kxz6w\""
Jul 20 17:06:39.880657 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:39.880552    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-be281d-serving-cert\""
Jul 20 17:06:39.893366 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:39.893343    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd"]
Jul 20 17:06:39.920699 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:39.920660    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:06:39.922795 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:39.922774    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcc58\" (UniqueName: \"kubernetes.io/projected/480852e7-8a9c-4a79-bcba-a761abb45b73-kube-api-access-dcc58\") pod \"maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd\" (UID: \"480852e7-8a9c-4a79-bcba-a761abb45b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd"
Jul 20 17:06:39.922890 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:39.922809    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/480852e7-8a9c-4a79-bcba-a761abb45b73-maas-api-tls\") pod \"maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd\" (UID: \"480852e7-8a9c-4a79-bcba-a761abb45b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd"
Jul 20 17:06:39.922942 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:39.922907    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/480852e7-8a9c-4a79-bcba-a761abb45b73-openshift-service-ca\") pod \"maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd\" (UID: \"480852e7-8a9c-4a79-bcba-a761abb45b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd"
Jul 20 17:06:40.023317 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:40.023276    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/480852e7-8a9c-4a79-bcba-a761abb45b73-openshift-service-ca\") pod \"maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd\" (UID: \"480852e7-8a9c-4a79-bcba-a761abb45b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd"
Jul 20 17:06:40.023678 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:40.023364    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcc58\" (UniqueName: \"kubernetes.io/projected/480852e7-8a9c-4a79-bcba-a761abb45b73-kube-api-access-dcc58\") pod \"maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd\" (UID: \"480852e7-8a9c-4a79-bcba-a761abb45b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd"
Jul 20 17:06:40.023678 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:40.023389    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/480852e7-8a9c-4a79-bcba-a761abb45b73-maas-api-tls\") pod \"maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd\" (UID: \"480852e7-8a9c-4a79-bcba-a761abb45b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd"
Jul 20 17:06:40.023678 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:06:40.023471    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-be281d-serving-cert: secret "maas-api-e2e-inf-a-be281d-serving-cert" not found
Jul 20 17:06:40.023678 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:06:40.023530    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/480852e7-8a9c-4a79-bcba-a761abb45b73-maas-api-tls podName:480852e7-8a9c-4a79-bcba-a761abb45b73 nodeName:}" failed. No retries permitted until 2026-07-20 17:06:40.523514032 +0000 UTC m=+2223.773008032 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/480852e7-8a9c-4a79-bcba-a761abb45b73-maas-api-tls") pod "maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd" (UID: "480852e7-8a9c-4a79-bcba-a761abb45b73") : secret "maas-api-e2e-inf-a-be281d-serving-cert" not found
Jul 20 17:06:40.023966 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:40.023948    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/480852e7-8a9c-4a79-bcba-a761abb45b73-openshift-service-ca\") pod \"maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd\" (UID: \"480852e7-8a9c-4a79-bcba-a761abb45b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd"
Jul 20 17:06:40.033864 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:40.033843    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcc58\" (UniqueName: \"kubernetes.io/projected/480852e7-8a9c-4a79-bcba-a761abb45b73-kube-api-access-dcc58\") pod \"maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd\" (UID: \"480852e7-8a9c-4a79-bcba-a761abb45b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd"
Jul 20 17:06:40.503650 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:40.503614    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:06:40.528087 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:40.528058    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/480852e7-8a9c-4a79-bcba-a761abb45b73-maas-api-tls\") pod \"maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd\" (UID: \"480852e7-8a9c-4a79-bcba-a761abb45b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd"
Jul 20 17:06:40.530521 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:40.530496    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/480852e7-8a9c-4a79-bcba-a761abb45b73-maas-api-tls\") pod \"maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd\" (UID: \"480852e7-8a9c-4a79-bcba-a761abb45b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd"
Jul 20 17:06:40.607006 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:40.606982    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:06:40.791184 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:40.791095    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd"
Jul 20 17:06:40.827193 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:40.827157    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:06:40.919871 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:40.919840    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd"]
Jul 20 17:06:40.923751 ip-10-0-130-85 kubenswrapper[2577]: W0720 17:06:40.923718    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod480852e7_8a9c_4a79_bcba_a761abb45b73.slice/crio-36c36672942b4359c2b65eeeffec9071c52fc5258cd3e2e5752ed18e8ea5a601 WatchSource:0}: Error finding container 36c36672942b4359c2b65eeeffec9071c52fc5258cd3e2e5752ed18e8ea5a601: Status 404 returned error can't find the container with id 36c36672942b4359c2b65eeeffec9071c52fc5258cd3e2e5752ed18e8ea5a601
Jul 20 17:06:41.656911 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:41.656828    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd" event={"ID":"480852e7-8a9c-4a79-bcba-a761abb45b73","Type":"ContainerStarted","Data":"269acf9ad73e042daf5b7c16d77ee56407ac4d4570f9b638f4ff3641554149c2"}
Jul 20 17:06:41.656911 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:41.656887    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd" event={"ID":"480852e7-8a9c-4a79-bcba-a761abb45b73","Type":"ContainerStarted","Data":"36c36672942b4359c2b65eeeffec9071c52fc5258cd3e2e5752ed18e8ea5a601"}
Jul 20 17:06:41.657353 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:41.657014    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd"
Jul 20 17:06:41.675271 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:41.675217    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd" podStartSLOduration=2.227332448 podStartE2EDuration="2.675186092s" totalImagesPullingTime="447.853644ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:06:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:06:41.674189769 +0000 UTC m=+2224.923683788" watchObservedRunningTime="2026-07-20 17:06:41.675186092 +0000 UTC m=+2224.924680113"
Jul 20 17:06:44.084591 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.084547    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:06:44.403809 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.403729    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:06:44.693407 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.693311    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"]
Jul 20 17:06:44.696497 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.696473    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:06:44.699100 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.699053    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-ab00cf-openshift-default-dockercfg-v85m7\""
Jul 20 17:06:44.709380 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.709360    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"]
Jul 20 17:06:44.744731 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.744707    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:06:44.763878 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.763853    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-istiod-ca-cert\") pod \"e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") " pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:06:44.764001 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.763893    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-workload-socket\") pod \"e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") " pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:06:44.764001 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.763913    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-istio-envoy\") pod \"e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") " pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:06:44.764001 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.763961    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87slx\" (UniqueName: \"kubernetes.io/projected/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-kube-api-access-87slx\") pod \"e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") " pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:06:44.764157 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.764016    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-workload-certs\") pod \"e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") " pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:06:44.764157 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.764047    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-istio-podinfo\") pod \"e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") " pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:06:44.764157 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.764101    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-credential-socket\") pod \"e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") " pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:06:44.764157 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.764124    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-istio-data\") pod \"e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") " pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:06:44.764372 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.764166    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-istio-token\") pod \"e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") " pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:06:44.844938 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.844914    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:06:44.865164 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.865139    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-istiod-ca-cert\") pod \"e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") " pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:06:44.865342 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.865174    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-workload-socket\") pod \"e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") " pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:06:44.865342 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.865197    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-istio-envoy\") pod \"e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") " pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:06:44.865342 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.865215    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87slx\" (UniqueName: \"kubernetes.io/projected/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-kube-api-access-87slx\") pod \"e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") " pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:06:44.865520 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.865498    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-workload-certs\") pod \"e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") " pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:06:44.865701 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.865675    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-workload-socket\") pod \"e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") " pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:06:44.865879 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.865831    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-workload-certs\") pod \"e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") " pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:06:44.865995 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.865929    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-istiod-ca-cert\") pod \"e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") " pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:06:44.866132 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.866079    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-istio-podinfo\") pod \"e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") " pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:06:44.866211 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.866177    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-credential-socket\") pod \"e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") " pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:06:44.866281 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.866209    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-istio-data\") pod \"e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") " pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:06:44.866281 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.866234    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-istio-token\") pod \"e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") " pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:06:44.866749 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.866719    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-credential-socket\") pod \"e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") " pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:06:44.866872 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.866754    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-istio-data\") pod \"e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") " pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:06:44.867992 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.867965    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-istio-envoy\") pod \"e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") " pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:06:44.868984 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.868959    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-istio-podinfo\") pod \"e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") " pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:06:44.874135 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.874102    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-istio-token\") pod \"e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") " pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:06:44.874428 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.874408    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87slx\" (UniqueName: \"kubernetes.io/projected/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-kube-api-access-87slx\") pod \"e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") " pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:06:44.947156 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:44.947094    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:06:45.007098 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:45.007075    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:06:45.143473 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:45.143448    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"]
Jul 20 17:06:45.145319 ip-10-0-130-85 kubenswrapper[2577]: W0720 17:06:45.145279    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c8cf702_2c5b_4a83_8d0d_e11139c6a62e.slice/crio-922d299122dca10f7efa9d1f6ce371c506837bf5a54c80b3f27edc8d4bbe637f WatchSource:0}: Error finding container 922d299122dca10f7efa9d1f6ce371c506837bf5a54c80b3f27edc8d4bbe637f: Status 404 returned error can't find the container with id 922d299122dca10f7efa9d1f6ce371c506837bf5a54c80b3f27edc8d4bbe637f
Jul 20 17:06:45.147713 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:45.147679    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:06:45.147814 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:45.147760    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:06:45.147879 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:45.147809    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:06:45.678907 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:45.678871    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd" event={"ID":"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e","Type":"ContainerStarted","Data":"2dab258602e26296fd6522bd47d48961ce1fd850e86876d7ac5cad3901c63a76"}
Jul 20 17:06:45.679093 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:45.678923    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd" event={"ID":"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e","Type":"ContainerStarted","Data":"922d299122dca10f7efa9d1f6ce371c506837bf5a54c80b3f27edc8d4bbe637f"}
Jul 20 17:06:45.701282 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:45.701232    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd" podStartSLOduration=1.701215344 podStartE2EDuration="1.701215344s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:06:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:06:45.699437003 +0000 UTC m=+2228.948931047" watchObservedRunningTime="2026-07-20 17:06:45.701215344 +0000 UTC m=+2228.950709362"
Jul 20 17:06:46.007331 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:46.007226    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:06:46.012100 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:46.012076    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:06:46.683336 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:46.683289    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:06:46.684222 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:46.684204    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:06:47.666607 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:47.666571    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:06:47.668918 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:47.668897    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd"
Jul 20 17:06:47.766943 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:47.766741    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:06:47.849322 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:47.848905    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:06:48.043663 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:48.043627    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:06:48.485980 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:48.485892    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:06:48.658317 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:48.658267    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:06:52.167258 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:52.167228    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:06:52.710959 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:52.710924    2577 generic.go:356] "Generic (PLEG): container finished" podID="e60b68f6-7c5c-4343-a5af-7fba55a4bab7" containerID="5660ad0944e1eb757011272e60438c594d0b4f07b2c7d9f74b710362b97fd7dc" exitCode=0
Jul 20 17:06:52.711119 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:52.710994    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5" event={"ID":"e60b68f6-7c5c-4343-a5af-7fba55a4bab7","Type":"ContainerDied","Data":"5660ad0944e1eb757011272e60438c594d0b4f07b2c7d9f74b710362b97fd7dc"}
Jul 20 17:06:52.711425 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:52.711409    2577 scope.go:117] "RemoveContainer" containerID="5660ad0944e1eb757011272e60438c594d0b4f07b2c7d9f74b710362b97fd7dc"
Jul 20 17:06:52.712430 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:52.712405    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:06:52.712499 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:52.712466    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:06:52.712536 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:52.712501    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:06:53.718159 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:53.718121    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5" event={"ID":"e60b68f6-7c5c-4343-a5af-7fba55a4bab7","Type":"ContainerStarted","Data":"c5457f7809f26a7b95fa5e53a95e67740c2fd9aee0b4b3eac9163818d7a5ed3a"}
Jul 20 17:06:53.935705 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:53.935668    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:53.937002 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:53.936971    2577 patch_prober.go:28] interesting pod/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Jul 20 17:06:53.937246 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:53.937214    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5" podUID="e60b68f6-7c5c-4343-a5af-7fba55a4bab7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Jul 20 17:06:54.646880 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:54.646830    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:54.935750 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:54.935657    2577 patch_prober.go:28] interesting pod/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Jul 20 17:06:54.936708 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:54.936288    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5" podUID="e60b68f6-7c5c-4343-a5af-7fba55a4bab7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Jul 20 17:06:55.936375 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:55.935905    2577 patch_prober.go:28] interesting pod/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Jul 20 17:06:55.936375 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:55.935967    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5" podUID="e60b68f6-7c5c-4343-a5af-7fba55a4bab7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Jul 20 17:06:56.935914 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:56.935876    2577 patch_prober.go:28] interesting pod/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Jul 20 17:06:56.936187 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:56.936159    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5" podUID="e60b68f6-7c5c-4343-a5af-7fba55a4bab7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Jul 20 17:06:57.947914 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:57.947875    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:06:57.956759 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:06:57.956735    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:07:01.684554 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:01.684515    2577 patch_prober.go:28] interesting pod/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 20 17:07:01.684952 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:01.684584    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd" podUID="8c8cf702-2c5b-4a83-8d0d-e11139c6a62e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 20 17:07:01.765047 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:01.765015    2577 generic.go:356] "Generic (PLEG): container finished" podID="8c8cf702-2c5b-4a83-8d0d-e11139c6a62e" containerID="2dab258602e26296fd6522bd47d48961ce1fd850e86876d7ac5cad3901c63a76" exitCode=0
Jul 20 17:07:01.765205 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:01.765092    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd" event={"ID":"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e","Type":"ContainerDied","Data":"2dab258602e26296fd6522bd47d48961ce1fd850e86876d7ac5cad3901c63a76"}
Jul 20 17:07:01.765509 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:01.765495    2577 scope.go:117] "RemoveContainer" containerID="2dab258602e26296fd6522bd47d48961ce1fd850e86876d7ac5cad3901c63a76"
Jul 20 17:07:01.766467 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:01.766436    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:07:01.766531 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:01.766505    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:07:01.766571 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:01.766535    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:07:02.484026 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:02.483988    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:07:02.585388 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:02.585348    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:07:02.771901 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:02.771816    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd" event={"ID":"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e","Type":"ContainerStarted","Data":"68e108990036d9a065b8d57eabf51eedf17e4eec82af2f5983da2f2f2453b9c1"}
Jul 20 17:07:03.007496 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:03.007459    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:07:03.007763 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:03.007740    2577 patch_prober.go:28] interesting pod/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 20 17:07:03.007825 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:03.007803    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd" podUID="8c8cf702-2c5b-4a83-8d0d-e11139c6a62e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 20 17:07:04.007492 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:04.007451    2577 patch_prober.go:28] interesting pod/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 20 17:07:04.007881 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:04.007512    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd" podUID="8c8cf702-2c5b-4a83-8d0d-e11139c6a62e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 20 17:07:04.309580 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:04.309545    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:07:05.008189 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:05.008150    2577 patch_prober.go:28] interesting pod/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 20 17:07:05.008613 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:05.008211    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd" podUID="8c8cf702-2c5b-4a83-8d0d-e11139c6a62e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 20 17:07:06.007922 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:06.007883    2577 patch_prober.go:28] interesting pod/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 20 17:07:06.008094 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:06.007945    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd" podUID="8c8cf702-2c5b-4a83-8d0d-e11139c6a62e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 20 17:07:07.011707 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:07.011676    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:07:07.012172 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:07.012052    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:07:07.012622 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:07.012603    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:07:09.309612 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:09.309573    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:07:09.417306 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:09.417276    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:07:12.398796 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:12.398766    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86"]
Jul 20 17:07:12.401899 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:12.401881    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86"
Jul 20 17:07:12.404675 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:12.404649    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-be281d\"/\"kube-root-ca.crt\""
Jul 20 17:07:12.405771 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:12.405747    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-be281d\"/\"default-dockercfg-v5z7x\""
Jul 20 17:07:12.405771 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:12.405747    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-be281d\"/\"test-model-be281d-kserve-self-signed-certs\""
Jul 20 17:07:12.405920 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:12.405758    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-be281d\"/\"openshift-service-ca.crt\""
Jul 20 17:07:12.410269 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:12.410243    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86"]
Jul 20 17:07:12.422447 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:12.422419    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:07:12.432494 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:12.432469    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce2ac216-f089-407d-ad32-0db36e52f684-tmp-dir\") pod \"test-model-be281d-kserve-7589486645-52c86\" (UID: \"ce2ac216-f089-407d-ad32-0db36e52f684\") " pod="ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86"
Jul 20 17:07:12.432612 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:12.432532    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce2ac216-f089-407d-ad32-0db36e52f684-tls-certs\") pod \"test-model-be281d-kserve-7589486645-52c86\" (UID: \"ce2ac216-f089-407d-ad32-0db36e52f684\") " pod="ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86"
Jul 20 17:07:12.432612 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:12.432581    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce2ac216-f089-407d-ad32-0db36e52f684-model-cache\") pod \"test-model-be281d-kserve-7589486645-52c86\" (UID: \"ce2ac216-f089-407d-ad32-0db36e52f684\") " pod="ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86"
Jul 20 17:07:12.432705 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:12.432625    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce2ac216-f089-407d-ad32-0db36e52f684-home\") pod \"test-model-be281d-kserve-7589486645-52c86\" (UID: \"ce2ac216-f089-407d-ad32-0db36e52f684\") " pod="ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86"
Jul 20 17:07:12.432705 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:12.432660    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce2ac216-f089-407d-ad32-0db36e52f684-dshm\") pod \"test-model-be281d-kserve-7589486645-52c86\" (UID: \"ce2ac216-f089-407d-ad32-0db36e52f684\") " pod="ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86"
Jul 20 17:07:12.432705 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:12.432678    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pc9nz\" (UniqueName: \"kubernetes.io/projected/ce2ac216-f089-407d-ad32-0db36e52f684-kube-api-access-pc9nz\") pod \"test-model-be281d-kserve-7589486645-52c86\" (UID: \"ce2ac216-f089-407d-ad32-0db36e52f684\") " pod="ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86"
Jul 20 17:07:12.533630 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:12.533595    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce2ac216-f089-407d-ad32-0db36e52f684-model-cache\") pod \"test-model-be281d-kserve-7589486645-52c86\" (UID: \"ce2ac216-f089-407d-ad32-0db36e52f684\") " pod="ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86"
Jul 20 17:07:12.533795 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:12.533639    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce2ac216-f089-407d-ad32-0db36e52f684-home\") pod \"test-model-be281d-kserve-7589486645-52c86\" (UID: \"ce2ac216-f089-407d-ad32-0db36e52f684\") " pod="ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86"
Jul 20 17:07:12.533795 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:12.533656    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce2ac216-f089-407d-ad32-0db36e52f684-dshm\") pod \"test-model-be281d-kserve-7589486645-52c86\" (UID: \"ce2ac216-f089-407d-ad32-0db36e52f684\") " pod="ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86"
Jul 20 17:07:12.533795 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:12.533679    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pc9nz\" (UniqueName: \"kubernetes.io/projected/ce2ac216-f089-407d-ad32-0db36e52f684-kube-api-access-pc9nz\") pod \"test-model-be281d-kserve-7589486645-52c86\" (UID: \"ce2ac216-f089-407d-ad32-0db36e52f684\") " pod="ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86"
Jul 20 17:07:12.533795 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:12.533758    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce2ac216-f089-407d-ad32-0db36e52f684-tmp-dir\") pod \"test-model-be281d-kserve-7589486645-52c86\" (UID: \"ce2ac216-f089-407d-ad32-0db36e52f684\") " pod="ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86"
Jul 20 17:07:12.533943 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:12.533810    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce2ac216-f089-407d-ad32-0db36e52f684-tls-certs\") pod \"test-model-be281d-kserve-7589486645-52c86\" (UID: \"ce2ac216-f089-407d-ad32-0db36e52f684\") " pod="ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86"
Jul 20 17:07:12.534080 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:12.534059    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce2ac216-f089-407d-ad32-0db36e52f684-model-cache\") pod \"test-model-be281d-kserve-7589486645-52c86\" (UID: \"ce2ac216-f089-407d-ad32-0db36e52f684\") " pod="ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86"
Jul 20 17:07:12.534142 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:12.534127    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce2ac216-f089-407d-ad32-0db36e52f684-home\") pod \"test-model-be281d-kserve-7589486645-52c86\" (UID: \"ce2ac216-f089-407d-ad32-0db36e52f684\") " pod="ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86"
Jul 20 17:07:12.534233 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:12.534212    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce2ac216-f089-407d-ad32-0db36e52f684-tmp-dir\") pod \"test-model-be281d-kserve-7589486645-52c86\" (UID: \"ce2ac216-f089-407d-ad32-0db36e52f684\") " pod="ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86"
Jul 20 17:07:12.536013 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:12.535991    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce2ac216-f089-407d-ad32-0db36e52f684-dshm\") pod \"test-model-be281d-kserve-7589486645-52c86\" (UID: \"ce2ac216-f089-407d-ad32-0db36e52f684\") " pod="ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86"
Jul 20 17:07:12.536117 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:12.536092    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce2ac216-f089-407d-ad32-0db36e52f684-tls-certs\") pod \"test-model-be281d-kserve-7589486645-52c86\" (UID: \"ce2ac216-f089-407d-ad32-0db36e52f684\") " pod="ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86"
Jul 20 17:07:12.542473 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:12.542449    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pc9nz\" (UniqueName: \"kubernetes.io/projected/ce2ac216-f089-407d-ad32-0db36e52f684-kube-api-access-pc9nz\") pod \"test-model-be281d-kserve-7589486645-52c86\" (UID: \"ce2ac216-f089-407d-ad32-0db36e52f684\") " pod="ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86"
Jul 20 17:07:12.711527 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:12.711450    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86"
Jul 20 17:07:12.846092 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:12.846068    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86"]
Jul 20 17:07:12.848056 ip-10-0-130-85 kubenswrapper[2577]: W0720 17:07:12.848023    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podce2ac216_f089_407d_ad32_0db36e52f684.slice/crio-814df98e9cefbb70a73a0b7b26f19278f391fd74b611705e6a111e47242f39cd WatchSource:0}: Error finding container 814df98e9cefbb70a73a0b7b26f19278f391fd74b611705e6a111e47242f39cd: Status 404 returned error can't find the container with id 814df98e9cefbb70a73a0b7b26f19278f391fd74b611705e6a111e47242f39cd
Jul 20 17:07:13.822017 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:13.821984    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86" event={"ID":"ce2ac216-f089-407d-ad32-0db36e52f684","Type":"ContainerStarted","Data":"7a1716ddabe1c05b5f12bf855b2d00e0b104d47f27b8fbed9156c42edecef2ec"}
Jul 20 17:07:13.822017 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:13.822022    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86" event={"ID":"ce2ac216-f089-407d-ad32-0db36e52f684","Type":"ContainerStarted","Data":"814df98e9cefbb70a73a0b7b26f19278f391fd74b611705e6a111e47242f39cd"}
Jul 20 17:07:13.843858 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:13.843815    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86" podStartSLOduration=1.843800553 podStartE2EDuration="1.843800553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:07:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:07:13.840344758 +0000 UTC m=+2257.089838776" watchObservedRunningTime="2026-07-20 17:07:13.843800553 +0000 UTC m=+2257.093294571"
Jul 20 17:07:22.712099 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:22.712061    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86"
Jul 20 17:07:22.712625 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:22.712112    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86"
Jul 20 17:07:22.724688 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:22.724662    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86"
Jul 20 17:07:22.872802 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:22.872769    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86"
Jul 20 17:07:42.185031 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:07:42.184997    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:08:03.012066 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:03.012034    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:08:04.333561 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:04.333522    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:08:22.149609 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:22.149577    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86"]
Jul 20 17:08:22.171424 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:22.171384    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:08:23.151661 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:23.151620    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:08:24.044125 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:24.044095    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:08:24.143573 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:24.143540    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:08:33.678797 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:33.678711    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd"]
Jul 20 17:08:33.679332 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:33.678955    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd" podUID="480852e7-8a9c-4a79-bcba-a761abb45b73" containerName="maas-api" containerID="cri-o://269acf9ad73e042daf5b7c16d77ee56407ac4d4570f9b638f4ff3641554149c2" gracePeriod=30
Jul 20 17:08:33.716699 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:33.716675    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:08:34.119256 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:34.119234    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd"
Jul 20 17:08:34.179900 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:34.179870    2577 generic.go:356] "Generic (PLEG): container finished" podID="480852e7-8a9c-4a79-bcba-a761abb45b73" containerID="269acf9ad73e042daf5b7c16d77ee56407ac4d4570f9b638f4ff3641554149c2" exitCode=0
Jul 20 17:08:34.180046 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:34.179911    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd"
Jul 20 17:08:34.180046 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:34.179951    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd" event={"ID":"480852e7-8a9c-4a79-bcba-a761abb45b73","Type":"ContainerDied","Data":"269acf9ad73e042daf5b7c16d77ee56407ac4d4570f9b638f4ff3641554149c2"}
Jul 20 17:08:34.180046 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:34.179998    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd" event={"ID":"480852e7-8a9c-4a79-bcba-a761abb45b73","Type":"ContainerDied","Data":"36c36672942b4359c2b65eeeffec9071c52fc5258cd3e2e5752ed18e8ea5a601"}
Jul 20 17:08:34.180046 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:34.180021    2577 scope.go:117] "RemoveContainer" containerID="269acf9ad73e042daf5b7c16d77ee56407ac4d4570f9b638f4ff3641554149c2"
Jul 20 17:08:34.189327 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:34.189290    2577 scope.go:117] "RemoveContainer" containerID="269acf9ad73e042daf5b7c16d77ee56407ac4d4570f9b638f4ff3641554149c2"
Jul 20 17:08:34.189577 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:08:34.189559    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"269acf9ad73e042daf5b7c16d77ee56407ac4d4570f9b638f4ff3641554149c2\": container with ID starting with 269acf9ad73e042daf5b7c16d77ee56407ac4d4570f9b638f4ff3641554149c2 not found: ID does not exist" containerID="269acf9ad73e042daf5b7c16d77ee56407ac4d4570f9b638f4ff3641554149c2"
Jul 20 17:08:34.189624 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:34.189586    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"269acf9ad73e042daf5b7c16d77ee56407ac4d4570f9b638f4ff3641554149c2"} err="failed to get container status \"269acf9ad73e042daf5b7c16d77ee56407ac4d4570f9b638f4ff3641554149c2\": rpc error: code = NotFound desc = could not find container \"269acf9ad73e042daf5b7c16d77ee56407ac4d4570f9b638f4ff3641554149c2\": container with ID starting with 269acf9ad73e042daf5b7c16d77ee56407ac4d4570f9b638f4ff3641554149c2 not found: ID does not exist"
Jul 20 17:08:34.213556 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:34.213500    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/480852e7-8a9c-4a79-bcba-a761abb45b73-maas-api-tls\") pod \"480852e7-8a9c-4a79-bcba-a761abb45b73\" (UID: \"480852e7-8a9c-4a79-bcba-a761abb45b73\") "
Jul 20 17:08:34.213655 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:34.213610    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dcc58\" (UniqueName: \"kubernetes.io/projected/480852e7-8a9c-4a79-bcba-a761abb45b73-kube-api-access-dcc58\") pod \"480852e7-8a9c-4a79-bcba-a761abb45b73\" (UID: \"480852e7-8a9c-4a79-bcba-a761abb45b73\") "
Jul 20 17:08:34.213655 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:34.213645    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/480852e7-8a9c-4a79-bcba-a761abb45b73-openshift-service-ca\") pod \"480852e7-8a9c-4a79-bcba-a761abb45b73\" (UID: \"480852e7-8a9c-4a79-bcba-a761abb45b73\") "
Jul 20 17:08:34.213983 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:34.213962    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/480852e7-8a9c-4a79-bcba-a761abb45b73-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "480852e7-8a9c-4a79-bcba-a761abb45b73" (UID: "480852e7-8a9c-4a79-bcba-a761abb45b73"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 17:08:34.215434 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:34.215408    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/480852e7-8a9c-4a79-bcba-a761abb45b73-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "480852e7-8a9c-4a79-bcba-a761abb45b73" (UID: "480852e7-8a9c-4a79-bcba-a761abb45b73"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:08:34.215569 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:34.215549    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/480852e7-8a9c-4a79-bcba-a761abb45b73-kube-api-access-dcc58" (OuterVolumeSpecName: "kube-api-access-dcc58") pod "480852e7-8a9c-4a79-bcba-a761abb45b73" (UID: "480852e7-8a9c-4a79-bcba-a761abb45b73"). InnerVolumeSpecName "kube-api-access-dcc58". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:08:34.320672 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:34.320604    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/480852e7-8a9c-4a79-bcba-a761abb45b73-maas-api-tls\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:08:34.320672 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:34.320646    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dcc58\" (UniqueName: \"kubernetes.io/projected/480852e7-8a9c-4a79-bcba-a761abb45b73-kube-api-access-dcc58\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:08:34.320929 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:34.320900    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/480852e7-8a9c-4a79-bcba-a761abb45b73-openshift-service-ca\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:08:34.501586 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:34.501559    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd"]
Jul 20 17:08:34.507034 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:34.507012    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-be281d-84f55bb5f4-xkfrd"]
Jul 20 17:08:35.211375 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:35.211339    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="480852e7-8a9c-4a79-bcba-a761abb45b73" path="/var/lib/kubelet/pods/480852e7-8a9c-4a79-bcba-a761abb45b73/volumes"
Jul 20 17:08:37.170599 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:37.170540    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86" podUID="ce2ac216-f089-407d-ad32-0db36e52f684" containerName="main" containerID="cri-o://7a1716ddabe1c05b5f12bf855b2d00e0b104d47f27b8fbed9156c42edecef2ec" gracePeriod=45
Jul 20 17:08:37.418078 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:37.418058    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86"
Jul 20 17:08:37.549964 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:37.549928    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce2ac216-f089-407d-ad32-0db36e52f684-home\") pod \"ce2ac216-f089-407d-ad32-0db36e52f684\" (UID: \"ce2ac216-f089-407d-ad32-0db36e52f684\") "
Jul 20 17:08:37.550126 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:37.549978    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce2ac216-f089-407d-ad32-0db36e52f684-tls-certs\") pod \"ce2ac216-f089-407d-ad32-0db36e52f684\" (UID: \"ce2ac216-f089-407d-ad32-0db36e52f684\") "
Jul 20 17:08:37.550126 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:37.550004    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce2ac216-f089-407d-ad32-0db36e52f684-tmp-dir\") pod \"ce2ac216-f089-407d-ad32-0db36e52f684\" (UID: \"ce2ac216-f089-407d-ad32-0db36e52f684\") "
Jul 20 17:08:37.550126 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:37.550032    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce2ac216-f089-407d-ad32-0db36e52f684-model-cache\") pod \"ce2ac216-f089-407d-ad32-0db36e52f684\" (UID: \"ce2ac216-f089-407d-ad32-0db36e52f684\") "
Jul 20 17:08:37.550126 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:37.550054    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce2ac216-f089-407d-ad32-0db36e52f684-dshm\") pod \"ce2ac216-f089-407d-ad32-0db36e52f684\" (UID: \"ce2ac216-f089-407d-ad32-0db36e52f684\") "
Jul 20 17:08:37.550346 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:37.550213    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pc9nz\" (UniqueName: \"kubernetes.io/projected/ce2ac216-f089-407d-ad32-0db36e52f684-kube-api-access-pc9nz\") pod \"ce2ac216-f089-407d-ad32-0db36e52f684\" (UID: \"ce2ac216-f089-407d-ad32-0db36e52f684\") "
Jul 20 17:08:37.550346 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:37.550269    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce2ac216-f089-407d-ad32-0db36e52f684-home" (OuterVolumeSpecName: "home") pod "ce2ac216-f089-407d-ad32-0db36e52f684" (UID: "ce2ac216-f089-407d-ad32-0db36e52f684"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:08:37.550346 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:37.550321    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce2ac216-f089-407d-ad32-0db36e52f684-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ce2ac216-f089-407d-ad32-0db36e52f684" (UID: "ce2ac216-f089-407d-ad32-0db36e52f684"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:08:37.550518 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:37.550353    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce2ac216-f089-407d-ad32-0db36e52f684-model-cache" (OuterVolumeSpecName: "model-cache") pod "ce2ac216-f089-407d-ad32-0db36e52f684" (UID: "ce2ac216-f089-407d-ad32-0db36e52f684"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:08:37.550563 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:37.550517    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce2ac216-f089-407d-ad32-0db36e52f684-home\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:08:37.550563 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:37.550540    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce2ac216-f089-407d-ad32-0db36e52f684-tmp-dir\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:08:37.550563 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:37.550559    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce2ac216-f089-407d-ad32-0db36e52f684-model-cache\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:08:37.552372 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:37.552341    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce2ac216-f089-407d-ad32-0db36e52f684-dshm" (OuterVolumeSpecName: "dshm") pod "ce2ac216-f089-407d-ad32-0db36e52f684" (UID: "ce2ac216-f089-407d-ad32-0db36e52f684"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:08:37.552372 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:37.552345    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce2ac216-f089-407d-ad32-0db36e52f684-kube-api-access-pc9nz" (OuterVolumeSpecName: "kube-api-access-pc9nz") pod "ce2ac216-f089-407d-ad32-0db36e52f684" (UID: "ce2ac216-f089-407d-ad32-0db36e52f684"). InnerVolumeSpecName "kube-api-access-pc9nz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:08:37.552520 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:37.552399    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce2ac216-f089-407d-ad32-0db36e52f684-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ce2ac216-f089-407d-ad32-0db36e52f684" (UID: "ce2ac216-f089-407d-ad32-0db36e52f684"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:08:37.651330 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:37.651289    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pc9nz\" (UniqueName: \"kubernetes.io/projected/ce2ac216-f089-407d-ad32-0db36e52f684-kube-api-access-pc9nz\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:08:37.651445 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:37.651332    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce2ac216-f089-407d-ad32-0db36e52f684-tls-certs\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:08:37.651445 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:37.651344    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce2ac216-f089-407d-ad32-0db36e52f684-dshm\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:08:38.199087 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:38.199049    2577 generic.go:356] "Generic (PLEG): container finished" podID="ce2ac216-f089-407d-ad32-0db36e52f684" containerID="7a1716ddabe1c05b5f12bf855b2d00e0b104d47f27b8fbed9156c42edecef2ec" exitCode=0
Jul 20 17:08:38.199575 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:38.199093    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86"
Jul 20 17:08:38.199575 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:38.199101    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86" event={"ID":"ce2ac216-f089-407d-ad32-0db36e52f684","Type":"ContainerDied","Data":"7a1716ddabe1c05b5f12bf855b2d00e0b104d47f27b8fbed9156c42edecef2ec"}
Jul 20 17:08:38.199575 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:38.199139    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86" event={"ID":"ce2ac216-f089-407d-ad32-0db36e52f684","Type":"ContainerDied","Data":"814df98e9cefbb70a73a0b7b26f19278f391fd74b611705e6a111e47242f39cd"}
Jul 20 17:08:38.199575 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:38.199160    2577 scope.go:117] "RemoveContainer" containerID="7a1716ddabe1c05b5f12bf855b2d00e0b104d47f27b8fbed9156c42edecef2ec"
Jul 20 17:08:38.208067 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:38.208047    2577 scope.go:117] "RemoveContainer" containerID="7a1716ddabe1c05b5f12bf855b2d00e0b104d47f27b8fbed9156c42edecef2ec"
Jul 20 17:08:38.208323 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:08:38.208289    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a1716ddabe1c05b5f12bf855b2d00e0b104d47f27b8fbed9156c42edecef2ec\": container with ID starting with 7a1716ddabe1c05b5f12bf855b2d00e0b104d47f27b8fbed9156c42edecef2ec not found: ID does not exist" containerID="7a1716ddabe1c05b5f12bf855b2d00e0b104d47f27b8fbed9156c42edecef2ec"
Jul 20 17:08:38.208388 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:38.208336    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a1716ddabe1c05b5f12bf855b2d00e0b104d47f27b8fbed9156c42edecef2ec"} err="failed to get container status \"7a1716ddabe1c05b5f12bf855b2d00e0b104d47f27b8fbed9156c42edecef2ec\": rpc error: code = NotFound desc = could not find container \"7a1716ddabe1c05b5f12bf855b2d00e0b104d47f27b8fbed9156c42edecef2ec\": container with ID starting with 7a1716ddabe1c05b5f12bf855b2d00e0b104d47f27b8fbed9156c42edecef2ec not found: ID does not exist"
Jul 20 17:08:38.222421 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:38.222392    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86"]
Jul 20 17:08:38.228484 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:38.228463    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-be281d/test-model-be281d-kserve-7589486645-52c86"]
Jul 20 17:08:38.334863 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:38.334818    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:08:38.739028 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:38.738956    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:08:39.214177 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:39.214144    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce2ac216-f089-407d-ad32-0db36e52f684" path="/var/lib/kubelet/pods/ce2ac216-f089-407d-ad32-0db36e52f684/volumes"
Jul 20 17:08:44.649023 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:44.648975    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:08:44.748422 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:44.748392    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:08:44.880404 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:44.880373    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"]
Jul 20 17:08:44.880613 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:44.880576    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5" podUID="e60b68f6-7c5c-4343-a5af-7fba55a4bab7" containerName="istio-proxy" containerID="cri-o://c5457f7809f26a7b95fa5e53a95e67740c2fd9aee0b4b3eac9163818d7a5ed3a" gracePeriod=30
Jul 20 17:08:44.905556 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:44.905477    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:08:45.135834 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:45.135758    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:08:45.793876 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:45.793839    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:08:45.895806 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:45.895772    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:08:50.132560 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.132538    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:08:50.154921 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.154866    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-istio-podinfo\") pod \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") "
Jul 20 17:08:50.154921 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.154920    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-istiod-ca-cert\") pod \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") "
Jul 20 17:08:50.155054 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.154945    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-istio-data\") pod \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") "
Jul 20 17:08:50.155054 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.154982    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-workload-socket\") pod \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") "
Jul 20 17:08:50.155054 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.155019    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-istio-envoy\") pod \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") "
Jul 20 17:08:50.155054 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.155047    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-credential-socket\") pod \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") "
Jul 20 17:08:50.155232 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.155078    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-istio-token\") pod \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") "
Jul 20 17:08:50.155232 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.155103    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fv4nj\" (UniqueName: \"kubernetes.io/projected/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-kube-api-access-fv4nj\") pod \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") "
Jul 20 17:08:50.155232 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.155128    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-workload-certs\") pod \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\" (UID: \"e60b68f6-7c5c-4343-a5af-7fba55a4bab7\") "
Jul 20 17:08:50.155393 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.155323    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e60b68f6-7c5c-4343-a5af-7fba55a4bab7" (UID: "e60b68f6-7c5c-4343-a5af-7fba55a4bab7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:08:50.155467 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.155385    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e60b68f6-7c5c-4343-a5af-7fba55a4bab7" (UID: "e60b68f6-7c5c-4343-a5af-7fba55a4bab7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:08:50.155541 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.155499    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-workload-socket\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:08:50.155541 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.155519    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-credential-socket\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:08:50.155541 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.155515    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e60b68f6-7c5c-4343-a5af-7fba55a4bab7" (UID: "e60b68f6-7c5c-4343-a5af-7fba55a4bab7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:08:50.155740 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.155696    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e60b68f6-7c5c-4343-a5af-7fba55a4bab7" (UID: "e60b68f6-7c5c-4343-a5af-7fba55a4bab7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 17:08:50.156380 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.155872    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-istio-data" (OuterVolumeSpecName: "istio-data") pod "e60b68f6-7c5c-4343-a5af-7fba55a4bab7" (UID: "e60b68f6-7c5c-4343-a5af-7fba55a4bab7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:08:50.158619 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.158597    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e60b68f6-7c5c-4343-a5af-7fba55a4bab7" (UID: "e60b68f6-7c5c-4343-a5af-7fba55a4bab7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 20 17:08:50.159396 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.159366    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-istio-token" (OuterVolumeSpecName: "istio-token") pod "e60b68f6-7c5c-4343-a5af-7fba55a4bab7" (UID: "e60b68f6-7c5c-4343-a5af-7fba55a4bab7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:08:50.159396 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.159376    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e60b68f6-7c5c-4343-a5af-7fba55a4bab7" (UID: "e60b68f6-7c5c-4343-a5af-7fba55a4bab7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:08:50.159559 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.159453    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-kube-api-access-fv4nj" (OuterVolumeSpecName: "kube-api-access-fv4nj") pod "e60b68f6-7c5c-4343-a5af-7fba55a4bab7" (UID: "e60b68f6-7c5c-4343-a5af-7fba55a4bab7"). InnerVolumeSpecName "kube-api-access-fv4nj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:08:50.256217 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.256098    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-istio-envoy\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:08:50.256217 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.256127    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-istio-token\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:08:50.256217 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.256142    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fv4nj\" (UniqueName: \"kubernetes.io/projected/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-kube-api-access-fv4nj\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:08:50.256217 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.256154    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-workload-certs\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:08:50.256217 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.256168    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-istio-podinfo\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:08:50.256217 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.256180    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-istiod-ca-cert\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:08:50.256217 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.256194    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e60b68f6-7c5c-4343-a5af-7fba55a4bab7-istio-data\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:08:50.257439 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.257412    2577 generic.go:356] "Generic (PLEG): container finished" podID="e60b68f6-7c5c-4343-a5af-7fba55a4bab7" containerID="c5457f7809f26a7b95fa5e53a95e67740c2fd9aee0b4b3eac9163818d7a5ed3a" exitCode=0
Jul 20 17:08:50.257566 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.257461    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"
Jul 20 17:08:50.257566 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.257462    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5" event={"ID":"e60b68f6-7c5c-4343-a5af-7fba55a4bab7","Type":"ContainerDied","Data":"c5457f7809f26a7b95fa5e53a95e67740c2fd9aee0b4b3eac9163818d7a5ed3a"}
Jul 20 17:08:50.257669 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.257597    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5" event={"ID":"e60b68f6-7c5c-4343-a5af-7fba55a4bab7","Type":"ContainerDied","Data":"232a9db7075cdeeda92aa9984714f6e5f969877c2a8c66c9308ce2cfce265154"}
Jul 20 17:08:50.257669 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.257627    2577 scope.go:117] "RemoveContainer" containerID="c5457f7809f26a7b95fa5e53a95e67740c2fd9aee0b4b3eac9163818d7a5ed3a"
Jul 20 17:08:50.265982 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.265958    2577 scope.go:117] "RemoveContainer" containerID="5660ad0944e1eb757011272e60438c594d0b4f07b2c7d9f74b710362b97fd7dc"
Jul 20 17:08:50.273313 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.273284    2577 scope.go:117] "RemoveContainer" containerID="c5457f7809f26a7b95fa5e53a95e67740c2fd9aee0b4b3eac9163818d7a5ed3a"
Jul 20 17:08:50.273565 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:08:50.273545    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5457f7809f26a7b95fa5e53a95e67740c2fd9aee0b4b3eac9163818d7a5ed3a\": container with ID starting with c5457f7809f26a7b95fa5e53a95e67740c2fd9aee0b4b3eac9163818d7a5ed3a not found: ID does not exist" containerID="c5457f7809f26a7b95fa5e53a95e67740c2fd9aee0b4b3eac9163818d7a5ed3a"
Jul 20 17:08:50.273614 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.273572    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5457f7809f26a7b95fa5e53a95e67740c2fd9aee0b4b3eac9163818d7a5ed3a"} err="failed to get container status \"c5457f7809f26a7b95fa5e53a95e67740c2fd9aee0b4b3eac9163818d7a5ed3a\": rpc error: code = NotFound desc = could not find container \"c5457f7809f26a7b95fa5e53a95e67740c2fd9aee0b4b3eac9163818d7a5ed3a\": container with ID starting with c5457f7809f26a7b95fa5e53a95e67740c2fd9aee0b4b3eac9163818d7a5ed3a not found: ID does not exist"
Jul 20 17:08:50.273614 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.273589    2577 scope.go:117] "RemoveContainer" containerID="5660ad0944e1eb757011272e60438c594d0b4f07b2c7d9f74b710362b97fd7dc"
Jul 20 17:08:50.273787 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:08:50.273772    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5660ad0944e1eb757011272e60438c594d0b4f07b2c7d9f74b710362b97fd7dc\": container with ID starting with 5660ad0944e1eb757011272e60438c594d0b4f07b2c7d9f74b710362b97fd7dc not found: ID does not exist" containerID="5660ad0944e1eb757011272e60438c594d0b4f07b2c7d9f74b710362b97fd7dc"
Jul 20 17:08:50.273823 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.273791    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5660ad0944e1eb757011272e60438c594d0b4f07b2c7d9f74b710362b97fd7dc"} err="failed to get container status \"5660ad0944e1eb757011272e60438c594d0b4f07b2c7d9f74b710362b97fd7dc\": rpc error: code = NotFound desc = could not find container \"5660ad0944e1eb757011272e60438c594d0b4f07b2c7d9f74b710362b97fd7dc\": container with ID starting with 5660ad0944e1eb757011272e60438c594d0b4f07b2c7d9f74b710362b97fd7dc not found: ID does not exist"
Jul 20 17:08:50.279928 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.279907    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"]
Jul 20 17:08:50.287134 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:50.287114    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-be281d-openshift-default-77c8dcb649-qvtn5"]
Jul 20 17:08:51.211387 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:51.211353    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e60b68f6-7c5c-4343-a5af-7fba55a4bab7" path="/var/lib/kubelet/pods/e60b68f6-7c5c-4343-a5af-7fba55a4bab7/volumes"
Jul 20 17:08:55.844206 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:08:55.844172    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:00.345157 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:00.345120    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:00.751142 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:00.751068    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:06.834844 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:06.834789    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:06.910280 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:06.910246    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:07.102174 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:07.102092    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"]
Jul 20 17:09:07.102403 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:07.102371    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd" podUID="8c8cf702-2c5b-4a83-8d0d-e11139c6a62e" containerName="istio-proxy" containerID="cri-o://68e108990036d9a065b8d57eabf51eedf17e4eec82af2f5983da2f2f2453b9c1" gracePeriod=30
Jul 20 17:09:07.126909 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:07.126885    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:07.378712 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:07.378619    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:07.710920 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:07.710850    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:07.994230 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:07.994152    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"]
Jul 20 17:09:07.994606 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:07.994485    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="480852e7-8a9c-4a79-bcba-a761abb45b73" containerName="maas-api"
Jul 20 17:09:07.994606 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:07.994497    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="480852e7-8a9c-4a79-bcba-a761abb45b73" containerName="maas-api"
Jul 20 17:09:07.994606 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:07.994509    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce2ac216-f089-407d-ad32-0db36e52f684" containerName="main"
Jul 20 17:09:07.994606 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:07.994515    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce2ac216-f089-407d-ad32-0db36e52f684" containerName="main"
Jul 20 17:09:07.994606 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:07.994522    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e60b68f6-7c5c-4343-a5af-7fba55a4bab7" containerName="istio-proxy"
Jul 20 17:09:07.994606 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:07.994528    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e60b68f6-7c5c-4343-a5af-7fba55a4bab7" containerName="istio-proxy"
Jul 20 17:09:07.994606 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:07.994580    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e60b68f6-7c5c-4343-a5af-7fba55a4bab7" containerName="istio-proxy"
Jul 20 17:09:07.994606 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:07.994588    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="480852e7-8a9c-4a79-bcba-a761abb45b73" containerName="maas-api"
Jul 20 17:09:07.994606 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:07.994596    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce2ac216-f089-407d-ad32-0db36e52f684" containerName="main"
Jul 20 17:09:07.998920 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:07.998901    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:08.001486 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.001465    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-cdcb96-openshift-default-dockercfg-8chs4\""
Jul 20 17:09:08.014601 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.014580    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"]
Jul 20 17:09:08.022999 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.022982    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:08.098949 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.098920    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8138c4ee-2a73-4764-bb51-aa7d56b536fd-istio-data\") pod \"e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") " pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:08.099095 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.098965    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8138c4ee-2a73-4764-bb51-aa7d56b536fd-istiod-ca-cert\") pod \"e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") " pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:08.099095 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.098990    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8138c4ee-2a73-4764-bb51-aa7d56b536fd-istio-podinfo\") pod \"e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") " pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:08.099095 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.099079    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8138c4ee-2a73-4764-bb51-aa7d56b536fd-workload-socket\") pod \"e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") " pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:08.099226 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.099118    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7fb6\" (UniqueName: \"kubernetes.io/projected/8138c4ee-2a73-4764-bb51-aa7d56b536fd-kube-api-access-q7fb6\") pod \"e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") " pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:08.099226 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.099145    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8138c4ee-2a73-4764-bb51-aa7d56b536fd-istio-envoy\") pod \"e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") " pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:08.099226 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.099207    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8138c4ee-2a73-4764-bb51-aa7d56b536fd-istio-token\") pod \"e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") " pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:08.099368 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.099253    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8138c4ee-2a73-4764-bb51-aa7d56b536fd-workload-certs\") pod \"e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") " pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:08.099368 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.099289    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8138c4ee-2a73-4764-bb51-aa7d56b536fd-credential-socket\") pod \"e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") " pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:08.199951 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.199924    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8138c4ee-2a73-4764-bb51-aa7d56b536fd-workload-certs\") pod \"e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") " pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:08.200093 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.199959    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8138c4ee-2a73-4764-bb51-aa7d56b536fd-credential-socket\") pod \"e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") " pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:08.200093 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.200082    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8138c4ee-2a73-4764-bb51-aa7d56b536fd-istio-data\") pod \"e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") " pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:08.200217 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.200134    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8138c4ee-2a73-4764-bb51-aa7d56b536fd-istiod-ca-cert\") pod \"e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") " pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:08.200217 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.200171    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8138c4ee-2a73-4764-bb51-aa7d56b536fd-istio-podinfo\") pod \"e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") " pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:08.200217 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.200198    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8138c4ee-2a73-4764-bb51-aa7d56b536fd-workload-socket\") pod \"e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") " pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:08.200416 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.200239    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7fb6\" (UniqueName: \"kubernetes.io/projected/8138c4ee-2a73-4764-bb51-aa7d56b536fd-kube-api-access-q7fb6\") pod \"e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") " pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:08.200416 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.200269    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8138c4ee-2a73-4764-bb51-aa7d56b536fd-istio-envoy\") pod \"e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") " pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:08.200416 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.200319    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8138c4ee-2a73-4764-bb51-aa7d56b536fd-istio-token\") pod \"e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") " pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:08.200416 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.200405    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8138c4ee-2a73-4764-bb51-aa7d56b536fd-credential-socket\") pod \"e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") " pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:08.200613 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.200442    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8138c4ee-2a73-4764-bb51-aa7d56b536fd-workload-certs\") pod \"e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") " pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:08.200613 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.200563    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8138c4ee-2a73-4764-bb51-aa7d56b536fd-istio-data\") pod \"e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") " pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:08.200834 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.200815    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8138c4ee-2a73-4764-bb51-aa7d56b536fd-workload-socket\") pod \"e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") " pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:08.201030 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.201004    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8138c4ee-2a73-4764-bb51-aa7d56b536fd-istiod-ca-cert\") pod \"e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") " pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:08.202637 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.202614    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8138c4ee-2a73-4764-bb51-aa7d56b536fd-istio-envoy\") pod \"e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") " pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:08.202723 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.202689    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8138c4ee-2a73-4764-bb51-aa7d56b536fd-istio-podinfo\") pod \"e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") " pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:08.208124 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.208102    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8138c4ee-2a73-4764-bb51-aa7d56b536fd-istio-token\") pod \"e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") " pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:08.208232 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.208208    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7fb6\" (UniqueName: \"kubernetes.io/projected/8138c4ee-2a73-4764-bb51-aa7d56b536fd-kube-api-access-q7fb6\") pod \"e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") " pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:08.309370 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.309349    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:08.451083 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.451052    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:08.451659 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.451637    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"]
Jul 20 17:09:08.453886 ip-10-0-130-85 kubenswrapper[2577]: W0720 17:09:08.453858    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8138c4ee_2a73_4764_bb51_aa7d56b536fd.slice/crio-5f7974ba115e4212d205e5459a88b7bcec89a7ff83fe77b260dd1567c9374724 WatchSource:0}: Error finding container 5f7974ba115e4212d205e5459a88b7bcec89a7ff83fe77b260dd1567c9374724: Status 404 returned error can't find the container with id 5f7974ba115e4212d205e5459a88b7bcec89a7ff83fe77b260dd1567c9374724
Jul 20 17:09:08.456196 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.456168    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:09:08.456257 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.456228    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:09:08.456319 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:08.456255    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:09:09.341221 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:09.341189    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v" event={"ID":"8138c4ee-2a73-4764-bb51-aa7d56b536fd","Type":"ContainerStarted","Data":"3da5cf74e7bcc1e33dd83a63b7d798e958246a57bdcd0b4ecc31c3d6f12c20a3"}
Jul 20 17:09:09.341619 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:09.341236    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v" event={"ID":"8138c4ee-2a73-4764-bb51-aa7d56b536fd","Type":"ContainerStarted","Data":"5f7974ba115e4212d205e5459a88b7bcec89a7ff83fe77b260dd1567c9374724"}
Jul 20 17:09:09.361125 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:09.361084    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v" podStartSLOduration=2.3610698 podStartE2EDuration="2.3610698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:09:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:09:09.360384095 +0000 UTC m=+2372.609878115" watchObservedRunningTime="2026-07-20 17:09:09.3610698 +0000 UTC m=+2372.610563820"
Jul 20 17:09:10.310266 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:10.310230    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:10.315143 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:10.315119    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:10.345869 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:10.345850    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:10.346537 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:10.346520    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:11.314579 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:11.314546    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:11.388750 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:11.388719    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cdcb96-5b8589578d-548xc"]
Jul 20 17:09:11.389106 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:11.389055    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e60b68f6-7c5c-4343-a5af-7fba55a4bab7" containerName="istio-proxy"
Jul 20 17:09:11.389106 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:11.389071    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e60b68f6-7c5c-4343-a5af-7fba55a4bab7" containerName="istio-proxy"
Jul 20 17:09:11.389182 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:11.389132    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e60b68f6-7c5c-4343-a5af-7fba55a4bab7" containerName="istio-proxy"
Jul 20 17:09:11.392144 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:11.392126    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdcb96-5b8589578d-548xc"
Jul 20 17:09:11.394761 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:11.394738    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-cdcb96-serving-cert\""
Jul 20 17:09:11.394891 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:11.394770    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-kxz6w\""
Jul 20 17:09:11.402078 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:11.402051    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cdcb96-5b8589578d-548xc"]
Jul 20 17:09:11.406826 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:11.406791    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:11.428160 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:11.428137    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhjh7\" (UniqueName: \"kubernetes.io/projected/d4cbf8a6-d3ff-4241-8826-e73fbf995298-kube-api-access-fhjh7\") pod \"maas-api-e2e-api-a-cdcb96-5b8589578d-548xc\" (UID: \"d4cbf8a6-d3ff-4241-8826-e73fbf995298\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdcb96-5b8589578d-548xc"
Jul 20 17:09:11.428284 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:11.428206    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4cbf8a6-d3ff-4241-8826-e73fbf995298-maas-api-tls\") pod \"maas-api-e2e-api-a-cdcb96-5b8589578d-548xc\" (UID: \"d4cbf8a6-d3ff-4241-8826-e73fbf995298\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdcb96-5b8589578d-548xc"
Jul 20 17:09:11.428357 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:11.428323    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4cbf8a6-d3ff-4241-8826-e73fbf995298-openshift-service-ca\") pod \"maas-api-e2e-api-a-cdcb96-5b8589578d-548xc\" (UID: \"d4cbf8a6-d3ff-4241-8826-e73fbf995298\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdcb96-5b8589578d-548xc"
Jul 20 17:09:11.474668 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:11.474630    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:11.529651 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:11.529625    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4cbf8a6-d3ff-4241-8826-e73fbf995298-openshift-service-ca\") pod \"maas-api-e2e-api-a-cdcb96-5b8589578d-548xc\" (UID: \"d4cbf8a6-d3ff-4241-8826-e73fbf995298\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdcb96-5b8589578d-548xc"
Jul 20 17:09:11.529803 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:11.529672    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhjh7\" (UniqueName: \"kubernetes.io/projected/d4cbf8a6-d3ff-4241-8826-e73fbf995298-kube-api-access-fhjh7\") pod \"maas-api-e2e-api-a-cdcb96-5b8589578d-548xc\" (UID: \"d4cbf8a6-d3ff-4241-8826-e73fbf995298\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdcb96-5b8589578d-548xc"
Jul 20 17:09:11.529803 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:11.529722    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4cbf8a6-d3ff-4241-8826-e73fbf995298-maas-api-tls\") pod \"maas-api-e2e-api-a-cdcb96-5b8589578d-548xc\" (UID: \"d4cbf8a6-d3ff-4241-8826-e73fbf995298\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdcb96-5b8589578d-548xc"
Jul 20 17:09:11.529922 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:09:11.529845    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-cdcb96-serving-cert: secret "maas-api-e2e-api-a-cdcb96-serving-cert" not found
Jul 20 17:09:11.529922 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:09:11.529919    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4cbf8a6-d3ff-4241-8826-e73fbf995298-maas-api-tls podName:d4cbf8a6-d3ff-4241-8826-e73fbf995298 nodeName:}" failed. No retries permitted until 2026-07-20 17:09:12.029898843 +0000 UTC m=+2375.279392844 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d4cbf8a6-d3ff-4241-8826-e73fbf995298-maas-api-tls") pod "maas-api-e2e-api-a-cdcb96-5b8589578d-548xc" (UID: "d4cbf8a6-d3ff-4241-8826-e73fbf995298") : secret "maas-api-e2e-api-a-cdcb96-serving-cert" not found
Jul 20 17:09:11.530352 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:11.530329    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4cbf8a6-d3ff-4241-8826-e73fbf995298-openshift-service-ca\") pod \"maas-api-e2e-api-a-cdcb96-5b8589578d-548xc\" (UID: \"d4cbf8a6-d3ff-4241-8826-e73fbf995298\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdcb96-5b8589578d-548xc"
Jul 20 17:09:11.538737 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:11.538717    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhjh7\" (UniqueName: \"kubernetes.io/projected/d4cbf8a6-d3ff-4241-8826-e73fbf995298-kube-api-access-fhjh7\") pod \"maas-api-e2e-api-a-cdcb96-5b8589578d-548xc\" (UID: \"d4cbf8a6-d3ff-4241-8826-e73fbf995298\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdcb96-5b8589578d-548xc"
Jul 20 17:09:11.775337 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:11.775243    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:11.974227 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:11.974195    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:12.034798 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.034702    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4cbf8a6-d3ff-4241-8826-e73fbf995298-maas-api-tls\") pod \"maas-api-e2e-api-a-cdcb96-5b8589578d-548xc\" (UID: \"d4cbf8a6-d3ff-4241-8826-e73fbf995298\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdcb96-5b8589578d-548xc"
Jul 20 17:09:12.037108 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.037085    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4cbf8a6-d3ff-4241-8826-e73fbf995298-maas-api-tls\") pod \"maas-api-e2e-api-a-cdcb96-5b8589578d-548xc\" (UID: \"d4cbf8a6-d3ff-4241-8826-e73fbf995298\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdcb96-5b8589578d-548xc"
Jul 20 17:09:12.074753 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.074725    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:12.304734 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.304707    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdcb96-5b8589578d-548xc"
Jul 20 17:09:12.344492 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.344465    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:09:12.357366 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.357336    2577 generic.go:356] "Generic (PLEG): container finished" podID="8c8cf702-2c5b-4a83-8d0d-e11139c6a62e" containerID="68e108990036d9a065b8d57eabf51eedf17e4eec82af2f5983da2f2f2453b9c1" exitCode=0
Jul 20 17:09:12.357482 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.357421    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"
Jul 20 17:09:12.357482 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.357457    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd" event={"ID":"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e","Type":"ContainerDied","Data":"68e108990036d9a065b8d57eabf51eedf17e4eec82af2f5983da2f2f2453b9c1"}
Jul 20 17:09:12.357553 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.357507    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd" event={"ID":"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e","Type":"ContainerDied","Data":"922d299122dca10f7efa9d1f6ce371c506837bf5a54c80b3f27edc8d4bbe637f"}
Jul 20 17:09:12.357553 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.357529    2577 scope.go:117] "RemoveContainer" containerID="68e108990036d9a065b8d57eabf51eedf17e4eec82af2f5983da2f2f2453b9c1"
Jul 20 17:09:12.371663 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.371636    2577 scope.go:117] "RemoveContainer" containerID="2dab258602e26296fd6522bd47d48961ce1fd850e86876d7ac5cad3901c63a76"
Jul 20 17:09:12.383288 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.383246    2577 scope.go:117] "RemoveContainer" containerID="68e108990036d9a065b8d57eabf51eedf17e4eec82af2f5983da2f2f2453b9c1"
Jul 20 17:09:12.383977 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:09:12.383567    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68e108990036d9a065b8d57eabf51eedf17e4eec82af2f5983da2f2f2453b9c1\": container with ID starting with 68e108990036d9a065b8d57eabf51eedf17e4eec82af2f5983da2f2f2453b9c1 not found: ID does not exist" containerID="68e108990036d9a065b8d57eabf51eedf17e4eec82af2f5983da2f2f2453b9c1"
Jul 20 17:09:12.383977 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.383599    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68e108990036d9a065b8d57eabf51eedf17e4eec82af2f5983da2f2f2453b9c1"} err="failed to get container status \"68e108990036d9a065b8d57eabf51eedf17e4eec82af2f5983da2f2f2453b9c1\": rpc error: code = NotFound desc = could not find container \"68e108990036d9a065b8d57eabf51eedf17e4eec82af2f5983da2f2f2453b9c1\": container with ID starting with 68e108990036d9a065b8d57eabf51eedf17e4eec82af2f5983da2f2f2453b9c1 not found: ID does not exist"
Jul 20 17:09:12.383977 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.383619    2577 scope.go:117] "RemoveContainer" containerID="2dab258602e26296fd6522bd47d48961ce1fd850e86876d7ac5cad3901c63a76"
Jul 20 17:09:12.383977 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:09:12.383826    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2dab258602e26296fd6522bd47d48961ce1fd850e86876d7ac5cad3901c63a76\": container with ID starting with 2dab258602e26296fd6522bd47d48961ce1fd850e86876d7ac5cad3901c63a76 not found: ID does not exist" containerID="2dab258602e26296fd6522bd47d48961ce1fd850e86876d7ac5cad3901c63a76"
Jul 20 17:09:12.383977 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.383843    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dab258602e26296fd6522bd47d48961ce1fd850e86876d7ac5cad3901c63a76"} err="failed to get container status \"2dab258602e26296fd6522bd47d48961ce1fd850e86876d7ac5cad3901c63a76\": rpc error: code = NotFound desc = could not find container \"2dab258602e26296fd6522bd47d48961ce1fd850e86876d7ac5cad3901c63a76\": container with ID starting with 2dab258602e26296fd6522bd47d48961ce1fd850e86876d7ac5cad3901c63a76 not found: ID does not exist"
Jul 20 17:09:12.438263 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.438236    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-istio-podinfo\") pod \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") "
Jul 20 17:09:12.438737 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.438268    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87slx\" (UniqueName: \"kubernetes.io/projected/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-kube-api-access-87slx\") pod \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") "
Jul 20 17:09:12.438737 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.438349    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-istiod-ca-cert\") pod \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") "
Jul 20 17:09:12.438737 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.438365    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-workload-certs\") pod \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") "
Jul 20 17:09:12.438737 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.438435    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-istio-envoy\") pod \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") "
Jul 20 17:09:12.438737 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.438467    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-workload-socket\") pod \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") "
Jul 20 17:09:12.438737 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.438507    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-credential-socket\") pod \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") "
Jul 20 17:09:12.438737 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.438530    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-istio-token\") pod \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") "
Jul 20 17:09:12.438737 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.438575    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-istio-data\") pod \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\" (UID: \"8c8cf702-2c5b-4a83-8d0d-e11139c6a62e\") "
Jul 20 17:09:12.438737 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.438695    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8c8cf702-2c5b-4a83-8d0d-e11139c6a62e" (UID: "8c8cf702-2c5b-4a83-8d0d-e11139c6a62e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:09:12.439246 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.438749    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8c8cf702-2c5b-4a83-8d0d-e11139c6a62e" (UID: "8c8cf702-2c5b-4a83-8d0d-e11139c6a62e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 17:09:12.439246 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.439102    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-istiod-ca-cert\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:09:12.439246 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.439123    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-workload-certs\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:09:12.439246 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.439160    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8c8cf702-2c5b-4a83-8d0d-e11139c6a62e" (UID: "8c8cf702-2c5b-4a83-8d0d-e11139c6a62e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:09:12.439488 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.439369    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8c8cf702-2c5b-4a83-8d0d-e11139c6a62e" (UID: "8c8cf702-2c5b-4a83-8d0d-e11139c6a62e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:09:12.439574 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.439547    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-istio-data" (OuterVolumeSpecName: "istio-data") pod "8c8cf702-2c5b-4a83-8d0d-e11139c6a62e" (UID: "8c8cf702-2c5b-4a83-8d0d-e11139c6a62e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:09:12.440638 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.440605    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-kube-api-access-87slx" (OuterVolumeSpecName: "kube-api-access-87slx") pod "8c8cf702-2c5b-4a83-8d0d-e11139c6a62e" (UID: "8c8cf702-2c5b-4a83-8d0d-e11139c6a62e"). InnerVolumeSpecName "kube-api-access-87slx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:09:12.440766 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.440631    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8c8cf702-2c5b-4a83-8d0d-e11139c6a62e" (UID: "8c8cf702-2c5b-4a83-8d0d-e11139c6a62e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 20 17:09:12.440766 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.440724    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-istio-token" (OuterVolumeSpecName: "istio-token") pod "8c8cf702-2c5b-4a83-8d0d-e11139c6a62e" (UID: "8c8cf702-2c5b-4a83-8d0d-e11139c6a62e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:09:12.441401 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.441382    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8c8cf702-2c5b-4a83-8d0d-e11139c6a62e" (UID: "8c8cf702-2c5b-4a83-8d0d-e11139c6a62e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:09:12.446855 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.446836    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cdcb96-5b8589578d-548xc"]
Jul 20 17:09:12.448556 ip-10-0-130-85 kubenswrapper[2577]: W0720 17:09:12.448538    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4cbf8a6_d3ff_4241_8826_e73fbf995298.slice/crio-67606623f3c0f0802bf5bab3f7dc99560bd531cdca2257ef249c41d11438645a WatchSource:0}: Error finding container 67606623f3c0f0802bf5bab3f7dc99560bd531cdca2257ef249c41d11438645a: Status 404 returned error can't find the container with id 67606623f3c0f0802bf5bab3f7dc99560bd531cdca2257ef249c41d11438645a
Jul 20 17:09:12.450194 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.450174    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 17:09:12.539999 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.539976    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-istio-data\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:09:12.539999 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.539997    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-istio-podinfo\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:09:12.540158 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.540006    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87slx\" (UniqueName: \"kubernetes.io/projected/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-kube-api-access-87slx\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:09:12.540158 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.540017    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-istio-envoy\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:09:12.540158 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.540028    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-workload-socket\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:09:12.540158 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.540036    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-credential-socket\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:09:12.540158 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.540044    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e-istio-token\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:09:12.679979 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.679953    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"]
Jul 20 17:09:12.683852 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:12.683830    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-ab00cf-openshift-default-5b5684fd9f-wqjtd"]
Jul 20 17:09:13.212914 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:13.212836    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c8cf702-2c5b-4a83-8d0d-e11139c6a62e" path="/var/lib/kubelet/pods/8c8cf702-2c5b-4a83-8d0d-e11139c6a62e/volumes"
Jul 20 17:09:13.364124 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:13.364083    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdcb96-5b8589578d-548xc" event={"ID":"d4cbf8a6-d3ff-4241-8826-e73fbf995298","Type":"ContainerStarted","Data":"1c3ba400bd6c8de88671a73a3776a72b3673724a397de94804ad5fd7d23917a7"}
Jul 20 17:09:13.364345 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:13.364136    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdcb96-5b8589578d-548xc" event={"ID":"d4cbf8a6-d3ff-4241-8826-e73fbf995298","Type":"ContainerStarted","Data":"67606623f3c0f0802bf5bab3f7dc99560bd531cdca2257ef249c41d11438645a"}
Jul 20 17:09:13.364345 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:13.364168    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdcb96-5b8589578d-548xc"
Jul 20 17:09:13.384260 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:13.384214    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdcb96-5b8589578d-548xc" podStartSLOduration=1.945576022 podStartE2EDuration="2.384199791s" totalImagesPullingTime="438.623769ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:09:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:09:13.382289825 +0000 UTC m=+2376.631783846" watchObservedRunningTime="2026-07-20 17:09:13.384199791 +0000 UTC m=+2376.633693810"
Jul 20 17:09:15.833869 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:15.833830    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:16.323648 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.323593    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:16.614834 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.614752    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"]
Jul 20 17:09:16.615090 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.615078    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c8cf702-2c5b-4a83-8d0d-e11139c6a62e" containerName="istio-proxy"
Jul 20 17:09:16.615134 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.615092    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c8cf702-2c5b-4a83-8d0d-e11139c6a62e" containerName="istio-proxy"
Jul 20 17:09:16.615179 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.615155    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c8cf702-2c5b-4a83-8d0d-e11139c6a62e" containerName="istio-proxy"
Jul 20 17:09:16.615179 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.615167    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c8cf702-2c5b-4a83-8d0d-e11139c6a62e" containerName="istio-proxy"
Jul 20 17:09:16.618326 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.618286    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:16.625352 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.625174    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-3ee404-openshift-default-dockercfg-bhvqj\""
Jul 20 17:09:16.632259 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.632237    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"]
Jul 20 17:09:16.655032 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.654919    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:16.676811 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.676780    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b202c328-33d4-40f6-a1bc-ec2f6ead0836-istio-envoy\") pod \"e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") " pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:16.676811 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.676816    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b202c328-33d4-40f6-a1bc-ec2f6ead0836-istio-token\") pod \"e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") " pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:16.677044 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.676838    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b202c328-33d4-40f6-a1bc-ec2f6ead0836-credential-socket\") pod \"e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") " pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:16.677044 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.676876    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvbr2\" (UniqueName: \"kubernetes.io/projected/b202c328-33d4-40f6-a1bc-ec2f6ead0836-kube-api-access-jvbr2\") pod \"e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") " pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:16.677044 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.676956    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b202c328-33d4-40f6-a1bc-ec2f6ead0836-workload-socket\") pod \"e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") " pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:16.677044 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.677002    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b202c328-33d4-40f6-a1bc-ec2f6ead0836-istiod-ca-cert\") pod \"e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") " pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:16.677206 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.677069    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b202c328-33d4-40f6-a1bc-ec2f6ead0836-istio-data\") pod \"e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") " pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:16.677206 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.677121    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b202c328-33d4-40f6-a1bc-ec2f6ead0836-istio-podinfo\") pod \"e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") " pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:16.677206 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.677145    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b202c328-33d4-40f6-a1bc-ec2f6ead0836-workload-certs\") pod \"e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") " pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:16.777995 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.777958    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b202c328-33d4-40f6-a1bc-ec2f6ead0836-istio-envoy\") pod \"e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") " pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:16.777995 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.777994    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b202c328-33d4-40f6-a1bc-ec2f6ead0836-istio-token\") pod \"e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") " pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:16.778238 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.778017    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b202c328-33d4-40f6-a1bc-ec2f6ead0836-credential-socket\") pod \"e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") " pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:16.778238 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.778035    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvbr2\" (UniqueName: \"kubernetes.io/projected/b202c328-33d4-40f6-a1bc-ec2f6ead0836-kube-api-access-jvbr2\") pod \"e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") " pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:16.778238 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.778087    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b202c328-33d4-40f6-a1bc-ec2f6ead0836-workload-socket\") pod \"e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") " pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:16.778238 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.778118    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b202c328-33d4-40f6-a1bc-ec2f6ead0836-istiod-ca-cert\") pod \"e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") " pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:16.778238 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.778153    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b202c328-33d4-40f6-a1bc-ec2f6ead0836-istio-data\") pod \"e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") " pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:16.778238 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.778212    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b202c328-33d4-40f6-a1bc-ec2f6ead0836-istio-podinfo\") pod \"e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") " pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:16.778238 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.778237    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b202c328-33d4-40f6-a1bc-ec2f6ead0836-workload-certs\") pod \"e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") " pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:16.778635 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.778516    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b202c328-33d4-40f6-a1bc-ec2f6ead0836-credential-socket\") pod \"e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") " pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:16.778635 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.778594    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b202c328-33d4-40f6-a1bc-ec2f6ead0836-workload-socket\") pod \"e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") " pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:16.778635 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.778627    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b202c328-33d4-40f6-a1bc-ec2f6ead0836-workload-certs\") pod \"e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") " pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:16.778767 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.778667    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b202c328-33d4-40f6-a1bc-ec2f6ead0836-istio-data\") pod \"e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") " pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:16.779122 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.779098    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b202c328-33d4-40f6-a1bc-ec2f6ead0836-istiod-ca-cert\") pod \"e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") " pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:16.780459 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.780437    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b202c328-33d4-40f6-a1bc-ec2f6ead0836-istio-envoy\") pod \"e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") " pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:16.780696 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.780680    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b202c328-33d4-40f6-a1bc-ec2f6ead0836-istio-podinfo\") pod \"e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") " pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:16.785439 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.785421    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b202c328-33d4-40f6-a1bc-ec2f6ead0836-istio-token\") pod \"e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") " pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:16.785820 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.785800    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvbr2\" (UniqueName: \"kubernetes.io/projected/b202c328-33d4-40f6-a1bc-ec2f6ead0836-kube-api-access-jvbr2\") pod \"e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") " pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:16.803903 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.803880    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:16.929068 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:16.928980    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:17.119491 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:17.119459    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"]
Jul 20 17:09:17.126445 ip-10-0-130-85 kubenswrapper[2577]: W0720 17:09:17.126410    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb202c328_33d4_40f6_a1bc_ec2f6ead0836.slice/crio-727ff8e8608c188fd28e788e13abced1001dd3a990ebf0198bfcf03cacf12d97 WatchSource:0}: Error finding container 727ff8e8608c188fd28e788e13abced1001dd3a990ebf0198bfcf03cacf12d97: Status 404 returned error can't find the container with id 727ff8e8608c188fd28e788e13abced1001dd3a990ebf0198bfcf03cacf12d97
Jul 20 17:09:17.128946 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:17.128911    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:09:17.129023 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:17.128976    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:09:17.129023 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:17.129003    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:09:17.383456 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:17.383413    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q" event={"ID":"b202c328-33d4-40f6-a1bc-ec2f6ead0836","Type":"ContainerStarted","Data":"ad1f551371b378accb2e95015fbffbce4fcaca4a94242b25a75a256a8e9cb3c8"}
Jul 20 17:09:17.383654 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:17.383490    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q" event={"ID":"b202c328-33d4-40f6-a1bc-ec2f6ead0836","Type":"ContainerStarted","Data":"727ff8e8608c188fd28e788e13abced1001dd3a990ebf0198bfcf03cacf12d97"}
Jul 20 17:09:17.405962 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:17.405916    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q" podStartSLOduration=1.40589974 podStartE2EDuration="1.40589974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:09:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:09:17.404829249 +0000 UTC m=+2380.654323269" watchObservedRunningTime="2026-07-20 17:09:17.40589974 +0000 UTC m=+2380.655393759"
Jul 20 17:09:17.929405 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:17.929370    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:18.939435 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:18.939399    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:19.333950 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:19.333921    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:19.374748 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:19.374725    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdcb96-5b8589578d-548xc"
Jul 20 17:09:19.392916 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:19.392892    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:19.394121 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:19.394097    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:19.411013 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:19.410983    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3ee404-75d449b655-2v9mm"]
Jul 20 17:09:19.411328 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:19.411313    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c8cf702-2c5b-4a83-8d0d-e11139c6a62e" containerName="istio-proxy"
Jul 20 17:09:19.411328 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:19.411329    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c8cf702-2c5b-4a83-8d0d-e11139c6a62e" containerName="istio-proxy"
Jul 20 17:09:19.414320 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:19.414287    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3ee404-75d449b655-2v9mm"
Jul 20 17:09:19.416920 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:19.416895    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-3ee404-serving-cert\""
Jul 20 17:09:19.430639 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:19.430615    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3ee404-75d449b655-2v9mm"]
Jul 20 17:09:19.436799 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:19.436765    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:19.503789 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:19.503744    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0aac8cb-d161-436b-bb02-3f9e7cae7cda-openshift-service-ca\") pod \"maas-api-e2e-api-b-3ee404-75d449b655-2v9mm\" (UID: \"a0aac8cb-d161-436b-bb02-3f9e7cae7cda\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3ee404-75d449b655-2v9mm"
Jul 20 17:09:19.503989 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:19.503930    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0aac8cb-d161-436b-bb02-3f9e7cae7cda-maas-api-tls\") pod \"maas-api-e2e-api-b-3ee404-75d449b655-2v9mm\" (UID: \"a0aac8cb-d161-436b-bb02-3f9e7cae7cda\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3ee404-75d449b655-2v9mm"
Jul 20 17:09:19.504117 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:19.504095    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfhzd\" (UniqueName: \"kubernetes.io/projected/a0aac8cb-d161-436b-bb02-3f9e7cae7cda-kube-api-access-lfhzd\") pod \"maas-api-e2e-api-b-3ee404-75d449b655-2v9mm\" (UID: \"a0aac8cb-d161-436b-bb02-3f9e7cae7cda\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3ee404-75d449b655-2v9mm"
Jul 20 17:09:19.507999 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:19.507967    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:19.605063 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:19.604966    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfhzd\" (UniqueName: \"kubernetes.io/projected/a0aac8cb-d161-436b-bb02-3f9e7cae7cda-kube-api-access-lfhzd\") pod \"maas-api-e2e-api-b-3ee404-75d449b655-2v9mm\" (UID: \"a0aac8cb-d161-436b-bb02-3f9e7cae7cda\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3ee404-75d449b655-2v9mm"
Jul 20 17:09:19.605063 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:19.605031    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0aac8cb-d161-436b-bb02-3f9e7cae7cda-openshift-service-ca\") pod \"maas-api-e2e-api-b-3ee404-75d449b655-2v9mm\" (UID: \"a0aac8cb-d161-436b-bb02-3f9e7cae7cda\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3ee404-75d449b655-2v9mm"
Jul 20 17:09:19.605315 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:19.605089    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0aac8cb-d161-436b-bb02-3f9e7cae7cda-maas-api-tls\") pod \"maas-api-e2e-api-b-3ee404-75d449b655-2v9mm\" (UID: \"a0aac8cb-d161-436b-bb02-3f9e7cae7cda\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3ee404-75d449b655-2v9mm"
Jul 20 17:09:19.605315 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:09:19.605212    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-3ee404-serving-cert: secret "maas-api-e2e-api-b-3ee404-serving-cert" not found
Jul 20 17:09:19.605315 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:09:19.605275    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0aac8cb-d161-436b-bb02-3f9e7cae7cda-maas-api-tls podName:a0aac8cb-d161-436b-bb02-3f9e7cae7cda nodeName:}" failed. No retries permitted until 2026-07-20 17:09:20.105252777 +0000 UTC m=+2383.354746774 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a0aac8cb-d161-436b-bb02-3f9e7cae7cda-maas-api-tls") pod "maas-api-e2e-api-b-3ee404-75d449b655-2v9mm" (UID: "a0aac8cb-d161-436b-bb02-3f9e7cae7cda") : secret "maas-api-e2e-api-b-3ee404-serving-cert" not found
Jul 20 17:09:19.605810 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:19.605789    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0aac8cb-d161-436b-bb02-3f9e7cae7cda-openshift-service-ca\") pod \"maas-api-e2e-api-b-3ee404-75d449b655-2v9mm\" (UID: \"a0aac8cb-d161-436b-bb02-3f9e7cae7cda\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3ee404-75d449b655-2v9mm"
Jul 20 17:09:19.615320 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:19.615260    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfhzd\" (UniqueName: \"kubernetes.io/projected/a0aac8cb-d161-436b-bb02-3f9e7cae7cda-kube-api-access-lfhzd\") pod \"maas-api-e2e-api-b-3ee404-75d449b655-2v9mm\" (UID: \"a0aac8cb-d161-436b-bb02-3f9e7cae7cda\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3ee404-75d449b655-2v9mm"
Jul 20 17:09:19.929396 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:19.929289    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:20.031120 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:20.031088    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:20.108863 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:20.108828    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0aac8cb-d161-436b-bb02-3f9e7cae7cda-maas-api-tls\") pod \"maas-api-e2e-api-b-3ee404-75d449b655-2v9mm\" (UID: \"a0aac8cb-d161-436b-bb02-3f9e7cae7cda\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3ee404-75d449b655-2v9mm"
Jul 20 17:09:20.111477 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:20.111453    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0aac8cb-d161-436b-bb02-3f9e7cae7cda-maas-api-tls\") pod \"maas-api-e2e-api-b-3ee404-75d449b655-2v9mm\" (UID: \"a0aac8cb-d161-436b-bb02-3f9e7cae7cda\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3ee404-75d449b655-2v9mm"
Jul 20 17:09:20.129152 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:20.129116    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:20.326141 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:20.326098    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3ee404-75d449b655-2v9mm"
Jul 20 17:09:20.471870 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:20.471836    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3ee404-75d449b655-2v9mm"]
Jul 20 17:09:20.472361 ip-10-0-130-85 kubenswrapper[2577]: W0720 17:09:20.472332    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0aac8cb_d161_436b_bb02_3f9e7cae7cda.slice/crio-af0833467eb5d614b63376270e975d54f3bf46dbd468ec2d937664bf4a3d16cc WatchSource:0}: Error finding container af0833467eb5d614b63376270e975d54f3bf46dbd468ec2d937664bf4a3d16cc: Status 404 returned error can't find the container with id af0833467eb5d614b63376270e975d54f3bf46dbd468ec2d937664bf4a3d16cc
Jul 20 17:09:21.405724 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:21.405679    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3ee404-75d449b655-2v9mm" event={"ID":"a0aac8cb-d161-436b-bb02-3f9e7cae7cda","Type":"ContainerStarted","Data":"ce47e93545c38810efd8204b2ed0cff0c2bd78d463fb81247040af8c7384d093"}
Jul 20 17:09:21.406217 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:21.405741    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3ee404-75d449b655-2v9mm" event={"ID":"a0aac8cb-d161-436b-bb02-3f9e7cae7cda","Type":"ContainerStarted","Data":"af0833467eb5d614b63376270e975d54f3bf46dbd468ec2d937664bf4a3d16cc"}
Jul 20 17:09:21.406217 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:21.405869    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3ee404-75d449b655-2v9mm"
Jul 20 17:09:21.428443 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:21.428396    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3ee404-75d449b655-2v9mm" podStartSLOduration=2.054361072 podStartE2EDuration="2.428377411s" totalImagesPullingTime="374.016339ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:09:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:09:21.426635201 +0000 UTC m=+2384.676129221" watchObservedRunningTime="2026-07-20 17:09:21.428377411 +0000 UTC m=+2384.677871432"
Jul 20 17:09:22.412029 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:22.411999    2577 generic.go:356] "Generic (PLEG): container finished" podID="8138c4ee-2a73-4764-bb51-aa7d56b536fd" containerID="3da5cf74e7bcc1e33dd83a63b7d798e958246a57bdcd0b4ecc31c3d6f12c20a3" exitCode=0
Jul 20 17:09:22.412534 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:22.412079    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v" event={"ID":"8138c4ee-2a73-4764-bb51-aa7d56b536fd","Type":"ContainerDied","Data":"3da5cf74e7bcc1e33dd83a63b7d798e958246a57bdcd0b4ecc31c3d6f12c20a3"}
Jul 20 17:09:22.412605 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:22.412570    2577 scope.go:117] "RemoveContainer" containerID="3da5cf74e7bcc1e33dd83a63b7d798e958246a57bdcd0b4ecc31c3d6f12c20a3"
Jul 20 17:09:22.413521 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:22.413494    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:09:22.413623 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:22.413590    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:09:22.413679 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:22.413629    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:09:23.419771 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:23.419735    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v" event={"ID":"8138c4ee-2a73-4764-bb51-aa7d56b536fd","Type":"ContainerStarted","Data":"0495c2a5e5b0d8d1db908babd1e79b6148d32f1ad1e082210c42d294c733a01c"}
Jul 20 17:09:23.832350 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:23.832317    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:24.310230 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:24.310204    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:24.311216 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:24.311192    2577 patch_prober.go:28] interesting pod/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 20 17:09:24.311344 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:24.311251    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v" podUID="8138c4ee-2a73-4764-bb51-aa7d56b536fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 20 17:09:25.310229 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:25.310194    2577 patch_prober.go:28] interesting pod/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 20 17:09:25.310637 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:25.310261    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v" podUID="8138c4ee-2a73-4764-bb51-aa7d56b536fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 20 17:09:25.346142 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:25.346086    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:26.310537 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:26.310501    2577 patch_prober.go:28] interesting pod/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 20 17:09:26.310924 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:26.310554    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v" podUID="8138c4ee-2a73-4764-bb51-aa7d56b536fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 20 17:09:27.343031 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:27.343002    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:27.343887 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:27.343870    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:09:27.417749 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:27.417722    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3ee404-75d449b655-2v9mm"
Jul 20 17:09:32.440511 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:32.440468    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:32.541095 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:32.541045    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:35.434489 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:35.434435    2577 patch_prober.go:28] interesting pod/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 20 17:09:35.434996 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:35.434508    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q" podUID="b202c328-33d4-40f6-a1bc-ec2f6ead0836" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 20 17:09:36.482808 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:36.482735    2577 generic.go:356] "Generic (PLEG): container finished" podID="b202c328-33d4-40f6-a1bc-ec2f6ead0836" containerID="ad1f551371b378accb2e95015fbffbce4fcaca4a94242b25a75a256a8e9cb3c8" exitCode=0
Jul 20 17:09:36.482808 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:36.482796    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q" event={"ID":"b202c328-33d4-40f6-a1bc-ec2f6ead0836","Type":"ContainerDied","Data":"ad1f551371b378accb2e95015fbffbce4fcaca4a94242b25a75a256a8e9cb3c8"}
Jul 20 17:09:36.483284 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:36.483124    2577 scope.go:117] "RemoveContainer" containerID="ad1f551371b378accb2e95015fbffbce4fcaca4a94242b25a75a256a8e9cb3c8"
Jul 20 17:09:36.484103 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:36.484076    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:09:36.484153 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:36.484141    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:09:36.484194 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:36.484174    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:09:37.275584 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:37.275555    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gd4p4_fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c/ovn-acl-logging/0.log"
Jul 20 17:09:37.278545 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:37.278523    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gd4p4_fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c/ovn-acl-logging/0.log"
Jul 20 17:09:37.489197 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:37.489158    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q" event={"ID":"b202c328-33d4-40f6-a1bc-ec2f6ead0836","Type":"ContainerStarted","Data":"29d3f6cc9974f8c005bcc1975264571ffcaaea737ffcffc1effdaf4c7d60cc58"}
Jul 20 17:09:37.930150 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:37.930112    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:37.930429 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:37.930339    2577 patch_prober.go:28] interesting pod/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 20 17:09:37.930429 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:37.930395    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q" podUID="b202c328-33d4-40f6-a1bc-ec2f6ead0836" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 20 17:09:38.929670 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:38.929634    2577 patch_prober.go:28] interesting pod/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 20 17:09:38.930105 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:38.929695    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q" podUID="b202c328-33d4-40f6-a1bc-ec2f6ead0836" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 20 17:09:39.930013 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:39.929972    2577 patch_prober.go:28] interesting pod/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 20 17:09:39.930460 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:39.930037    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q" podUID="b202c328-33d4-40f6-a1bc-ec2f6ead0836" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 20 17:09:40.789458 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:40.789421    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:40.889911 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:40.889879    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:40.930191 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:40.930160    2577 patch_prober.go:28] interesting pod/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 20 17:09:40.930562 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:40.930218    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q" podUID="b202c328-33d4-40f6-a1bc-ec2f6ead0836" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 20 17:09:41.933720 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:41.933686    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:41.934158 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:41.933982    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:41.934732 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:41.934710    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:09:42.467778 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:42.467731    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:42.569794 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:42.569760    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:47.761316 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:47.758079    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cdcb96-5b8589578d-548xc"]
Jul 20 17:09:47.761316 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:47.758401    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdcb96-5b8589578d-548xc" podUID="d4cbf8a6-d3ff-4241-8826-e73fbf995298" containerName="maas-api" containerID="cri-o://1c3ba400bd6c8de88671a73a3776a72b3673724a397de94804ad5fd7d23917a7" gracePeriod=30
Jul 20 17:09:47.771448 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:47.771419    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:48.014717 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:48.014659    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdcb96-5b8589578d-548xc"
Jul 20 17:09:48.058408 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:48.058371    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhjh7\" (UniqueName: \"kubernetes.io/projected/d4cbf8a6-d3ff-4241-8826-e73fbf995298-kube-api-access-fhjh7\") pod \"d4cbf8a6-d3ff-4241-8826-e73fbf995298\" (UID: \"d4cbf8a6-d3ff-4241-8826-e73fbf995298\") "
Jul 20 17:09:48.058554 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:48.058468    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4cbf8a6-d3ff-4241-8826-e73fbf995298-maas-api-tls\") pod \"d4cbf8a6-d3ff-4241-8826-e73fbf995298\" (UID: \"d4cbf8a6-d3ff-4241-8826-e73fbf995298\") "
Jul 20 17:09:48.058554 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:48.058511    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4cbf8a6-d3ff-4241-8826-e73fbf995298-openshift-service-ca\") pod \"d4cbf8a6-d3ff-4241-8826-e73fbf995298\" (UID: \"d4cbf8a6-d3ff-4241-8826-e73fbf995298\") "
Jul 20 17:09:48.058911 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:48.058883    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4cbf8a6-d3ff-4241-8826-e73fbf995298-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d4cbf8a6-d3ff-4241-8826-e73fbf995298" (UID: "d4cbf8a6-d3ff-4241-8826-e73fbf995298"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 17:09:48.060519 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:48.060493    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4cbf8a6-d3ff-4241-8826-e73fbf995298-kube-api-access-fhjh7" (OuterVolumeSpecName: "kube-api-access-fhjh7") pod "d4cbf8a6-d3ff-4241-8826-e73fbf995298" (UID: "d4cbf8a6-d3ff-4241-8826-e73fbf995298"). InnerVolumeSpecName "kube-api-access-fhjh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:09:48.060884 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:48.060869    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4cbf8a6-d3ff-4241-8826-e73fbf995298-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d4cbf8a6-d3ff-4241-8826-e73fbf995298" (UID: "d4cbf8a6-d3ff-4241-8826-e73fbf995298"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:09:48.159345 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:48.159286    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4cbf8a6-d3ff-4241-8826-e73fbf995298-maas-api-tls\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:09:48.159345 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:48.159345    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4cbf8a6-d3ff-4241-8826-e73fbf995298-openshift-service-ca\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:09:48.159345 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:48.159356    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhjh7\" (UniqueName: \"kubernetes.io/projected/d4cbf8a6-d3ff-4241-8826-e73fbf995298-kube-api-access-fhjh7\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:09:48.538245 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:48.538211    2577 generic.go:356] "Generic (PLEG): container finished" podID="d4cbf8a6-d3ff-4241-8826-e73fbf995298" containerID="1c3ba400bd6c8de88671a73a3776a72b3673724a397de94804ad5fd7d23917a7" exitCode=0
Jul 20 17:09:48.538433 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:48.538259    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdcb96-5b8589578d-548xc"
Jul 20 17:09:48.538433 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:48.538293    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdcb96-5b8589578d-548xc" event={"ID":"d4cbf8a6-d3ff-4241-8826-e73fbf995298","Type":"ContainerDied","Data":"1c3ba400bd6c8de88671a73a3776a72b3673724a397de94804ad5fd7d23917a7"}
Jul 20 17:09:48.538433 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:48.538368    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cdcb96-5b8589578d-548xc" event={"ID":"d4cbf8a6-d3ff-4241-8826-e73fbf995298","Type":"ContainerDied","Data":"67606623f3c0f0802bf5bab3f7dc99560bd531cdca2257ef249c41d11438645a"}
Jul 20 17:09:48.538433 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:48.538392    2577 scope.go:117] "RemoveContainer" containerID="1c3ba400bd6c8de88671a73a3776a72b3673724a397de94804ad5fd7d23917a7"
Jul 20 17:09:48.547177 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:48.547162    2577 scope.go:117] "RemoveContainer" containerID="1c3ba400bd6c8de88671a73a3776a72b3673724a397de94804ad5fd7d23917a7"
Jul 20 17:09:48.547462 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:09:48.547443    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c3ba400bd6c8de88671a73a3776a72b3673724a397de94804ad5fd7d23917a7\": container with ID starting with 1c3ba400bd6c8de88671a73a3776a72b3673724a397de94804ad5fd7d23917a7 not found: ID does not exist" containerID="1c3ba400bd6c8de88671a73a3776a72b3673724a397de94804ad5fd7d23917a7"
Jul 20 17:09:48.547529 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:48.547471    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c3ba400bd6c8de88671a73a3776a72b3673724a397de94804ad5fd7d23917a7"} err="failed to get container status \"1c3ba400bd6c8de88671a73a3776a72b3673724a397de94804ad5fd7d23917a7\": rpc error: code = NotFound desc = could not find container \"1c3ba400bd6c8de88671a73a3776a72b3673724a397de94804ad5fd7d23917a7\": container with ID starting with 1c3ba400bd6c8de88671a73a3776a72b3673724a397de94804ad5fd7d23917a7 not found: ID does not exist"
Jul 20 17:09:48.561198 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:48.561171    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cdcb96-5b8589578d-548xc"]
Jul 20 17:09:48.565328 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:48.565292    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cdcb96-5b8589578d-548xc"]
Jul 20 17:09:49.211704 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:49.211661    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4cbf8a6-d3ff-4241-8826-e73fbf995298" path="/var/lib/kubelet/pods/d4cbf8a6-d3ff-4241-8826-e73fbf995298/volumes"
Jul 20 17:09:51.976284 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:51.976238    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:52.355960 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:52.355928    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:58.308139 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:58.308101    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:58.451480 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:58.451452    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:58.596522 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:58.596447    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"]
Jul 20 17:09:58.596710 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:58.596684    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v" podUID="8138c4ee-2a73-4764-bb51-aa7d56b536fd" containerName="istio-proxy" containerID="cri-o://0495c2a5e5b0d8d1db908babd1e79b6148d32f1ad1e082210c42d294c733a01c" gracePeriod=30
Jul 20 17:09:58.615050 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:58.615020    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:58.826129 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:58.826092    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:59.501898 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:59.501819    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:09:59.601458 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:09:59.601429    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:03.839369 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:03.839345    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:10:03.885784 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:03.885712    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8138c4ee-2a73-4764-bb51-aa7d56b536fd-workload-socket\") pod \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") "
Jul 20 17:10:03.885941 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:03.885790    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8138c4ee-2a73-4764-bb51-aa7d56b536fd-istio-data\") pod \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") "
Jul 20 17:10:03.885941 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:03.885832    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8138c4ee-2a73-4764-bb51-aa7d56b536fd-istio-podinfo\") pod \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") "
Jul 20 17:10:03.885941 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:03.885847    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8138c4ee-2a73-4764-bb51-aa7d56b536fd-istio-token\") pod \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") "
Jul 20 17:10:03.885941 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:03.885863    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8138c4ee-2a73-4764-bb51-aa7d56b536fd-workload-certs\") pod \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") "
Jul 20 17:10:03.885941 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:03.885890    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7fb6\" (UniqueName: \"kubernetes.io/projected/8138c4ee-2a73-4764-bb51-aa7d56b536fd-kube-api-access-q7fb6\") pod \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") "
Jul 20 17:10:03.886221 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:03.886008    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8138c4ee-2a73-4764-bb51-aa7d56b536fd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8138c4ee-2a73-4764-bb51-aa7d56b536fd" (UID: "8138c4ee-2a73-4764-bb51-aa7d56b536fd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:10:03.886221 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:03.886079    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8138c4ee-2a73-4764-bb51-aa7d56b536fd-credential-socket\") pod \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") "
Jul 20 17:10:03.886221 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:03.886127    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8138c4ee-2a73-4764-bb51-aa7d56b536fd-istiod-ca-cert\") pod \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") "
Jul 20 17:10:03.886221 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:03.886165    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8138c4ee-2a73-4764-bb51-aa7d56b536fd-istio-envoy\") pod \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\" (UID: \"8138c4ee-2a73-4764-bb51-aa7d56b536fd\") "
Jul 20 17:10:03.886431 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:03.886397    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8138c4ee-2a73-4764-bb51-aa7d56b536fd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8138c4ee-2a73-4764-bb51-aa7d56b536fd" (UID: "8138c4ee-2a73-4764-bb51-aa7d56b536fd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:10:03.886504 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:03.886477    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8138c4ee-2a73-4764-bb51-aa7d56b536fd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8138c4ee-2a73-4764-bb51-aa7d56b536fd" (UID: "8138c4ee-2a73-4764-bb51-aa7d56b536fd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:10:03.886602 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:03.886586    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8138c4ee-2a73-4764-bb51-aa7d56b536fd-workload-socket\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:10:03.886656 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:03.886608    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8138c4ee-2a73-4764-bb51-aa7d56b536fd-workload-certs\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:10:03.886656 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:03.886621    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8138c4ee-2a73-4764-bb51-aa7d56b536fd-credential-socket\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:10:03.886750 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:03.886660    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8138c4ee-2a73-4764-bb51-aa7d56b536fd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8138c4ee-2a73-4764-bb51-aa7d56b536fd" (UID: "8138c4ee-2a73-4764-bb51-aa7d56b536fd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 17:10:03.886884 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:03.886823    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8138c4ee-2a73-4764-bb51-aa7d56b536fd-istio-data" (OuterVolumeSpecName: "istio-data") pod "8138c4ee-2a73-4764-bb51-aa7d56b536fd" (UID: "8138c4ee-2a73-4764-bb51-aa7d56b536fd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:10:03.888819 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:03.888789    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8138c4ee-2a73-4764-bb51-aa7d56b536fd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8138c4ee-2a73-4764-bb51-aa7d56b536fd" (UID: "8138c4ee-2a73-4764-bb51-aa7d56b536fd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 20 17:10:03.888927 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:03.888814    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8138c4ee-2a73-4764-bb51-aa7d56b536fd-istio-token" (OuterVolumeSpecName: "istio-token") pod "8138c4ee-2a73-4764-bb51-aa7d56b536fd" (UID: "8138c4ee-2a73-4764-bb51-aa7d56b536fd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:10:03.888927 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:03.888841    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8138c4ee-2a73-4764-bb51-aa7d56b536fd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8138c4ee-2a73-4764-bb51-aa7d56b536fd" (UID: "8138c4ee-2a73-4764-bb51-aa7d56b536fd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:10:03.889037 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:03.888921    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8138c4ee-2a73-4764-bb51-aa7d56b536fd-kube-api-access-q7fb6" (OuterVolumeSpecName: "kube-api-access-q7fb6") pod "8138c4ee-2a73-4764-bb51-aa7d56b536fd" (UID: "8138c4ee-2a73-4764-bb51-aa7d56b536fd"). InnerVolumeSpecName "kube-api-access-q7fb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:10:03.987240 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:03.987214    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8138c4ee-2a73-4764-bb51-aa7d56b536fd-istio-data\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:10:03.987240 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:03.987236    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8138c4ee-2a73-4764-bb51-aa7d56b536fd-istio-podinfo\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:10:03.987240 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:03.987245    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8138c4ee-2a73-4764-bb51-aa7d56b536fd-istio-token\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:10:03.987443 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:03.987255    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7fb6\" (UniqueName: \"kubernetes.io/projected/8138c4ee-2a73-4764-bb51-aa7d56b536fd-kube-api-access-q7fb6\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:10:03.987443 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:03.987264    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8138c4ee-2a73-4764-bb51-aa7d56b536fd-istiod-ca-cert\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:10:03.987443 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:03.987273    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8138c4ee-2a73-4764-bb51-aa7d56b536fd-istio-envoy\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:10:04.612723 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:04.612689    2577 generic.go:356] "Generic (PLEG): container finished" podID="8138c4ee-2a73-4764-bb51-aa7d56b536fd" containerID="0495c2a5e5b0d8d1db908babd1e79b6148d32f1ad1e082210c42d294c733a01c" exitCode=0
Jul 20 17:10:04.612928 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:04.612734    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"
Jul 20 17:10:04.612928 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:04.612748    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v" event={"ID":"8138c4ee-2a73-4764-bb51-aa7d56b536fd","Type":"ContainerDied","Data":"0495c2a5e5b0d8d1db908babd1e79b6148d32f1ad1e082210c42d294c733a01c"}
Jul 20 17:10:04.612928 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:04.612798    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v" event={"ID":"8138c4ee-2a73-4764-bb51-aa7d56b536fd","Type":"ContainerDied","Data":"5f7974ba115e4212d205e5459a88b7bcec89a7ff83fe77b260dd1567c9374724"}
Jul 20 17:10:04.612928 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:04.612828    2577 scope.go:117] "RemoveContainer" containerID="0495c2a5e5b0d8d1db908babd1e79b6148d32f1ad1e082210c42d294c733a01c"
Jul 20 17:10:04.621999 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:04.621983    2577 scope.go:117] "RemoveContainer" containerID="3da5cf74e7bcc1e33dd83a63b7d798e958246a57bdcd0b4ecc31c3d6f12c20a3"
Jul 20 17:10:04.631473 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:04.631335    2577 scope.go:117] "RemoveContainer" containerID="0495c2a5e5b0d8d1db908babd1e79b6148d32f1ad1e082210c42d294c733a01c"
Jul 20 17:10:04.632390 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:10:04.632369    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0495c2a5e5b0d8d1db908babd1e79b6148d32f1ad1e082210c42d294c733a01c\": container with ID starting with 0495c2a5e5b0d8d1db908babd1e79b6148d32f1ad1e082210c42d294c733a01c not found: ID does not exist" containerID="0495c2a5e5b0d8d1db908babd1e79b6148d32f1ad1e082210c42d294c733a01c"
Jul 20 17:10:04.632491 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:04.632403    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0495c2a5e5b0d8d1db908babd1e79b6148d32f1ad1e082210c42d294c733a01c"} err="failed to get container status \"0495c2a5e5b0d8d1db908babd1e79b6148d32f1ad1e082210c42d294c733a01c\": rpc error: code = NotFound desc = could not find container \"0495c2a5e5b0d8d1db908babd1e79b6148d32f1ad1e082210c42d294c733a01c\": container with ID starting with 0495c2a5e5b0d8d1db908babd1e79b6148d32f1ad1e082210c42d294c733a01c not found: ID does not exist"
Jul 20 17:10:04.632491 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:04.632427    2577 scope.go:117] "RemoveContainer" containerID="3da5cf74e7bcc1e33dd83a63b7d798e958246a57bdcd0b4ecc31c3d6f12c20a3"
Jul 20 17:10:04.632723 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:10:04.632700    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3da5cf74e7bcc1e33dd83a63b7d798e958246a57bdcd0b4ecc31c3d6f12c20a3\": container with ID starting with 3da5cf74e7bcc1e33dd83a63b7d798e958246a57bdcd0b4ecc31c3d6f12c20a3 not found: ID does not exist" containerID="3da5cf74e7bcc1e33dd83a63b7d798e958246a57bdcd0b4ecc31c3d6f12c20a3"
Jul 20 17:10:04.632774 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:04.632731    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3da5cf74e7bcc1e33dd83a63b7d798e958246a57bdcd0b4ecc31c3d6f12c20a3"} err="failed to get container status \"3da5cf74e7bcc1e33dd83a63b7d798e958246a57bdcd0b4ecc31c3d6f12c20a3\": rpc error: code = NotFound desc = could not find container \"3da5cf74e7bcc1e33dd83a63b7d798e958246a57bdcd0b4ecc31c3d6f12c20a3\": container with ID starting with 3da5cf74e7bcc1e33dd83a63b7d798e958246a57bdcd0b4ecc31c3d6f12c20a3 not found: ID does not exist"
Jul 20 17:10:04.633919 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:04.633901    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"]
Jul 20 17:10:04.637848 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:04.637828    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-cdcb96-openshift-default-7c5d4c4cb5-hvb7v"]
Jul 20 17:10:05.212165 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:05.212130    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8138c4ee-2a73-4764-bb51-aa7d56b536fd" path="/var/lib/kubelet/pods/8138c4ee-2a73-4764-bb51-aa7d56b536fd/volumes"
Jul 20 17:10:14.428149 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:14.428113    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3ee404-75d449b655-2v9mm"]
Jul 20 17:10:14.428668 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:14.428324    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3ee404-75d449b655-2v9mm" podUID="a0aac8cb-d161-436b-bb02-3f9e7cae7cda" containerName="maas-api" containerID="cri-o://ce47e93545c38810efd8204b2ed0cff0c2bd78d463fb81247040af8c7384d093" gracePeriod=30
Jul 20 17:10:14.461445 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:14.461420    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:14.659748 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:14.659722    2577 generic.go:356] "Generic (PLEG): container finished" podID="a0aac8cb-d161-436b-bb02-3f9e7cae7cda" containerID="ce47e93545c38810efd8204b2ed0cff0c2bd78d463fb81247040af8c7384d093" exitCode=0
Jul 20 17:10:14.659856 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:14.659794    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3ee404-75d449b655-2v9mm" event={"ID":"a0aac8cb-d161-436b-bb02-3f9e7cae7cda","Type":"ContainerDied","Data":"ce47e93545c38810efd8204b2ed0cff0c2bd78d463fb81247040af8c7384d093"}
Jul 20 17:10:14.659856 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:14.659846    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3ee404-75d449b655-2v9mm" event={"ID":"a0aac8cb-d161-436b-bb02-3f9e7cae7cda","Type":"ContainerDied","Data":"af0833467eb5d614b63376270e975d54f3bf46dbd468ec2d937664bf4a3d16cc"}
Jul 20 17:10:14.659928 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:14.659868    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="af0833467eb5d614b63376270e975d54f3bf46dbd468ec2d937664bf4a3d16cc"
Jul 20 17:10:14.668551 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:14.668533    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3ee404-75d449b655-2v9mm"
Jul 20 17:10:14.776055 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:14.775983    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfhzd\" (UniqueName: \"kubernetes.io/projected/a0aac8cb-d161-436b-bb02-3f9e7cae7cda-kube-api-access-lfhzd\") pod \"a0aac8cb-d161-436b-bb02-3f9e7cae7cda\" (UID: \"a0aac8cb-d161-436b-bb02-3f9e7cae7cda\") "
Jul 20 17:10:14.776055 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:14.776025    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0aac8cb-d161-436b-bb02-3f9e7cae7cda-openshift-service-ca\") pod \"a0aac8cb-d161-436b-bb02-3f9e7cae7cda\" (UID: \"a0aac8cb-d161-436b-bb02-3f9e7cae7cda\") "
Jul 20 17:10:14.776252 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:14.776111    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0aac8cb-d161-436b-bb02-3f9e7cae7cda-maas-api-tls\") pod \"a0aac8cb-d161-436b-bb02-3f9e7cae7cda\" (UID: \"a0aac8cb-d161-436b-bb02-3f9e7cae7cda\") "
Jul 20 17:10:14.776479 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:14.776455    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0aac8cb-d161-436b-bb02-3f9e7cae7cda-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a0aac8cb-d161-436b-bb02-3f9e7cae7cda" (UID: "a0aac8cb-d161-436b-bb02-3f9e7cae7cda"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 17:10:14.778010 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:14.777978    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0aac8cb-d161-436b-bb02-3f9e7cae7cda-kube-api-access-lfhzd" (OuterVolumeSpecName: "kube-api-access-lfhzd") pod "a0aac8cb-d161-436b-bb02-3f9e7cae7cda" (UID: "a0aac8cb-d161-436b-bb02-3f9e7cae7cda"). InnerVolumeSpecName "kube-api-access-lfhzd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:10:14.778102 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:14.778056    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0aac8cb-d161-436b-bb02-3f9e7cae7cda-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a0aac8cb-d161-436b-bb02-3f9e7cae7cda" (UID: "a0aac8cb-d161-436b-bb02-3f9e7cae7cda"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:10:14.876574 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:14.876548    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0aac8cb-d161-436b-bb02-3f9e7cae7cda-openshift-service-ca\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:10:14.876574 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:14.876574    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0aac8cb-d161-436b-bb02-3f9e7cae7cda-maas-api-tls\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:10:14.876711 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:14.876583    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfhzd\" (UniqueName: \"kubernetes.io/projected/a0aac8cb-d161-436b-bb02-3f9e7cae7cda-kube-api-access-lfhzd\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:10:15.665079 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:15.665051    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3ee404-75d449b655-2v9mm"
Jul 20 17:10:15.684754 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:15.684723    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3ee404-75d449b655-2v9mm"]
Jul 20 17:10:15.688158 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:15.688135    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3ee404-75d449b655-2v9mm"]
Jul 20 17:10:17.211428 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:17.211399    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0aac8cb-d161-436b-bb02-3f9e7cae7cda" path="/var/lib/kubelet/pods/a0aac8cb-d161-436b-bb02-3f9e7cae7cda/volumes"
Jul 20 17:10:19.041707 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:19.041656    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:19.429165 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:19.429129    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:23.005750 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:23.005712    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:23.122503 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:23.122472    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:25.242174 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:25.242139    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:25.342738 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:25.342703    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:25.537983 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:25.537904    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"]
Jul 20 17:10:25.538156 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:25.538133    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q" podUID="b202c328-33d4-40f6-a1bc-ec2f6ead0836" containerName="istio-proxy" containerID="cri-o://29d3f6cc9974f8c005bcc1975264571ffcaaea737ffcffc1effdaf4c7d60cc58" gracePeriod=30
Jul 20 17:10:25.555883 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:25.555852    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:25.876712 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:25.876678    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:26.263038 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.262950    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:26.629456 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.629423    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"]
Jul 20 17:10:26.629733 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.629720    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8138c4ee-2a73-4764-bb51-aa7d56b536fd" containerName="istio-proxy"
Jul 20 17:10:26.629733 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.629732    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8138c4ee-2a73-4764-bb51-aa7d56b536fd" containerName="istio-proxy"
Jul 20 17:10:26.629854 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.629740    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4cbf8a6-d3ff-4241-8826-e73fbf995298" containerName="maas-api"
Jul 20 17:10:26.629854 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.629746    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4cbf8a6-d3ff-4241-8826-e73fbf995298" containerName="maas-api"
Jul 20 17:10:26.629854 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.629753    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0aac8cb-d161-436b-bb02-3f9e7cae7cda" containerName="maas-api"
Jul 20 17:10:26.629854 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.629759    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0aac8cb-d161-436b-bb02-3f9e7cae7cda" containerName="maas-api"
Jul 20 17:10:26.629854 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.629838    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0aac8cb-d161-436b-bb02-3f9e7cae7cda" containerName="maas-api"
Jul 20 17:10:26.629854 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.629844    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8138c4ee-2a73-4764-bb51-aa7d56b536fd" containerName="istio-proxy"
Jul 20 17:10:26.629854 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.629854    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4cbf8a6-d3ff-4241-8826-e73fbf995298" containerName="maas-api"
Jul 20 17:10:26.630209 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.629859    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8138c4ee-2a73-4764-bb51-aa7d56b536fd" containerName="istio-proxy"
Jul 20 17:10:26.633058 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.633039    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:26.635739 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.635720    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-a16895-openshift-default-dockercfg-f7hx4\""
Jul 20 17:10:26.651517 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.651494    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"]
Jul 20 17:10:26.668896 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.668870    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:26.780988 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.780949    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1893d618-2628-4baa-a0de-803c6b15be38-workload-certs\") pod \"e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") " pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:26.781133 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.781031    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1893d618-2628-4baa-a0de-803c6b15be38-istio-envoy\") pod \"e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") " pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:26.781133 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.781064    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlp9m\" (UniqueName: \"kubernetes.io/projected/1893d618-2628-4baa-a0de-803c6b15be38-kube-api-access-rlp9m\") pod \"e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") " pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:26.781133 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.781126    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1893d618-2628-4baa-a0de-803c6b15be38-credential-socket\") pod \"e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") " pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:26.781241 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.781155    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1893d618-2628-4baa-a0de-803c6b15be38-istio-token\") pod \"e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") " pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:26.781241 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.781186    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1893d618-2628-4baa-a0de-803c6b15be38-istiod-ca-cert\") pod \"e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") " pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:26.781241 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.781214    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1893d618-2628-4baa-a0de-803c6b15be38-istio-podinfo\") pod \"e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") " pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:26.781361 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.781241    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1893d618-2628-4baa-a0de-803c6b15be38-workload-socket\") pod \"e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") " pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:26.781361 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.781341    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1893d618-2628-4baa-a0de-803c6b15be38-istio-data\") pod \"e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") " pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:26.882142 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.882073    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1893d618-2628-4baa-a0de-803c6b15be38-istio-data\") pod \"e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") " pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:26.882142 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.882105    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1893d618-2628-4baa-a0de-803c6b15be38-workload-certs\") pod \"e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") " pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:26.882326 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.882145    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1893d618-2628-4baa-a0de-803c6b15be38-istio-envoy\") pod \"e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") " pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:26.882326 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.882166    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlp9m\" (UniqueName: \"kubernetes.io/projected/1893d618-2628-4baa-a0de-803c6b15be38-kube-api-access-rlp9m\") pod \"e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") " pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:26.882326 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.882195    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1893d618-2628-4baa-a0de-803c6b15be38-credential-socket\") pod \"e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") " pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:26.882326 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.882210    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1893d618-2628-4baa-a0de-803c6b15be38-istio-token\") pod \"e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") " pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:26.882326 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.882226    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1893d618-2628-4baa-a0de-803c6b15be38-istiod-ca-cert\") pod \"e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") " pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:26.882591 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.882382    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1893d618-2628-4baa-a0de-803c6b15be38-istio-podinfo\") pod \"e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") " pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:26.882591 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.882435    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1893d618-2628-4baa-a0de-803c6b15be38-workload-socket\") pod \"e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") " pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:26.882710 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.882582    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1893d618-2628-4baa-a0de-803c6b15be38-istio-data\") pod \"e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") " pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:26.882710 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.882604    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1893d618-2628-4baa-a0de-803c6b15be38-workload-certs\") pod \"e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") " pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:26.882820 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.882803    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1893d618-2628-4baa-a0de-803c6b15be38-workload-socket\") pod \"e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") " pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:26.882921 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.882898    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1893d618-2628-4baa-a0de-803c6b15be38-credential-socket\") pod \"e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") " pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:26.883036 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.883013    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1893d618-2628-4baa-a0de-803c6b15be38-istiod-ca-cert\") pod \"e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") " pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:26.884579 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.884554    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1893d618-2628-4baa-a0de-803c6b15be38-istio-envoy\") pod \"e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") " pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:26.884958 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.884939    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1893d618-2628-4baa-a0de-803c6b15be38-istio-podinfo\") pod \"e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") " pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:26.890182 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.890155    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1893d618-2628-4baa-a0de-803c6b15be38-istio-token\") pod \"e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") " pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:26.890259 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.890223    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlp9m\" (UniqueName: \"kubernetes.io/projected/1893d618-2628-4baa-a0de-803c6b15be38-kube-api-access-rlp9m\") pod \"e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") " pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:26.936682 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.936656    2577 patch_prober.go:28] interesting pod/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 20 17:10:26.936772 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.936711    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q" podUID="b202c328-33d4-40f6-a1bc-ec2f6ead0836" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 20 17:10:26.943493 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.943476    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:26.964463 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:26.964433    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:27.073492 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:27.073470    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"]
Jul 20 17:10:27.075724 ip-10-0-130-85 kubenswrapper[2577]: W0720 17:10:27.075697    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1893d618_2628_4baa_a0de_803c6b15be38.slice/crio-db6b6ca71d8e3ca13f3ecea6e1c838beb5274c908912631e26b1d36e3e04ea67 WatchSource:0}: Error finding container db6b6ca71d8e3ca13f3ecea6e1c838beb5274c908912631e26b1d36e3e04ea67: Status 404 returned error can't find the container with id db6b6ca71d8e3ca13f3ecea6e1c838beb5274c908912631e26b1d36e3e04ea67
Jul 20 17:10:27.078106 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:27.078069    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:10:27.078180 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:27.078148    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:10:27.078215 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:27.078191    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:10:27.718219 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:27.718188    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4" event={"ID":"1893d618-2628-4baa-a0de-803c6b15be38","Type":"ContainerStarted","Data":"0f966d33fedf2be859a0d310fc6d20bc4a14c61fa73d7d66a20fe32431b49272"}
Jul 20 17:10:27.718645 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:27.718240    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4" event={"ID":"1893d618-2628-4baa-a0de-803c6b15be38","Type":"ContainerStarted","Data":"db6b6ca71d8e3ca13f3ecea6e1c838beb5274c908912631e26b1d36e3e04ea67"}
Jul 20 17:10:27.739456 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:27.739399    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4" podStartSLOduration=1.7393830019999998 podStartE2EDuration="1.739383002s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:10:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:10:27.737794991 +0000 UTC m=+2450.987289011" watchObservedRunningTime="2026-07-20 17:10:27.739383002 +0000 UTC m=+2450.988877020"
Jul 20 17:10:27.944457 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:27.944433    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:28.948831 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:28.948801    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:29.711068 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:29.711023    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:29.726533 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:29.726507    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:29.727483 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:29.727459    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:29.802154 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:29.802119    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a16895-56cb9799b-9n96c"]
Jul 20 17:10:29.802625 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:29.802606    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8138c4ee-2a73-4764-bb51-aa7d56b536fd" containerName="istio-proxy"
Jul 20 17:10:29.802625 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:29.802626    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8138c4ee-2a73-4764-bb51-aa7d56b536fd" containerName="istio-proxy"
Jul 20 17:10:29.805807 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:29.805785    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a16895-56cb9799b-9n96c"
Jul 20 17:10:29.808627 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:29.808605    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-kxz6w\""
Jul 20 17:10:29.808727 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:29.808607    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-a16895-serving-cert\""
Jul 20 17:10:29.816876 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:29.816854    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a16895-56cb9799b-9n96c"]
Jul 20 17:10:29.821423 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:29.821400    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:29.907155 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:29.907128    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/161ff40d-6008-4ffe-8e60-0d5b53567e3c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-a16895-56cb9799b-9n96c\" (UID: \"161ff40d-6008-4ffe-8e60-0d5b53567e3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a16895-56cb9799b-9n96c"
Jul 20 17:10:29.907360 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:29.907263    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcfht\" (UniqueName: \"kubernetes.io/projected/161ff40d-6008-4ffe-8e60-0d5b53567e3c-kube-api-access-lcfht\") pod \"maas-api-e2e-shared-a-a16895-56cb9799b-9n96c\" (UID: \"161ff40d-6008-4ffe-8e60-0d5b53567e3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a16895-56cb9799b-9n96c"
Jul 20 17:10:29.907360 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:29.907354    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/161ff40d-6008-4ffe-8e60-0d5b53567e3c-maas-api-tls\") pod \"maas-api-e2e-shared-a-a16895-56cb9799b-9n96c\" (UID: \"161ff40d-6008-4ffe-8e60-0d5b53567e3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a16895-56cb9799b-9n96c"
Jul 20 17:10:29.914891 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:29.914866    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:30.008336 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.008267    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/161ff40d-6008-4ffe-8e60-0d5b53567e3c-maas-api-tls\") pod \"maas-api-e2e-shared-a-a16895-56cb9799b-9n96c\" (UID: \"161ff40d-6008-4ffe-8e60-0d5b53567e3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a16895-56cb9799b-9n96c"
Jul 20 17:10:30.008336 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.008325    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/161ff40d-6008-4ffe-8e60-0d5b53567e3c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-a16895-56cb9799b-9n96c\" (UID: \"161ff40d-6008-4ffe-8e60-0d5b53567e3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a16895-56cb9799b-9n96c"
Jul 20 17:10:30.008746 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:10:30.008411    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-a16895-serving-cert: secret "maas-api-e2e-shared-a-a16895-serving-cert" not found
Jul 20 17:10:30.008746 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.008427    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcfht\" (UniqueName: \"kubernetes.io/projected/161ff40d-6008-4ffe-8e60-0d5b53567e3c-kube-api-access-lcfht\") pod \"maas-api-e2e-shared-a-a16895-56cb9799b-9n96c\" (UID: \"161ff40d-6008-4ffe-8e60-0d5b53567e3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a16895-56cb9799b-9n96c"
Jul 20 17:10:30.008746 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:10:30.008470    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/161ff40d-6008-4ffe-8e60-0d5b53567e3c-maas-api-tls podName:161ff40d-6008-4ffe-8e60-0d5b53567e3c nodeName:}" failed. No retries permitted until 2026-07-20 17:10:30.508450525 +0000 UTC m=+2453.757944534 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/161ff40d-6008-4ffe-8e60-0d5b53567e3c-maas-api-tls") pod "maas-api-e2e-shared-a-a16895-56cb9799b-9n96c" (UID: "161ff40d-6008-4ffe-8e60-0d5b53567e3c") : secret "maas-api-e2e-shared-a-a16895-serving-cert" not found
Jul 20 17:10:30.008942 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.008924    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/161ff40d-6008-4ffe-8e60-0d5b53567e3c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-a16895-56cb9799b-9n96c\" (UID: \"161ff40d-6008-4ffe-8e60-0d5b53567e3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a16895-56cb9799b-9n96c"
Jul 20 17:10:30.020618 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.020595    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcfht\" (UniqueName: \"kubernetes.io/projected/161ff40d-6008-4ffe-8e60-0d5b53567e3c-kube-api-access-lcfht\") pod \"maas-api-e2e-shared-a-a16895-56cb9799b-9n96c\" (UID: \"161ff40d-6008-4ffe-8e60-0d5b53567e3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a16895-56cb9799b-9n96c"
Jul 20 17:10:30.073332 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.073288    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:30.510724 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.510683    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:30.512877 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.512848    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/161ff40d-6008-4ffe-8e60-0d5b53567e3c-maas-api-tls\") pod \"maas-api-e2e-shared-a-a16895-56cb9799b-9n96c\" (UID: \"161ff40d-6008-4ffe-8e60-0d5b53567e3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a16895-56cb9799b-9n96c"
Jul 20 17:10:30.515467 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.515446    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/161ff40d-6008-4ffe-8e60-0d5b53567e3c-maas-api-tls\") pod \"maas-api-e2e-shared-a-a16895-56cb9799b-9n96c\" (UID: \"161ff40d-6008-4ffe-8e60-0d5b53567e3c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a16895-56cb9799b-9n96c"
Jul 20 17:10:30.612820 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.612773    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:30.719028 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.718982    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a16895-56cb9799b-9n96c"
Jul 20 17:10:30.738048 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.738008    2577 generic.go:356] "Generic (PLEG): container finished" podID="b202c328-33d4-40f6-a1bc-ec2f6ead0836" containerID="29d3f6cc9974f8c005bcc1975264571ffcaaea737ffcffc1effdaf4c7d60cc58" exitCode=0
Jul 20 17:10:30.738175 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.738051    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q" event={"ID":"b202c328-33d4-40f6-a1bc-ec2f6ead0836","Type":"ContainerDied","Data":"29d3f6cc9974f8c005bcc1975264571ffcaaea737ffcffc1effdaf4c7d60cc58"}
Jul 20 17:10:30.738175 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.738163    2577 scope.go:117] "RemoveContainer" containerID="ad1f551371b378accb2e95015fbffbce4fcaca4a94242b25a75a256a8e9cb3c8"
Jul 20 17:10:30.778962 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.778936    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:10:30.815148 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.815120    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b202c328-33d4-40f6-a1bc-ec2f6ead0836-workload-socket\") pod \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") "
Jul 20 17:10:30.815284 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.815173    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b202c328-33d4-40f6-a1bc-ec2f6ead0836-credential-socket\") pod \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") "
Jul 20 17:10:30.815284 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.815199    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b202c328-33d4-40f6-a1bc-ec2f6ead0836-istio-data\") pod \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") "
Jul 20 17:10:30.815284 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.815258    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jvbr2\" (UniqueName: \"kubernetes.io/projected/b202c328-33d4-40f6-a1bc-ec2f6ead0836-kube-api-access-jvbr2\") pod \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") "
Jul 20 17:10:30.815481 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.815286    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b202c328-33d4-40f6-a1bc-ec2f6ead0836-workload-certs\") pod \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") "
Jul 20 17:10:30.815481 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.815330    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b202c328-33d4-40f6-a1bc-ec2f6ead0836-istio-token\") pod \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") "
Jul 20 17:10:30.815481 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.815359    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b202c328-33d4-40f6-a1bc-ec2f6ead0836-istiod-ca-cert\") pod \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") "
Jul 20 17:10:30.815481 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.815402    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b202c328-33d4-40f6-a1bc-ec2f6ead0836-istio-envoy\") pod \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") "
Jul 20 17:10:30.815481 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.815427    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b202c328-33d4-40f6-a1bc-ec2f6ead0836-istio-podinfo\") pod \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\" (UID: \"b202c328-33d4-40f6-a1bc-ec2f6ead0836\") "
Jul 20 17:10:30.815481 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.815437    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b202c328-33d4-40f6-a1bc-ec2f6ead0836-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b202c328-33d4-40f6-a1bc-ec2f6ead0836" (UID: "b202c328-33d4-40f6-a1bc-ec2f6ead0836"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:10:30.815767 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.815565    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b202c328-33d4-40f6-a1bc-ec2f6ead0836-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b202c328-33d4-40f6-a1bc-ec2f6ead0836" (UID: "b202c328-33d4-40f6-a1bc-ec2f6ead0836"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:10:30.815767 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.815749    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b202c328-33d4-40f6-a1bc-ec2f6ead0836-workload-socket\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:10:30.815872 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.815768    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b202c328-33d4-40f6-a1bc-ec2f6ead0836-credential-socket\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:10:30.816287 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.815963    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b202c328-33d4-40f6-a1bc-ec2f6ead0836-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b202c328-33d4-40f6-a1bc-ec2f6ead0836" (UID: "b202c328-33d4-40f6-a1bc-ec2f6ead0836"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:10:30.816287 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.816011    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b202c328-33d4-40f6-a1bc-ec2f6ead0836-istio-data" (OuterVolumeSpecName: "istio-data") pod "b202c328-33d4-40f6-a1bc-ec2f6ead0836" (UID: "b202c328-33d4-40f6-a1bc-ec2f6ead0836"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:10:30.816476 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.816386    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b202c328-33d4-40f6-a1bc-ec2f6ead0836-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b202c328-33d4-40f6-a1bc-ec2f6ead0836" (UID: "b202c328-33d4-40f6-a1bc-ec2f6ead0836"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 17:10:30.818561 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.818535    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b202c328-33d4-40f6-a1bc-ec2f6ead0836-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b202c328-33d4-40f6-a1bc-ec2f6ead0836" (UID: "b202c328-33d4-40f6-a1bc-ec2f6ead0836"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 20 17:10:30.818652 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.818581    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b202c328-33d4-40f6-a1bc-ec2f6ead0836-kube-api-access-jvbr2" (OuterVolumeSpecName: "kube-api-access-jvbr2") pod "b202c328-33d4-40f6-a1bc-ec2f6ead0836" (UID: "b202c328-33d4-40f6-a1bc-ec2f6ead0836"). InnerVolumeSpecName "kube-api-access-jvbr2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:10:30.820612 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.820586    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b202c328-33d4-40f6-a1bc-ec2f6ead0836-istio-token" (OuterVolumeSpecName: "istio-token") pod "b202c328-33d4-40f6-a1bc-ec2f6ead0836" (UID: "b202c328-33d4-40f6-a1bc-ec2f6ead0836"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:10:30.820696 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.820617    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b202c328-33d4-40f6-a1bc-ec2f6ead0836-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b202c328-33d4-40f6-a1bc-ec2f6ead0836" (UID: "b202c328-33d4-40f6-a1bc-ec2f6ead0836"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:10:30.857923 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.857895    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a16895-56cb9799b-9n96c"]
Jul 20 17:10:30.860542 ip-10-0-130-85 kubenswrapper[2577]: W0720 17:10:30.860521    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod161ff40d_6008_4ffe_8e60_0d5b53567e3c.slice/crio-6fff5e23fe5111cb39d8cbd8000488a6c6ef4aed95c09d08ac563cb5bb682744 WatchSource:0}: Error finding container 6fff5e23fe5111cb39d8cbd8000488a6c6ef4aed95c09d08ac563cb5bb682744: Status 404 returned error can't find the container with id 6fff5e23fe5111cb39d8cbd8000488a6c6ef4aed95c09d08ac563cb5bb682744
Jul 20 17:10:30.916860 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.916825    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b202c328-33d4-40f6-a1bc-ec2f6ead0836-istio-data\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:10:30.916860 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.916858    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvbr2\" (UniqueName: \"kubernetes.io/projected/b202c328-33d4-40f6-a1bc-ec2f6ead0836-kube-api-access-jvbr2\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:10:30.916860 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.916869    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b202c328-33d4-40f6-a1bc-ec2f6ead0836-workload-certs\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:10:30.917028 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.916878    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b202c328-33d4-40f6-a1bc-ec2f6ead0836-istio-token\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:10:30.917028 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.916887    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b202c328-33d4-40f6-a1bc-ec2f6ead0836-istiod-ca-cert\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:10:30.917028 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.916895    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b202c328-33d4-40f6-a1bc-ec2f6ead0836-istio-envoy\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:10:30.917028 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:30.916904    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b202c328-33d4-40f6-a1bc-ec2f6ead0836-istio-podinfo\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:10:31.743795 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:31.743713    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"
Jul 20 17:10:31.743795 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:31.743723    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q" event={"ID":"b202c328-33d4-40f6-a1bc-ec2f6ead0836","Type":"ContainerDied","Data":"727ff8e8608c188fd28e788e13abced1001dd3a990ebf0198bfcf03cacf12d97"}
Jul 20 17:10:31.743795 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:31.743773    2577 scope.go:117] "RemoveContainer" containerID="29d3f6cc9974f8c005bcc1975264571ffcaaea737ffcffc1effdaf4c7d60cc58"
Jul 20 17:10:31.745445 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:31.745416    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a16895-56cb9799b-9n96c" event={"ID":"161ff40d-6008-4ffe-8e60-0d5b53567e3c","Type":"ContainerStarted","Data":"85d8a53d8fa34760fdeda53059cf4394db57ec773c045d34bf22862333dd4719"}
Jul 20 17:10:31.745559 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:31.745463    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a16895-56cb9799b-9n96c" event={"ID":"161ff40d-6008-4ffe-8e60-0d5b53567e3c","Type":"ContainerStarted","Data":"6fff5e23fe5111cb39d8cbd8000488a6c6ef4aed95c09d08ac563cb5bb682744"}
Jul 20 17:10:31.763134 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:31.763112    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"]
Jul 20 17:10:31.767685 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:31.767661    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-3ee404-openshift-default-5c47c97db7-gvv7q"]
Jul 20 17:10:31.791061 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:31.791021    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a16895-56cb9799b-9n96c" podStartSLOduration=2.34174242 podStartE2EDuration="2.791007194s" totalImagesPullingTime="449.264774ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:10:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:10:31.789955811 +0000 UTC m=+2455.039449830" watchObservedRunningTime="2026-07-20 17:10:31.791007194 +0000 UTC m=+2455.040501212"
Jul 20 17:10:32.750981 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:32.750949    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a16895-56cb9799b-9n96c"
Jul 20 17:10:33.213172 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:33.213131    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b202c328-33d4-40f6-a1bc-ec2f6ead0836" path="/var/lib/kubelet/pods/b202c328-33d4-40f6-a1bc-ec2f6ead0836/volumes"
Jul 20 17:10:34.230940 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:34.230905    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:34.561057 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:34.561025    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:34.834407 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:34.834311    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"]
Jul 20 17:10:34.838321 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:34.834780    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b202c328-33d4-40f6-a1bc-ec2f6ead0836" containerName="istio-proxy"
Jul 20 17:10:34.838321 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:34.834801    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b202c328-33d4-40f6-a1bc-ec2f6ead0836" containerName="istio-proxy"
Jul 20 17:10:34.838321 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:34.834842    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b202c328-33d4-40f6-a1bc-ec2f6ead0836" containerName="istio-proxy"
Jul 20 17:10:34.838321 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:34.834851    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b202c328-33d4-40f6-a1bc-ec2f6ead0836" containerName="istio-proxy"
Jul 20 17:10:34.838321 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:34.834943    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b202c328-33d4-40f6-a1bc-ec2f6ead0836" containerName="istio-proxy"
Jul 20 17:10:34.856321 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:34.847559    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:34.856321 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:34.855698    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-e142d5-openshift-default-dockercfg-rfhcl\""
Jul 20 17:10:34.864859 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:34.858012    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"]
Jul 20 17:10:34.872336 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:34.869087    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:34.952321 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:34.951701    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9073619-7c5a-46a3-890b-e5fd42e7d35f-istio-envoy\") pod \"e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") " pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:34.952321 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:34.951750    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9073619-7c5a-46a3-890b-e5fd42e7d35f-istio-data\") pod \"e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") " pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:34.952321 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:34.951784    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9073619-7c5a-46a3-890b-e5fd42e7d35f-istio-podinfo\") pod \"e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") " pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:34.952321 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:34.951856    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9073619-7c5a-46a3-890b-e5fd42e7d35f-workload-socket\") pod \"e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") " pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:34.952321 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:34.951884    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9073619-7c5a-46a3-890b-e5fd42e7d35f-credential-socket\") pod \"e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") " pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:34.952321 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:34.951938    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9073619-7c5a-46a3-890b-e5fd42e7d35f-workload-certs\") pod \"e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") " pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:34.952321 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:34.951962    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9073619-7c5a-46a3-890b-e5fd42e7d35f-istio-token\") pod \"e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") " pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:34.952321 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:34.952006    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9073619-7c5a-46a3-890b-e5fd42e7d35f-istiod-ca-cert\") pod \"e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") " pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:34.952321 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:34.952032    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8n4q\" (UniqueName: \"kubernetes.io/projected/f9073619-7c5a-46a3-890b-e5fd42e7d35f-kube-api-access-l8n4q\") pod \"e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") " pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:35.053879 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:35.053627    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9073619-7c5a-46a3-890b-e5fd42e7d35f-workload-socket\") pod \"e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") " pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:35.054133 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:35.053910    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9073619-7c5a-46a3-890b-e5fd42e7d35f-credential-socket\") pod \"e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") " pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:35.054213 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:35.054195    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9073619-7c5a-46a3-890b-e5fd42e7d35f-workload-certs\") pod \"e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") " pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:35.054513 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:35.054484    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9073619-7c5a-46a3-890b-e5fd42e7d35f-istio-token\") pod \"e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") " pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:35.054790 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:35.054766    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9073619-7c5a-46a3-890b-e5fd42e7d35f-istiod-ca-cert\") pod \"e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") " pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:35.055038 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:35.054821    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8n4q\" (UniqueName: \"kubernetes.io/projected/f9073619-7c5a-46a3-890b-e5fd42e7d35f-kube-api-access-l8n4q\") pod \"e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") " pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:35.055325 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:35.055097    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9073619-7c5a-46a3-890b-e5fd42e7d35f-istio-envoy\") pod \"e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") " pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:35.055422 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:35.055349    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9073619-7c5a-46a3-890b-e5fd42e7d35f-istio-data\") pod \"e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") " pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:35.055602 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:35.055580    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9073619-7c5a-46a3-890b-e5fd42e7d35f-istio-podinfo\") pod \"e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") " pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:35.059799 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:35.059770    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9073619-7c5a-46a3-890b-e5fd42e7d35f-workload-certs\") pod \"e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") " pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:35.060103 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:35.060080    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9073619-7c5a-46a3-890b-e5fd42e7d35f-workload-socket\") pod \"e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") " pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:35.060418 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:35.060397    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9073619-7c5a-46a3-890b-e5fd42e7d35f-credential-socket\") pod \"e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") " pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:35.061767 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:35.061742    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9073619-7c5a-46a3-890b-e5fd42e7d35f-istiod-ca-cert\") pod \"e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") " pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:35.062411 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:35.062387    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9073619-7c5a-46a3-890b-e5fd42e7d35f-istio-data\") pod \"e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") " pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:35.064818 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:35.064795    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9073619-7c5a-46a3-890b-e5fd42e7d35f-istio-envoy\") pod \"e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") " pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:35.069239 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:35.069200    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8n4q\" (UniqueName: \"kubernetes.io/projected/f9073619-7c5a-46a3-890b-e5fd42e7d35f-kube-api-access-l8n4q\") pod \"e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") " pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:35.070986 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:35.070758    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9073619-7c5a-46a3-890b-e5fd42e7d35f-istio-token\") pod \"e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") " pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:35.070986 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:35.070950    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9073619-7c5a-46a3-890b-e5fd42e7d35f-istio-podinfo\") pod \"e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") " pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:35.178026 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:35.177936    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:35.257072 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:35.254679    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:35.360679 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:35.358204    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"]
Jul 20 17:10:35.362331 ip-10-0-130-85 kubenswrapper[2577]: W0720 17:10:35.362263    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9073619_7c5a_46a3_890b_e5fd42e7d35f.slice/crio-9c7a37bd10ced21d41e71602a006d13dc5a5b6e8382f569656cb0e8bbb9361b8 WatchSource:0}: Error finding container 9c7a37bd10ced21d41e71602a006d13dc5a5b6e8382f569656cb0e8bbb9361b8: Status 404 returned error can't find the container with id 9c7a37bd10ced21d41e71602a006d13dc5a5b6e8382f569656cb0e8bbb9361b8
Jul 20 17:10:35.365249 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:35.365205    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:10:35.365353 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:35.365330    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:10:35.365406 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:35.365374    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:10:35.769511 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:35.769472    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg" event={"ID":"f9073619-7c5a-46a3-890b-e5fd42e7d35f","Type":"ContainerStarted","Data":"e4ba2da396da97e6e9a1b2eaa39e4c7024930204ecb2809875db5d82347d492d"}
Jul 20 17:10:35.769728 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:35.769524    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg" event={"ID":"f9073619-7c5a-46a3-890b-e5fd42e7d35f","Type":"ContainerStarted","Data":"9c7a37bd10ced21d41e71602a006d13dc5a5b6e8382f569656cb0e8bbb9361b8"}
Jul 20 17:10:35.792122 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:35.792069    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg" podStartSLOduration=1.792052856 podStartE2EDuration="1.792052856s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:10:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:10:35.790056901 +0000 UTC m=+2459.039550927" watchObservedRunningTime="2026-07-20 17:10:35.792052856 +0000 UTC m=+2459.041546876"
Jul 20 17:10:36.178465 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:36.178434    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:36.183346 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:36.183321    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:36.773748 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:36.773711    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:36.774597 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:36.774573    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:37.733946 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:37.733855    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:37.839511 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:37.839474    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:37.983821 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:37.983770    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:38.146354 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:38.146315    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:38.457152 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:38.457088    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:38.558817 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:38.558786    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:38.761552 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:38.761480    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a16895-56cb9799b-9n96c"
Jul 20 17:10:41.798671 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:41.798639    2577 generic.go:356] "Generic (PLEG): container finished" podID="1893d618-2628-4baa-a0de-803c6b15be38" containerID="0f966d33fedf2be859a0d310fc6d20bc4a14c61fa73d7d66a20fe32431b49272" exitCode=0
Jul 20 17:10:41.799077 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:41.798685    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4" event={"ID":"1893d618-2628-4baa-a0de-803c6b15be38","Type":"ContainerDied","Data":"0f966d33fedf2be859a0d310fc6d20bc4a14c61fa73d7d66a20fe32431b49272"}
Jul 20 17:10:41.799077 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:41.799031    2577 scope.go:117] "RemoveContainer" containerID="0f966d33fedf2be859a0d310fc6d20bc4a14c61fa73d7d66a20fe32431b49272"
Jul 20 17:10:41.800039 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:41.800013    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:10:41.800142 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:41.800076    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:10:41.800142 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:41.800110    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:10:42.237096 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:42.237019    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:42.805119 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:42.805084    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4" event={"ID":"1893d618-2628-4baa-a0de-803c6b15be38","Type":"ContainerStarted","Data":"09c665d4873ddd07e12064f7b7544453d667df544c12317b6d26773b43213797"}
Jul 20 17:10:42.943857 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:42.943804    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:42.945276 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:42.945240    2577 patch_prober.go:28] interesting pod/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 20 17:10:42.945447 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:42.945340    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4" podUID="1893d618-2628-4baa-a0de-803c6b15be38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 20 17:10:43.944079 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:43.944040    2577 patch_prober.go:28] interesting pod/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 20 17:10:43.944599 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:43.944108    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4" podUID="1893d618-2628-4baa-a0de-803c6b15be38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 20 17:10:44.727400 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:44.727362    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:44.944134 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:44.944094    2577 patch_prober.go:28] interesting pod/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 20 17:10:44.944648 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:44.944175    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4" podUID="1893d618-2628-4baa-a0de-803c6b15be38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 20 17:10:45.943940 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:45.943889    2577 patch_prober.go:28] interesting pod/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 20 17:10:45.944250 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:45.943965    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4" podUID="1893d618-2628-4baa-a0de-803c6b15be38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 20 17:10:46.944726 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:46.944680    2577 patch_prober.go:28] interesting pod/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 20 17:10:46.945340 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:46.945276    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4" podUID="1893d618-2628-4baa-a0de-803c6b15be38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 20 17:10:47.952701 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:47.952668    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:47.957576 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:47.957551    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:10:51.291157 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:51.291117    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:51.388986 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:51.388946    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:52.815461 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:52.814099    2577 patch_prober.go:28] interesting pod/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 20 17:10:52.815461 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:52.814191    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg" podUID="f9073619-7c5a-46a3-890b-e5fd42e7d35f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 20 17:10:52.853209 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:52.853184    2577 generic.go:356] "Generic (PLEG): container finished" podID="f9073619-7c5a-46a3-890b-e5fd42e7d35f" containerID="e4ba2da396da97e6e9a1b2eaa39e4c7024930204ecb2809875db5d82347d492d" exitCode=0
Jul 20 17:10:52.853352 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:52.853249    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg" event={"ID":"f9073619-7c5a-46a3-890b-e5fd42e7d35f","Type":"ContainerDied","Data":"e4ba2da396da97e6e9a1b2eaa39e4c7024930204ecb2809875db5d82347d492d"}
Jul 20 17:10:52.853636 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:52.853621    2577 scope.go:117] "RemoveContainer" containerID="e4ba2da396da97e6e9a1b2eaa39e4c7024930204ecb2809875db5d82347d492d"
Jul 20 17:10:52.854687 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:52.854656    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:10:52.854758 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:52.854720    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:10:52.854758 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:52.854749    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 20 17:10:53.863117 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:53.863083    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg" event={"ID":"f9073619-7c5a-46a3-890b-e5fd42e7d35f","Type":"ContainerStarted","Data":"7decc791be6dae83e4d2752cc97517a4fb831b506758f68d630961fb5c33d0ae"}
Jul 20 17:10:54.179103 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:54.179007    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:54.179290 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:54.179264    2577 patch_prober.go:28] interesting pod/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 20 17:10:54.179390 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:54.179354    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg" podUID="f9073619-7c5a-46a3-890b-e5fd42e7d35f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 20 17:10:55.178817 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:55.178779    2577 patch_prober.go:28] interesting pod/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 20 17:10:55.179221 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:55.178843    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg" podUID="f9073619-7c5a-46a3-890b-e5fd42e7d35f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 20 17:10:56.179077 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:56.179042    2577 patch_prober.go:28] interesting pod/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 20 17:10:56.179478 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:56.179097    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg" podUID="f9073619-7c5a-46a3-890b-e5fd42e7d35f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 20 17:10:57.178489 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:57.178454    2577 patch_prober.go:28] interesting pod/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 20 17:10:57.178667 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:57.178509    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg" podUID="f9073619-7c5a-46a3-890b-e5fd42e7d35f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 20 17:10:57.927753 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:57.927720    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh"]
Jul 20 17:10:57.928118 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:57.928103    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b202c328-33d4-40f6-a1bc-ec2f6ead0836" containerName="istio-proxy"
Jul 20 17:10:57.930509 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:57.930491    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh"
Jul 20 17:10:57.935212 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:57.935181    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a16895\"/\"openshift-service-ca.crt\""
Jul 20 17:10:57.935212 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:57.935187    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a16895\"/\"auth-test-model-a16895-kserve-self-signed-certs\""
Jul 20 17:10:57.935453 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:57.935209    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a16895\"/\"kube-root-ca.crt\""
Jul 20 17:10:57.935788 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:57.935770    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a16895\"/\"default-dockercfg-z4cmd\""
Jul 20 17:10:57.947208 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:57.947188    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh"]
Jul 20 17:10:58.040649 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:58.040616    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:58.076194 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:58.076173    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e894668-da6b-47d3-9dee-fe005bcf15d3-dshm\") pod \"auth-test-model-a16895-kserve-bd9f456b7-nprfh\" (UID: \"4e894668-da6b-47d3-9dee-fe005bcf15d3\") " pod="ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh"
Jul 20 17:10:58.076352 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:58.076216    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e894668-da6b-47d3-9dee-fe005bcf15d3-home\") pod \"auth-test-model-a16895-kserve-bd9f456b7-nprfh\" (UID: \"4e894668-da6b-47d3-9dee-fe005bcf15d3\") " pod="ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh"
Jul 20 17:10:58.076352 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:58.076234    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7796v\" (UniqueName: \"kubernetes.io/projected/4e894668-da6b-47d3-9dee-fe005bcf15d3-kube-api-access-7796v\") pod \"auth-test-model-a16895-kserve-bd9f456b7-nprfh\" (UID: \"4e894668-da6b-47d3-9dee-fe005bcf15d3\") " pod="ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh"
Jul 20 17:10:58.076352 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:58.076336    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e894668-da6b-47d3-9dee-fe005bcf15d3-model-cache\") pod \"auth-test-model-a16895-kserve-bd9f456b7-nprfh\" (UID: \"4e894668-da6b-47d3-9dee-fe005bcf15d3\") " pod="ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh"
Jul 20 17:10:58.076529 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:58.076367    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e894668-da6b-47d3-9dee-fe005bcf15d3-tmp-dir\") pod \"auth-test-model-a16895-kserve-bd9f456b7-nprfh\" (UID: \"4e894668-da6b-47d3-9dee-fe005bcf15d3\") " pod="ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh"
Jul 20 17:10:58.076529 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:58.076453    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e894668-da6b-47d3-9dee-fe005bcf15d3-tls-certs\") pod \"auth-test-model-a16895-kserve-bd9f456b7-nprfh\" (UID: \"4e894668-da6b-47d3-9dee-fe005bcf15d3\") " pod="ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh"
Jul 20 17:10:58.177355 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:58.177328    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e894668-da6b-47d3-9dee-fe005bcf15d3-home\") pod \"auth-test-model-a16895-kserve-bd9f456b7-nprfh\" (UID: \"4e894668-da6b-47d3-9dee-fe005bcf15d3\") " pod="ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh"
Jul 20 17:10:58.177519 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:58.177361    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7796v\" (UniqueName: \"kubernetes.io/projected/4e894668-da6b-47d3-9dee-fe005bcf15d3-kube-api-access-7796v\") pod \"auth-test-model-a16895-kserve-bd9f456b7-nprfh\" (UID: \"4e894668-da6b-47d3-9dee-fe005bcf15d3\") " pod="ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh"
Jul 20 17:10:58.177519 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:58.177388    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e894668-da6b-47d3-9dee-fe005bcf15d3-model-cache\") pod \"auth-test-model-a16895-kserve-bd9f456b7-nprfh\" (UID: \"4e894668-da6b-47d3-9dee-fe005bcf15d3\") " pod="ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh"
Jul 20 17:10:58.177519 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:58.177404    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e894668-da6b-47d3-9dee-fe005bcf15d3-tmp-dir\") pod \"auth-test-model-a16895-kserve-bd9f456b7-nprfh\" (UID: \"4e894668-da6b-47d3-9dee-fe005bcf15d3\") " pod="ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh"
Jul 20 17:10:58.177519 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:58.177439    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e894668-da6b-47d3-9dee-fe005bcf15d3-tls-certs\") pod \"auth-test-model-a16895-kserve-bd9f456b7-nprfh\" (UID: \"4e894668-da6b-47d3-9dee-fe005bcf15d3\") " pod="ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh"
Jul 20 17:10:58.177519 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:58.177481    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e894668-da6b-47d3-9dee-fe005bcf15d3-dshm\") pod \"auth-test-model-a16895-kserve-bd9f456b7-nprfh\" (UID: \"4e894668-da6b-47d3-9dee-fe005bcf15d3\") " pod="ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh"
Jul 20 17:10:58.177842 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:58.177788    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e894668-da6b-47d3-9dee-fe005bcf15d3-model-cache\") pod \"auth-test-model-a16895-kserve-bd9f456b7-nprfh\" (UID: \"4e894668-da6b-47d3-9dee-fe005bcf15d3\") " pod="ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh"
Jul 20 17:10:58.177842 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:58.177801    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e894668-da6b-47d3-9dee-fe005bcf15d3-home\") pod \"auth-test-model-a16895-kserve-bd9f456b7-nprfh\" (UID: \"4e894668-da6b-47d3-9dee-fe005bcf15d3\") " pod="ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh"
Jul 20 17:10:58.177947 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:58.177861    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e894668-da6b-47d3-9dee-fe005bcf15d3-tmp-dir\") pod \"auth-test-model-a16895-kserve-bd9f456b7-nprfh\" (UID: \"4e894668-da6b-47d3-9dee-fe005bcf15d3\") " pod="ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh"
Jul 20 17:10:58.179827 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:58.179807    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e894668-da6b-47d3-9dee-fe005bcf15d3-dshm\") pod \"auth-test-model-a16895-kserve-bd9f456b7-nprfh\" (UID: \"4e894668-da6b-47d3-9dee-fe005bcf15d3\") " pod="ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh"
Jul 20 17:10:58.180226 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:58.180201    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e894668-da6b-47d3-9dee-fe005bcf15d3-tls-certs\") pod \"auth-test-model-a16895-kserve-bd9f456b7-nprfh\" (UID: \"4e894668-da6b-47d3-9dee-fe005bcf15d3\") " pod="ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh"
Jul 20 17:10:58.181639 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:58.181622    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:58.181917 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:58.181901    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:58.182580 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:58.182563    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:10:58.186527 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:58.186506    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7796v\" (UniqueName: \"kubernetes.io/projected/4e894668-da6b-47d3-9dee-fe005bcf15d3-kube-api-access-7796v\") pod \"auth-test-model-a16895-kserve-bd9f456b7-nprfh\" (UID: \"4e894668-da6b-47d3-9dee-fe005bcf15d3\") " pod="ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh"
Jul 20 17:10:58.239455 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:58.239429    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh"
Jul 20 17:10:58.367867 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:58.367756    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh"]
Jul 20 17:10:58.887348 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:58.887314    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh" event={"ID":"4e894668-da6b-47d3-9dee-fe005bcf15d3","Type":"ContainerStarted","Data":"5471d812b73000864465921eaa96423b6f2c2539567cdc2c7a3f8b3438649d95"}
Jul 20 17:10:58.887348 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:58.887356    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh" event={"ID":"4e894668-da6b-47d3-9dee-fe005bcf15d3","Type":"ContainerStarted","Data":"f329b17a73906761d7c25868741c20bc3f2d933e3562c6ebdc68ecd3d12e378c"}
Jul 20 17:10:58.905565 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:58.905521    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh" podStartSLOduration=1.9055059810000001 podStartE2EDuration="1.905505981s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:10:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:10:58.90475057 +0000 UTC m=+2482.154244586" watchObservedRunningTime="2026-07-20 17:10:58.905505981 +0000 UTC m=+2482.155000039"
Jul 20 17:10:59.503881 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:59.503842    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:10:59.604479 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:10:59.604447    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:11:08.240374 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:08.240329    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh"
Jul 20 17:11:08.240374 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:08.240386    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh"
Jul 20 17:11:08.252399 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:08.252373    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh"
Jul 20 17:11:08.944176 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:08.944141    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh"
Jul 20 17:11:35.505414 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:35.505337    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg"]
Jul 20 17:11:35.509565 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:35.509544    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg"
Jul 20 17:11:35.513017 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:35.512991    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e142d5\"/\"auth-test-model-e142d5-kserve-self-signed-certs\""
Jul 20 17:11:35.514088 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:35.514070    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e142d5\"/\"openshift-service-ca.crt\""
Jul 20 17:11:35.514174 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:35.514118    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e142d5\"/\"kube-root-ca.crt\""
Jul 20 17:11:35.514238 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:35.514127    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e142d5\"/\"default-dockercfg-4wd97\""
Jul 20 17:11:35.523875 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:35.523854    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg"]
Jul 20 17:11:35.575005 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:35.574965    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhkl7\" (UniqueName: \"kubernetes.io/projected/baed7fdf-e465-4689-83cf-37a53545d9be-kube-api-access-fhkl7\") pod \"auth-test-model-e142d5-kserve-56488bc67f-nphqg\" (UID: \"baed7fdf-e465-4689-83cf-37a53545d9be\") " pod="ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg"
Jul 20 17:11:35.575163 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:35.575023    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/baed7fdf-e465-4689-83cf-37a53545d9be-dshm\") pod \"auth-test-model-e142d5-kserve-56488bc67f-nphqg\" (UID: \"baed7fdf-e465-4689-83cf-37a53545d9be\") " pod="ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg"
Jul 20 17:11:35.575163 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:35.575060    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/baed7fdf-e465-4689-83cf-37a53545d9be-tls-certs\") pod \"auth-test-model-e142d5-kserve-56488bc67f-nphqg\" (UID: \"baed7fdf-e465-4689-83cf-37a53545d9be\") " pod="ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg"
Jul 20 17:11:35.575163 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:35.575142    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/baed7fdf-e465-4689-83cf-37a53545d9be-model-cache\") pod \"auth-test-model-e142d5-kserve-56488bc67f-nphqg\" (UID: \"baed7fdf-e465-4689-83cf-37a53545d9be\") " pod="ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg"
Jul 20 17:11:35.575288 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:35.575166    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/baed7fdf-e465-4689-83cf-37a53545d9be-tmp-dir\") pod \"auth-test-model-e142d5-kserve-56488bc67f-nphqg\" (UID: \"baed7fdf-e465-4689-83cf-37a53545d9be\") " pod="ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg"
Jul 20 17:11:35.575288 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:35.575184    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/baed7fdf-e465-4689-83cf-37a53545d9be-home\") pod \"auth-test-model-e142d5-kserve-56488bc67f-nphqg\" (UID: \"baed7fdf-e465-4689-83cf-37a53545d9be\") " pod="ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg"
Jul 20 17:11:35.675811 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:35.675780    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/baed7fdf-e465-4689-83cf-37a53545d9be-model-cache\") pod \"auth-test-model-e142d5-kserve-56488bc67f-nphqg\" (UID: \"baed7fdf-e465-4689-83cf-37a53545d9be\") " pod="ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg"
Jul 20 17:11:35.675811 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:35.675813    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/baed7fdf-e465-4689-83cf-37a53545d9be-tmp-dir\") pod \"auth-test-model-e142d5-kserve-56488bc67f-nphqg\" (UID: \"baed7fdf-e465-4689-83cf-37a53545d9be\") " pod="ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg"
Jul 20 17:11:35.675999 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:35.675841    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/baed7fdf-e465-4689-83cf-37a53545d9be-home\") pod \"auth-test-model-e142d5-kserve-56488bc67f-nphqg\" (UID: \"baed7fdf-e465-4689-83cf-37a53545d9be\") " pod="ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg"
Jul 20 17:11:35.675999 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:35.675898    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhkl7\" (UniqueName: \"kubernetes.io/projected/baed7fdf-e465-4689-83cf-37a53545d9be-kube-api-access-fhkl7\") pod \"auth-test-model-e142d5-kserve-56488bc67f-nphqg\" (UID: \"baed7fdf-e465-4689-83cf-37a53545d9be\") " pod="ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg"
Jul 20 17:11:35.675999 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:35.675943    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/baed7fdf-e465-4689-83cf-37a53545d9be-dshm\") pod \"auth-test-model-e142d5-kserve-56488bc67f-nphqg\" (UID: \"baed7fdf-e465-4689-83cf-37a53545d9be\") " pod="ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg"
Jul 20 17:11:35.675999 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:35.675969    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/baed7fdf-e465-4689-83cf-37a53545d9be-tls-certs\") pod \"auth-test-model-e142d5-kserve-56488bc67f-nphqg\" (UID: \"baed7fdf-e465-4689-83cf-37a53545d9be\") " pod="ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg"
Jul 20 17:11:35.676247 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:35.676225    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/baed7fdf-e465-4689-83cf-37a53545d9be-tmp-dir\") pod \"auth-test-model-e142d5-kserve-56488bc67f-nphqg\" (UID: \"baed7fdf-e465-4689-83cf-37a53545d9be\") " pod="ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg"
Jul 20 17:11:35.676247 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:35.676242    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/baed7fdf-e465-4689-83cf-37a53545d9be-home\") pod \"auth-test-model-e142d5-kserve-56488bc67f-nphqg\" (UID: \"baed7fdf-e465-4689-83cf-37a53545d9be\") " pod="ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg"
Jul 20 17:11:35.676426 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:35.676287    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/baed7fdf-e465-4689-83cf-37a53545d9be-model-cache\") pod \"auth-test-model-e142d5-kserve-56488bc67f-nphqg\" (UID: \"baed7fdf-e465-4689-83cf-37a53545d9be\") " pod="ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg"
Jul 20 17:11:35.678170 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:35.678151    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/baed7fdf-e465-4689-83cf-37a53545d9be-dshm\") pod \"auth-test-model-e142d5-kserve-56488bc67f-nphqg\" (UID: \"baed7fdf-e465-4689-83cf-37a53545d9be\") " pod="ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg"
Jul 20 17:11:35.678452 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:35.678434    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/baed7fdf-e465-4689-83cf-37a53545d9be-tls-certs\") pod \"auth-test-model-e142d5-kserve-56488bc67f-nphqg\" (UID: \"baed7fdf-e465-4689-83cf-37a53545d9be\") " pod="ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg"
Jul 20 17:11:35.684844 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:35.684819    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhkl7\" (UniqueName: \"kubernetes.io/projected/baed7fdf-e465-4689-83cf-37a53545d9be-kube-api-access-fhkl7\") pod \"auth-test-model-e142d5-kserve-56488bc67f-nphqg\" (UID: \"baed7fdf-e465-4689-83cf-37a53545d9be\") " pod="ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg"
Jul 20 17:11:35.735482 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:35.735452    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:11:35.819644 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:35.819621    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg"
Jul 20 17:11:35.951464 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:35.951440    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg"]
Jul 20 17:11:35.953867 ip-10-0-130-85 kubenswrapper[2577]: W0720 17:11:35.953840    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbaed7fdf_e465_4689_83cf_37a53545d9be.slice/crio-10fead80026c207bac41fe4b37eab84599b65f102a6cd8b89579bd6dd5c183a2 WatchSource:0}: Error finding container 10fead80026c207bac41fe4b37eab84599b65f102a6cd8b89579bd6dd5c183a2: Status 404 returned error can't find the container with id 10fead80026c207bac41fe4b37eab84599b65f102a6cd8b89579bd6dd5c183a2
Jul 20 17:11:36.060825 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:36.060798    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg" event={"ID":"baed7fdf-e465-4689-83cf-37a53545d9be","Type":"ContainerStarted","Data":"f4aedbb5571ff99e0f63e07dd5033182e94147282247ac3015b545bf367963ef"}
Jul 20 17:11:36.060974 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:36.060840    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg" event={"ID":"baed7fdf-e465-4689-83cf-37a53545d9be","Type":"ContainerStarted","Data":"10fead80026c207bac41fe4b37eab84599b65f102a6cd8b89579bd6dd5c183a2"}
Jul 20 17:11:36.081968 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:36.081887    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg" podStartSLOduration=1.081873988 podStartE2EDuration="1.081873988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:11:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:11:36.079503003 +0000 UTC m=+2519.328997034" watchObservedRunningTime="2026-07-20 17:11:36.081873988 +0000 UTC m=+2519.331368007"
Jul 20 17:11:45.820511 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:45.820476    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg"
Jul 20 17:11:45.820926 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:45.820752    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg"
Jul 20 17:11:45.832887 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:45.832857    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg"
Jul 20 17:11:46.118472 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:11:46.118400    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg"
Jul 20 17:12:52.098405 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:12:52.098366    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:12:52.165402 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:12:52.165367    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:12:54.644652 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:12:54.644609    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:13:40.494147 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:40.494111    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh"]
Jul 20 17:13:40.522807 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:40.522782    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:13:41.485346 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:41.485314    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg"]
Jul 20 17:13:42.168212 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:42.168177    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:13:55.513543 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:55.513492    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh" podUID="4e894668-da6b-47d3-9dee-fe005bcf15d3" containerName="main" containerID="cri-o://5471d812b73000864465921eaa96423b6f2c2539567cdc2c7a3f8b3438649d95" gracePeriod=45
Jul 20 17:13:55.685423 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:55.685395    2577 generic.go:356] "Generic (PLEG): container finished" podID="4e894668-da6b-47d3-9dee-fe005bcf15d3" containerID="5471d812b73000864465921eaa96423b6f2c2539567cdc2c7a3f8b3438649d95" exitCode=0
Jul 20 17:13:55.685550 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:55.685464    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh" event={"ID":"4e894668-da6b-47d3-9dee-fe005bcf15d3","Type":"ContainerDied","Data":"5471d812b73000864465921eaa96423b6f2c2539567cdc2c7a3f8b3438649d95"}
Jul 20 17:13:55.764708 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:55.764650    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh"
Jul 20 17:13:55.882039 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:55.882009    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e894668-da6b-47d3-9dee-fe005bcf15d3-dshm\") pod \"4e894668-da6b-47d3-9dee-fe005bcf15d3\" (UID: \"4e894668-da6b-47d3-9dee-fe005bcf15d3\") "
Jul 20 17:13:55.882202 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:55.882066    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e894668-da6b-47d3-9dee-fe005bcf15d3-tls-certs\") pod \"4e894668-da6b-47d3-9dee-fe005bcf15d3\" (UID: \"4e894668-da6b-47d3-9dee-fe005bcf15d3\") "
Jul 20 17:13:55.882202 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:55.882110    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e894668-da6b-47d3-9dee-fe005bcf15d3-tmp-dir\") pod \"4e894668-da6b-47d3-9dee-fe005bcf15d3\" (UID: \"4e894668-da6b-47d3-9dee-fe005bcf15d3\") "
Jul 20 17:13:55.882202 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:55.882158    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e894668-da6b-47d3-9dee-fe005bcf15d3-home\") pod \"4e894668-da6b-47d3-9dee-fe005bcf15d3\" (UID: \"4e894668-da6b-47d3-9dee-fe005bcf15d3\") "
Jul 20 17:13:55.882202 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:55.882193    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e894668-da6b-47d3-9dee-fe005bcf15d3-model-cache\") pod \"4e894668-da6b-47d3-9dee-fe005bcf15d3\" (UID: \"4e894668-da6b-47d3-9dee-fe005bcf15d3\") "
Jul 20 17:13:55.882429 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:55.882276    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7796v\" (UniqueName: \"kubernetes.io/projected/4e894668-da6b-47d3-9dee-fe005bcf15d3-kube-api-access-7796v\") pod \"4e894668-da6b-47d3-9dee-fe005bcf15d3\" (UID: \"4e894668-da6b-47d3-9dee-fe005bcf15d3\") "
Jul 20 17:13:55.882491 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:55.882385    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e894668-da6b-47d3-9dee-fe005bcf15d3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4e894668-da6b-47d3-9dee-fe005bcf15d3" (UID: "4e894668-da6b-47d3-9dee-fe005bcf15d3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:13:55.882549 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:55.882482    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e894668-da6b-47d3-9dee-fe005bcf15d3-home" (OuterVolumeSpecName: "home") pod "4e894668-da6b-47d3-9dee-fe005bcf15d3" (UID: "4e894668-da6b-47d3-9dee-fe005bcf15d3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:13:55.882604 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:55.882549    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e894668-da6b-47d3-9dee-fe005bcf15d3-model-cache" (OuterVolumeSpecName: "model-cache") pod "4e894668-da6b-47d3-9dee-fe005bcf15d3" (UID: "4e894668-da6b-47d3-9dee-fe005bcf15d3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:13:55.882751 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:55.882730    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e894668-da6b-47d3-9dee-fe005bcf15d3-tmp-dir\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:13:55.882819 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:55.882756    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e894668-da6b-47d3-9dee-fe005bcf15d3-home\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:13:55.882819 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:55.882769    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e894668-da6b-47d3-9dee-fe005bcf15d3-model-cache\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:13:55.884284 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:55.884237    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e894668-da6b-47d3-9dee-fe005bcf15d3-dshm" (OuterVolumeSpecName: "dshm") pod "4e894668-da6b-47d3-9dee-fe005bcf15d3" (UID: "4e894668-da6b-47d3-9dee-fe005bcf15d3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:13:55.884403 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:55.884371    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e894668-da6b-47d3-9dee-fe005bcf15d3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4e894668-da6b-47d3-9dee-fe005bcf15d3" (UID: "4e894668-da6b-47d3-9dee-fe005bcf15d3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:13:55.884403 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:55.884365    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e894668-da6b-47d3-9dee-fe005bcf15d3-kube-api-access-7796v" (OuterVolumeSpecName: "kube-api-access-7796v") pod "4e894668-da6b-47d3-9dee-fe005bcf15d3" (UID: "4e894668-da6b-47d3-9dee-fe005bcf15d3"). InnerVolumeSpecName "kube-api-access-7796v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:13:55.984100 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:55.984072    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e894668-da6b-47d3-9dee-fe005bcf15d3-tls-certs\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:13:55.984100 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:55.984097    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7796v\" (UniqueName: \"kubernetes.io/projected/4e894668-da6b-47d3-9dee-fe005bcf15d3-kube-api-access-7796v\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:13:55.984100 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:55.984106    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e894668-da6b-47d3-9dee-fe005bcf15d3-dshm\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:13:56.504783 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:56.504741    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg" podUID="baed7fdf-e465-4689-83cf-37a53545d9be" containerName="main" containerID="cri-o://f4aedbb5571ff99e0f63e07dd5033182e94147282247ac3015b545bf367963ef" gracePeriod=45
Jul 20 17:13:56.692369 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:56.692333    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh" event={"ID":"4e894668-da6b-47d3-9dee-fe005bcf15d3","Type":"ContainerDied","Data":"f329b17a73906761d7c25868741c20bc3f2d933e3562c6ebdc68ecd3d12e378c"}
Jul 20 17:13:56.692791 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:56.692344    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh"
Jul 20 17:13:56.692791 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:56.692390    2577 scope.go:117] "RemoveContainer" containerID="5471d812b73000864465921eaa96423b6f2c2539567cdc2c7a3f8b3438649d95"
Jul 20 17:13:56.694283 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:56.694256    2577 generic.go:356] "Generic (PLEG): container finished" podID="baed7fdf-e465-4689-83cf-37a53545d9be" containerID="f4aedbb5571ff99e0f63e07dd5033182e94147282247ac3015b545bf367963ef" exitCode=0
Jul 20 17:13:56.694421 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:56.694315    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg" event={"ID":"baed7fdf-e465-4689-83cf-37a53545d9be","Type":"ContainerDied","Data":"f4aedbb5571ff99e0f63e07dd5033182e94147282247ac3015b545bf367963ef"}
Jul 20 17:13:56.715361 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:56.715336    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh"]
Jul 20 17:13:56.721770 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:56.721747    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-a16895/auth-test-model-a16895-kserve-bd9f456b7-nprfh"]
Jul 20 17:13:56.745850 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:56.745831    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg"
Jul 20 17:13:56.892496 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:56.892472    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/baed7fdf-e465-4689-83cf-37a53545d9be-tmp-dir\") pod \"baed7fdf-e465-4689-83cf-37a53545d9be\" (UID: \"baed7fdf-e465-4689-83cf-37a53545d9be\") "
Jul 20 17:13:56.892645 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:56.892503    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/baed7fdf-e465-4689-83cf-37a53545d9be-tls-certs\") pod \"baed7fdf-e465-4689-83cf-37a53545d9be\" (UID: \"baed7fdf-e465-4689-83cf-37a53545d9be\") "
Jul 20 17:13:56.892645 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:56.892547    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhkl7\" (UniqueName: \"kubernetes.io/projected/baed7fdf-e465-4689-83cf-37a53545d9be-kube-api-access-fhkl7\") pod \"baed7fdf-e465-4689-83cf-37a53545d9be\" (UID: \"baed7fdf-e465-4689-83cf-37a53545d9be\") "
Jul 20 17:13:56.892645 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:56.892566    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/baed7fdf-e465-4689-83cf-37a53545d9be-model-cache\") pod \"baed7fdf-e465-4689-83cf-37a53545d9be\" (UID: \"baed7fdf-e465-4689-83cf-37a53545d9be\") "
Jul 20 17:13:56.892645 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:56.892589    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/baed7fdf-e465-4689-83cf-37a53545d9be-home\") pod \"baed7fdf-e465-4689-83cf-37a53545d9be\" (UID: \"baed7fdf-e465-4689-83cf-37a53545d9be\") "
Jul 20 17:13:56.892870 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:56.892646    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/baed7fdf-e465-4689-83cf-37a53545d9be-dshm\") pod \"baed7fdf-e465-4689-83cf-37a53545d9be\" (UID: \"baed7fdf-e465-4689-83cf-37a53545d9be\") "
Jul 20 17:13:56.892870 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:56.892775    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/baed7fdf-e465-4689-83cf-37a53545d9be-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "baed7fdf-e465-4689-83cf-37a53545d9be" (UID: "baed7fdf-e465-4689-83cf-37a53545d9be"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:13:56.892955 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:56.892869    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/baed7fdf-e465-4689-83cf-37a53545d9be-model-cache" (OuterVolumeSpecName: "model-cache") pod "baed7fdf-e465-4689-83cf-37a53545d9be" (UID: "baed7fdf-e465-4689-83cf-37a53545d9be"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:13:56.892955 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:56.892887    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/baed7fdf-e465-4689-83cf-37a53545d9be-home" (OuterVolumeSpecName: "home") pod "baed7fdf-e465-4689-83cf-37a53545d9be" (UID: "baed7fdf-e465-4689-83cf-37a53545d9be"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:13:56.893025 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:56.893001    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/baed7fdf-e465-4689-83cf-37a53545d9be-tmp-dir\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:13:56.893025 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:56.893015    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/baed7fdf-e465-4689-83cf-37a53545d9be-model-cache\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:13:56.893101 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:56.893028    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/baed7fdf-e465-4689-83cf-37a53545d9be-home\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:13:56.894819 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:56.894788    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/baed7fdf-e465-4689-83cf-37a53545d9be-dshm" (OuterVolumeSpecName: "dshm") pod "baed7fdf-e465-4689-83cf-37a53545d9be" (UID: "baed7fdf-e465-4689-83cf-37a53545d9be"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:13:56.894911 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:56.894837    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/baed7fdf-e465-4689-83cf-37a53545d9be-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "baed7fdf-e465-4689-83cf-37a53545d9be" (UID: "baed7fdf-e465-4689-83cf-37a53545d9be"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:13:56.894911 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:56.894901    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/baed7fdf-e465-4689-83cf-37a53545d9be-kube-api-access-fhkl7" (OuterVolumeSpecName: "kube-api-access-fhkl7") pod "baed7fdf-e465-4689-83cf-37a53545d9be" (UID: "baed7fdf-e465-4689-83cf-37a53545d9be"). InnerVolumeSpecName "kube-api-access-fhkl7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:13:56.993650 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:56.993625    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/baed7fdf-e465-4689-83cf-37a53545d9be-dshm\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:13:56.993650 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:56.993647    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/baed7fdf-e465-4689-83cf-37a53545d9be-tls-certs\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:13:56.993786 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:56.993661    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhkl7\" (UniqueName: \"kubernetes.io/projected/baed7fdf-e465-4689-83cf-37a53545d9be-kube-api-access-fhkl7\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:13:57.215007 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:57.212971    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e894668-da6b-47d3-9dee-fe005bcf15d3" path="/var/lib/kubelet/pods/4e894668-da6b-47d3-9dee-fe005bcf15d3/volumes"
Jul 20 17:13:57.700600 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:57.700563    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg" event={"ID":"baed7fdf-e465-4689-83cf-37a53545d9be","Type":"ContainerDied","Data":"10fead80026c207bac41fe4b37eab84599b65f102a6cd8b89579bd6dd5c183a2"}
Jul 20 17:13:57.700600 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:57.700594    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg"
Jul 20 17:13:57.701079 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:57.700613    2577 scope.go:117] "RemoveContainer" containerID="f4aedbb5571ff99e0f63e07dd5033182e94147282247ac3015b545bf367963ef"
Jul 20 17:13:57.719664 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:57.719637    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg"]
Jul 20 17:13:57.728698 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:57.728670    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-e142d5/auth-test-model-e142d5-kserve-56488bc67f-nphqg"]
Jul 20 17:13:59.212183 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:13:59.212151    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="baed7fdf-e465-4689-83cf-37a53545d9be" path="/var/lib/kubelet/pods/baed7fdf-e465-4689-83cf-37a53545d9be/volumes"
Jul 20 17:14:03.537719 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:03.537687    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw"]
Jul 20 17:14:03.538077 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:03.537998    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e894668-da6b-47d3-9dee-fe005bcf15d3" containerName="main"
Jul 20 17:14:03.538077 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:03.538008    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e894668-da6b-47d3-9dee-fe005bcf15d3" containerName="main"
Jul 20 17:14:03.538077 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:03.538017    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="baed7fdf-e465-4689-83cf-37a53545d9be" containerName="main"
Jul 20 17:14:03.538077 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:03.538023    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="baed7fdf-e465-4689-83cf-37a53545d9be" containerName="main"
Jul 20 17:14:03.538203 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:03.538084    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e894668-da6b-47d3-9dee-fe005bcf15d3" containerName="main"
Jul 20 17:14:03.538203 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:03.538096    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="baed7fdf-e465-4689-83cf-37a53545d9be" containerName="main"
Jul 20 17:14:03.542628 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:03.542613    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw"
Jul 20 17:14:03.545323 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:03.545288    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a16895\"/\"kube-root-ca.crt\""
Jul 20 17:14:03.546311 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:03.546277    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a16895\"/\"openshift-service-ca.crt\""
Jul 20 17:14:03.546396 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:03.546353    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a16895\"/\"default-dockercfg-z4cmd\""
Jul 20 17:14:03.547068 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:03.547045    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a16895\"/\"sub-test-model-a16895-kserve-self-signed-certs\""
Jul 20 17:14:03.554825 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:03.554805    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw"]
Jul 20 17:14:03.619007 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:03.618973    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:14:03.644505 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:03.644477    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e9de5be-5e31-4a00-be51-4ca60d36a8af-home\") pod \"sub-test-model-a16895-kserve-696b7f75f5-kf6lw\" (UID: \"2e9de5be-5e31-4a00-be51-4ca60d36a8af\") " pod="ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw"
Jul 20 17:14:03.644645 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:03.644508    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e9de5be-5e31-4a00-be51-4ca60d36a8af-dshm\") pod \"sub-test-model-a16895-kserve-696b7f75f5-kf6lw\" (UID: \"2e9de5be-5e31-4a00-be51-4ca60d36a8af\") " pod="ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw"
Jul 20 17:14:03.644645 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:03.644531    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e9de5be-5e31-4a00-be51-4ca60d36a8af-tmp-dir\") pod \"sub-test-model-a16895-kserve-696b7f75f5-kf6lw\" (UID: \"2e9de5be-5e31-4a00-be51-4ca60d36a8af\") " pod="ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw"
Jul 20 17:14:03.644645 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:03.644549    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e9de5be-5e31-4a00-be51-4ca60d36a8af-model-cache\") pod \"sub-test-model-a16895-kserve-696b7f75f5-kf6lw\" (UID: \"2e9de5be-5e31-4a00-be51-4ca60d36a8af\") " pod="ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw"
Jul 20 17:14:03.644645 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:03.644620    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wt7zb\" (UniqueName: \"kubernetes.io/projected/2e9de5be-5e31-4a00-be51-4ca60d36a8af-kube-api-access-wt7zb\") pod \"sub-test-model-a16895-kserve-696b7f75f5-kf6lw\" (UID: \"2e9de5be-5e31-4a00-be51-4ca60d36a8af\") " pod="ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw"
Jul 20 17:14:03.644799 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:03.644656    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e9de5be-5e31-4a00-be51-4ca60d36a8af-tls-certs\") pod \"sub-test-model-a16895-kserve-696b7f75f5-kf6lw\" (UID: \"2e9de5be-5e31-4a00-be51-4ca60d36a8af\") " pod="ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw"
Jul 20 17:14:03.745929 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:03.745895    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e9de5be-5e31-4a00-be51-4ca60d36a8af-model-cache\") pod \"sub-test-model-a16895-kserve-696b7f75f5-kf6lw\" (UID: \"2e9de5be-5e31-4a00-be51-4ca60d36a8af\") " pod="ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw"
Jul 20 17:14:03.746146 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:03.745941    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt7zb\" (UniqueName: \"kubernetes.io/projected/2e9de5be-5e31-4a00-be51-4ca60d36a8af-kube-api-access-wt7zb\") pod \"sub-test-model-a16895-kserve-696b7f75f5-kf6lw\" (UID: \"2e9de5be-5e31-4a00-be51-4ca60d36a8af\") " pod="ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw"
Jul 20 17:14:03.746146 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:03.745961    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e9de5be-5e31-4a00-be51-4ca60d36a8af-tls-certs\") pod \"sub-test-model-a16895-kserve-696b7f75f5-kf6lw\" (UID: \"2e9de5be-5e31-4a00-be51-4ca60d36a8af\") " pod="ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw"
Jul 20 17:14:03.746146 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:03.746033    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e9de5be-5e31-4a00-be51-4ca60d36a8af-home\") pod \"sub-test-model-a16895-kserve-696b7f75f5-kf6lw\" (UID: \"2e9de5be-5e31-4a00-be51-4ca60d36a8af\") " pod="ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw"
Jul 20 17:14:03.746146 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:03.746056    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e9de5be-5e31-4a00-be51-4ca60d36a8af-dshm\") pod \"sub-test-model-a16895-kserve-696b7f75f5-kf6lw\" (UID: \"2e9de5be-5e31-4a00-be51-4ca60d36a8af\") " pod="ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw"
Jul 20 17:14:03.746404 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:03.746185    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e9de5be-5e31-4a00-be51-4ca60d36a8af-tmp-dir\") pod \"sub-test-model-a16895-kserve-696b7f75f5-kf6lw\" (UID: \"2e9de5be-5e31-4a00-be51-4ca60d36a8af\") " pod="ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw"
Jul 20 17:14:03.746404 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:03.746353    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e9de5be-5e31-4a00-be51-4ca60d36a8af-model-cache\") pod \"sub-test-model-a16895-kserve-696b7f75f5-kf6lw\" (UID: \"2e9de5be-5e31-4a00-be51-4ca60d36a8af\") " pod="ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw"
Jul 20 17:14:03.746522 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:03.746446    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e9de5be-5e31-4a00-be51-4ca60d36a8af-home\") pod \"sub-test-model-a16895-kserve-696b7f75f5-kf6lw\" (UID: \"2e9de5be-5e31-4a00-be51-4ca60d36a8af\") " pod="ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw"
Jul 20 17:14:03.746637 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:03.746618    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e9de5be-5e31-4a00-be51-4ca60d36a8af-tmp-dir\") pod \"sub-test-model-a16895-kserve-696b7f75f5-kf6lw\" (UID: \"2e9de5be-5e31-4a00-be51-4ca60d36a8af\") " pod="ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw"
Jul 20 17:14:03.748312 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:03.748281    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e9de5be-5e31-4a00-be51-4ca60d36a8af-dshm\") pod \"sub-test-model-a16895-kserve-696b7f75f5-kf6lw\" (UID: \"2e9de5be-5e31-4a00-be51-4ca60d36a8af\") " pod="ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw"
Jul 20 17:14:03.748512 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:03.748494    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e9de5be-5e31-4a00-be51-4ca60d36a8af-tls-certs\") pod \"sub-test-model-a16895-kserve-696b7f75f5-kf6lw\" (UID: \"2e9de5be-5e31-4a00-be51-4ca60d36a8af\") " pod="ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw"
Jul 20 17:14:03.755405 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:03.755383    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wt7zb\" (UniqueName: \"kubernetes.io/projected/2e9de5be-5e31-4a00-be51-4ca60d36a8af-kube-api-access-wt7zb\") pod \"sub-test-model-a16895-kserve-696b7f75f5-kf6lw\" (UID: \"2e9de5be-5e31-4a00-be51-4ca60d36a8af\") " pod="ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw"
Jul 20 17:14:03.851525 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:03.851485    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw"
Jul 20 17:14:03.981191 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:03.981148    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw"]
Jul 20 17:14:03.986309 ip-10-0-130-85 kubenswrapper[2577]: W0720 17:14:03.986269    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e9de5be_5e31_4a00_be51_4ca60d36a8af.slice/crio-61f103e0c7e734185c1a8f4586887bb0c2f56411aeb0922fded237a62a5c62de WatchSource:0}: Error finding container 61f103e0c7e734185c1a8f4586887bb0c2f56411aeb0922fded237a62a5c62de: Status 404 returned error can't find the container with id 61f103e0c7e734185c1a8f4586887bb0c2f56411aeb0922fded237a62a5c62de
Jul 20 17:14:04.734249 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:04.734209    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw" event={"ID":"2e9de5be-5e31-4a00-be51-4ca60d36a8af","Type":"ContainerStarted","Data":"cb93245712f56c058ffa009c4aeb83a3f0d29d1923c10e837bb15a29d88a1c8b"}
Jul 20 17:14:04.734759 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:04.734258    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw" event={"ID":"2e9de5be-5e31-4a00-be51-4ca60d36a8af","Type":"ContainerStarted","Data":"61f103e0c7e734185c1a8f4586887bb0c2f56411aeb0922fded237a62a5c62de"}
Jul 20 17:14:04.756314 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:04.756237    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw" podStartSLOduration=1.756215207 podStartE2EDuration="1.756215207s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:14:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:14:04.75547932 +0000 UTC m=+2668.004973342" watchObservedRunningTime="2026-07-20 17:14:04.756215207 +0000 UTC m=+2668.005709227"
Jul 20 17:14:13.852372 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:13.852336    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw"
Jul 20 17:14:13.852845 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:13.852385    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw"
Jul 20 17:14:13.864724 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:13.864692    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw"
Jul 20 17:14:14.788876 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:14.788846    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw"
Jul 20 17:14:32.012323 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:32.012229    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh"]
Jul 20 17:14:32.015673 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:32.015656    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh"
Jul 20 17:14:32.018496 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:32.018477    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e142d5\"/\"sub-test-model-e142d5-kserve-self-signed-certs\""
Jul 20 17:14:32.019701 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:32.019686    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e142d5\"/\"default-dockercfg-4wd97\""
Jul 20 17:14:32.019857 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:32.019840    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e142d5\"/\"kube-root-ca.crt\""
Jul 20 17:14:32.020486 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:32.020468    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e142d5\"/\"openshift-service-ca.crt\""
Jul 20 17:14:32.029670 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:32.029651    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh"]
Jul 20 17:14:32.058397 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:32.058371    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:14:32.104700 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:32.104665    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39b7fdab-b4bc-4a31-b557-c9f911502865-tmp-dir\") pod \"sub-test-model-e142d5-kserve-ff9d49c9c-shwxh\" (UID: \"39b7fdab-b4bc-4a31-b557-c9f911502865\") " pod="ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh"
Jul 20 17:14:32.104813 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:32.104777    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hphbl\" (UniqueName: \"kubernetes.io/projected/39b7fdab-b4bc-4a31-b557-c9f911502865-kube-api-access-hphbl\") pod \"sub-test-model-e142d5-kserve-ff9d49c9c-shwxh\" (UID: \"39b7fdab-b4bc-4a31-b557-c9f911502865\") " pod="ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh"
Jul 20 17:14:32.104867 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:32.104813    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39b7fdab-b4bc-4a31-b557-c9f911502865-home\") pod \"sub-test-model-e142d5-kserve-ff9d49c9c-shwxh\" (UID: \"39b7fdab-b4bc-4a31-b557-c9f911502865\") " pod="ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh"
Jul 20 17:14:32.104867 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:32.104836    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39b7fdab-b4bc-4a31-b557-c9f911502865-dshm\") pod \"sub-test-model-e142d5-kserve-ff9d49c9c-shwxh\" (UID: \"39b7fdab-b4bc-4a31-b557-c9f911502865\") " pod="ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh"
Jul 20 17:14:32.104969 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:32.104871    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39b7fdab-b4bc-4a31-b557-c9f911502865-model-cache\") pod \"sub-test-model-e142d5-kserve-ff9d49c9c-shwxh\" (UID: \"39b7fdab-b4bc-4a31-b557-c9f911502865\") " pod="ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh"
Jul 20 17:14:32.104969 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:32.104963    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39b7fdab-b4bc-4a31-b557-c9f911502865-tls-certs\") pod \"sub-test-model-e142d5-kserve-ff9d49c9c-shwxh\" (UID: \"39b7fdab-b4bc-4a31-b557-c9f911502865\") " pod="ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh"
Jul 20 17:14:32.206028 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:32.205996    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hphbl\" (UniqueName: \"kubernetes.io/projected/39b7fdab-b4bc-4a31-b557-c9f911502865-kube-api-access-hphbl\") pod \"sub-test-model-e142d5-kserve-ff9d49c9c-shwxh\" (UID: \"39b7fdab-b4bc-4a31-b557-c9f911502865\") " pod="ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh"
Jul 20 17:14:32.206028 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:32.206029    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39b7fdab-b4bc-4a31-b557-c9f911502865-home\") pod \"sub-test-model-e142d5-kserve-ff9d49c9c-shwxh\" (UID: \"39b7fdab-b4bc-4a31-b557-c9f911502865\") " pod="ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh"
Jul 20 17:14:32.206229 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:32.206050    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39b7fdab-b4bc-4a31-b557-c9f911502865-dshm\") pod \"sub-test-model-e142d5-kserve-ff9d49c9c-shwxh\" (UID: \"39b7fdab-b4bc-4a31-b557-c9f911502865\") " pod="ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh"
Jul 20 17:14:32.206229 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:32.206074    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39b7fdab-b4bc-4a31-b557-c9f911502865-model-cache\") pod \"sub-test-model-e142d5-kserve-ff9d49c9c-shwxh\" (UID: \"39b7fdab-b4bc-4a31-b557-c9f911502865\") " pod="ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh"
Jul 20 17:14:32.206229 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:32.206112    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39b7fdab-b4bc-4a31-b557-c9f911502865-tls-certs\") pod \"sub-test-model-e142d5-kserve-ff9d49c9c-shwxh\" (UID: \"39b7fdab-b4bc-4a31-b557-c9f911502865\") " pod="ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh"
Jul 20 17:14:32.206229 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:32.206151    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39b7fdab-b4bc-4a31-b557-c9f911502865-tmp-dir\") pod \"sub-test-model-e142d5-kserve-ff9d49c9c-shwxh\" (UID: \"39b7fdab-b4bc-4a31-b557-c9f911502865\") " pod="ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh"
Jul 20 17:14:32.206486 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:32.206464    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39b7fdab-b4bc-4a31-b557-c9f911502865-home\") pod \"sub-test-model-e142d5-kserve-ff9d49c9c-shwxh\" (UID: \"39b7fdab-b4bc-4a31-b557-c9f911502865\") " pod="ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh"
Jul 20 17:14:32.206553 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:32.206507    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39b7fdab-b4bc-4a31-b557-c9f911502865-model-cache\") pod \"sub-test-model-e142d5-kserve-ff9d49c9c-shwxh\" (UID: \"39b7fdab-b4bc-4a31-b557-c9f911502865\") " pod="ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh"
Jul 20 17:14:32.206605 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:32.206594    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39b7fdab-b4bc-4a31-b557-c9f911502865-tmp-dir\") pod \"sub-test-model-e142d5-kserve-ff9d49c9c-shwxh\" (UID: \"39b7fdab-b4bc-4a31-b557-c9f911502865\") " pod="ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh"
Jul 20 17:14:32.208440 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:32.208422    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39b7fdab-b4bc-4a31-b557-c9f911502865-dshm\") pod \"sub-test-model-e142d5-kserve-ff9d49c9c-shwxh\" (UID: \"39b7fdab-b4bc-4a31-b557-c9f911502865\") " pod="ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh"
Jul 20 17:14:32.208676 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:32.208657    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39b7fdab-b4bc-4a31-b557-c9f911502865-tls-certs\") pod \"sub-test-model-e142d5-kserve-ff9d49c9c-shwxh\" (UID: \"39b7fdab-b4bc-4a31-b557-c9f911502865\") " pod="ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh"
Jul 20 17:14:32.214733 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:32.214711    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hphbl\" (UniqueName: \"kubernetes.io/projected/39b7fdab-b4bc-4a31-b557-c9f911502865-kube-api-access-hphbl\") pod \"sub-test-model-e142d5-kserve-ff9d49c9c-shwxh\" (UID: \"39b7fdab-b4bc-4a31-b557-c9f911502865\") " pod="ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh"
Jul 20 17:14:32.325893 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:32.325868    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh"
Jul 20 17:14:32.461552 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:32.461525    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh"]
Jul 20 17:14:32.463642 ip-10-0-130-85 kubenswrapper[2577]: W0720 17:14:32.463612    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod39b7fdab_b4bc_4a31_b557_c9f911502865.slice/crio-71d204dc44327db1a4f904fd1bd9262667bef8301fe1f105535dfd6ec501dfbc WatchSource:0}: Error finding container 71d204dc44327db1a4f904fd1bd9262667bef8301fe1f105535dfd6ec501dfbc: Status 404 returned error can't find the container with id 71d204dc44327db1a4f904fd1bd9262667bef8301fe1f105535dfd6ec501dfbc
Jul 20 17:14:32.857377 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:32.857335    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh" event={"ID":"39b7fdab-b4bc-4a31-b557-c9f911502865","Type":"ContainerStarted","Data":"1cb855280c9da04617c9bb589a2db9208b5e60508134a815811abe047680b724"}
Jul 20 17:14:32.857558 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:32.857385    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh" event={"ID":"39b7fdab-b4bc-4a31-b557-c9f911502865","Type":"ContainerStarted","Data":"71d204dc44327db1a4f904fd1bd9262667bef8301fe1f105535dfd6ec501dfbc"}
Jul 20 17:14:32.879870 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:32.879834    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh" podStartSLOduration=1.879821982 podStartE2EDuration="1.879821982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:14:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:14:32.879713628 +0000 UTC m=+2696.129207649" watchObservedRunningTime="2026-07-20 17:14:32.879821982 +0000 UTC m=+2696.129316001"
Jul 20 17:14:37.306007 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:37.305976    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gd4p4_fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c/ovn-acl-logging/0.log"
Jul 20 17:14:37.310441 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:37.310418    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gd4p4_fc3cc1d5-9e3b-43d8-9f9d-7e80dfc0f12c/ovn-acl-logging/0.log"
Jul 20 17:14:42.326469 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:42.326434    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh"
Jul 20 17:14:42.326865 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:42.326685    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh"
Jul 20 17:14:42.338784 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:42.338760    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh"
Jul 20 17:14:42.911862 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:42.911840    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh"
Jul 20 17:14:50.624112 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:50.624067    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:14:53.760312 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:53.760256    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:14:53.862915 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:14:53.862881    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:15:02.741312 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:02.741271    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:15:10.912217 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:10.912181    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw"]
Jul 20 17:15:10.972915 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:10.972887    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:15:11.705531 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:11.705498    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh"]
Jul 20 17:15:11.733071 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:11.733031    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:15:23.710678 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:23.710629    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:15:24.035395 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:24.035285    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:15:25.932255 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:25.932183    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw" podUID="2e9de5be-5e31-4a00-be51-4ca60d36a8af" containerName="main" containerID="cri-o://cb93245712f56c058ffa009c4aeb83a3f0d29d1923c10e837bb15a29d88a1c8b" gracePeriod=45
Jul 20 17:15:26.097166 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:26.097126    2577 generic.go:356] "Generic (PLEG): container finished" podID="2e9de5be-5e31-4a00-be51-4ca60d36a8af" containerID="cb93245712f56c058ffa009c4aeb83a3f0d29d1923c10e837bb15a29d88a1c8b" exitCode=0
Jul 20 17:15:26.097329 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:26.097189    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw" event={"ID":"2e9de5be-5e31-4a00-be51-4ca60d36a8af","Type":"ContainerDied","Data":"cb93245712f56c058ffa009c4aeb83a3f0d29d1923c10e837bb15a29d88a1c8b"}
Jul 20 17:15:26.181335 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:26.181314    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw"
Jul 20 17:15:26.367318 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:26.367266    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e9de5be-5e31-4a00-be51-4ca60d36a8af-home\") pod \"2e9de5be-5e31-4a00-be51-4ca60d36a8af\" (UID: \"2e9de5be-5e31-4a00-be51-4ca60d36a8af\") "
Jul 20 17:15:26.367499 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:26.367329    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e9de5be-5e31-4a00-be51-4ca60d36a8af-model-cache\") pod \"2e9de5be-5e31-4a00-be51-4ca60d36a8af\" (UID: \"2e9de5be-5e31-4a00-be51-4ca60d36a8af\") "
Jul 20 17:15:26.367499 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:26.367367    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e9de5be-5e31-4a00-be51-4ca60d36a8af-tmp-dir\") pod \"2e9de5be-5e31-4a00-be51-4ca60d36a8af\" (UID: \"2e9de5be-5e31-4a00-be51-4ca60d36a8af\") "
Jul 20 17:15:26.367499 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:26.367409    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e9de5be-5e31-4a00-be51-4ca60d36a8af-dshm\") pod \"2e9de5be-5e31-4a00-be51-4ca60d36a8af\" (UID: \"2e9de5be-5e31-4a00-be51-4ca60d36a8af\") "
Jul 20 17:15:26.367499 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:26.367454    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wt7zb\" (UniqueName: \"kubernetes.io/projected/2e9de5be-5e31-4a00-be51-4ca60d36a8af-kube-api-access-wt7zb\") pod \"2e9de5be-5e31-4a00-be51-4ca60d36a8af\" (UID: \"2e9de5be-5e31-4a00-be51-4ca60d36a8af\") "
Jul 20 17:15:26.367499 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:26.367483    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e9de5be-5e31-4a00-be51-4ca60d36a8af-tls-certs\") pod \"2e9de5be-5e31-4a00-be51-4ca60d36a8af\" (UID: \"2e9de5be-5e31-4a00-be51-4ca60d36a8af\") "
Jul 20 17:15:26.367779 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:26.367543    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e9de5be-5e31-4a00-be51-4ca60d36a8af-home" (OuterVolumeSpecName: "home") pod "2e9de5be-5e31-4a00-be51-4ca60d36a8af" (UID: "2e9de5be-5e31-4a00-be51-4ca60d36a8af"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:15:26.367779 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:26.367557    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e9de5be-5e31-4a00-be51-4ca60d36a8af-model-cache" (OuterVolumeSpecName: "model-cache") pod "2e9de5be-5e31-4a00-be51-4ca60d36a8af" (UID: "2e9de5be-5e31-4a00-be51-4ca60d36a8af"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:15:26.367779 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:26.367634    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e9de5be-5e31-4a00-be51-4ca60d36a8af-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2e9de5be-5e31-4a00-be51-4ca60d36a8af" (UID: "2e9de5be-5e31-4a00-be51-4ca60d36a8af"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:15:26.367922 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:26.367829    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e9de5be-5e31-4a00-be51-4ca60d36a8af-home\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:15:26.367922 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:26.367849    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e9de5be-5e31-4a00-be51-4ca60d36a8af-model-cache\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:15:26.367922 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:26.367864    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e9de5be-5e31-4a00-be51-4ca60d36a8af-tmp-dir\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:15:26.369652 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:26.369621    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e9de5be-5e31-4a00-be51-4ca60d36a8af-kube-api-access-wt7zb" (OuterVolumeSpecName: "kube-api-access-wt7zb") pod "2e9de5be-5e31-4a00-be51-4ca60d36a8af" (UID: "2e9de5be-5e31-4a00-be51-4ca60d36a8af"). InnerVolumeSpecName "kube-api-access-wt7zb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:15:26.369958 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:26.369940    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e9de5be-5e31-4a00-be51-4ca60d36a8af-dshm" (OuterVolumeSpecName: "dshm") pod "2e9de5be-5e31-4a00-be51-4ca60d36a8af" (UID: "2e9de5be-5e31-4a00-be51-4ca60d36a8af"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:15:26.370047 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:26.369966    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e9de5be-5e31-4a00-be51-4ca60d36a8af-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2e9de5be-5e31-4a00-be51-4ca60d36a8af" (UID: "2e9de5be-5e31-4a00-be51-4ca60d36a8af"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:15:26.468969 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:26.468944    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e9de5be-5e31-4a00-be51-4ca60d36a8af-dshm\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:15:26.468969 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:26.468970    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wt7zb\" (UniqueName: \"kubernetes.io/projected/2e9de5be-5e31-4a00-be51-4ca60d36a8af-kube-api-access-wt7zb\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:15:26.469134 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:26.468981    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e9de5be-5e31-4a00-be51-4ca60d36a8af-tls-certs\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:15:26.725085 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:26.724971    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh" podUID="39b7fdab-b4bc-4a31-b557-c9f911502865" containerName="main" containerID="cri-o://1cb855280c9da04617c9bb589a2db9208b5e60508134a815811abe047680b724" gracePeriod=45
Jul 20 17:15:26.968868 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:26.968847    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh"
Jul 20 17:15:27.072649 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:27.072625    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hphbl\" (UniqueName: \"kubernetes.io/projected/39b7fdab-b4bc-4a31-b557-c9f911502865-kube-api-access-hphbl\") pod \"39b7fdab-b4bc-4a31-b557-c9f911502865\" (UID: \"39b7fdab-b4bc-4a31-b557-c9f911502865\") "
Jul 20 17:15:27.072769 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:27.072666    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39b7fdab-b4bc-4a31-b557-c9f911502865-home\") pod \"39b7fdab-b4bc-4a31-b557-c9f911502865\" (UID: \"39b7fdab-b4bc-4a31-b557-c9f911502865\") "
Jul 20 17:15:27.072769 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:27.072715    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39b7fdab-b4bc-4a31-b557-c9f911502865-tmp-dir\") pod \"39b7fdab-b4bc-4a31-b557-c9f911502865\" (UID: \"39b7fdab-b4bc-4a31-b557-c9f911502865\") "
Jul 20 17:15:27.072769 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:27.072731    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39b7fdab-b4bc-4a31-b557-c9f911502865-dshm\") pod \"39b7fdab-b4bc-4a31-b557-c9f911502865\" (UID: \"39b7fdab-b4bc-4a31-b557-c9f911502865\") "
Jul 20 17:15:27.072769 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:27.072751    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39b7fdab-b4bc-4a31-b557-c9f911502865-tls-certs\") pod \"39b7fdab-b4bc-4a31-b557-c9f911502865\" (UID: \"39b7fdab-b4bc-4a31-b557-c9f911502865\") "
Jul 20 17:15:27.072980 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:27.072794    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39b7fdab-b4bc-4a31-b557-c9f911502865-model-cache\") pod \"39b7fdab-b4bc-4a31-b557-c9f911502865\" (UID: \"39b7fdab-b4bc-4a31-b557-c9f911502865\") "
Jul 20 17:15:27.073038 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:27.072968    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39b7fdab-b4bc-4a31-b557-c9f911502865-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "39b7fdab-b4bc-4a31-b557-c9f911502865" (UID: "39b7fdab-b4bc-4a31-b557-c9f911502865"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:15:27.073038 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:27.072999    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39b7fdab-b4bc-4a31-b557-c9f911502865-home" (OuterVolumeSpecName: "home") pod "39b7fdab-b4bc-4a31-b557-c9f911502865" (UID: "39b7fdab-b4bc-4a31-b557-c9f911502865"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:15:27.073164 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:27.073145    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39b7fdab-b4bc-4a31-b557-c9f911502865-home\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:15:27.073213 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:27.073163    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39b7fdab-b4bc-4a31-b557-c9f911502865-model-cache" (OuterVolumeSpecName: "model-cache") pod "39b7fdab-b4bc-4a31-b557-c9f911502865" (UID: "39b7fdab-b4bc-4a31-b557-c9f911502865"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:15:27.073213 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:27.073175    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39b7fdab-b4bc-4a31-b557-c9f911502865-tmp-dir\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:15:27.074787 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:27.074763    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39b7fdab-b4bc-4a31-b557-c9f911502865-dshm" (OuterVolumeSpecName: "dshm") pod "39b7fdab-b4bc-4a31-b557-c9f911502865" (UID: "39b7fdab-b4bc-4a31-b557-c9f911502865"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:15:27.075178 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:27.075159    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39b7fdab-b4bc-4a31-b557-c9f911502865-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "39b7fdab-b4bc-4a31-b557-c9f911502865" (UID: "39b7fdab-b4bc-4a31-b557-c9f911502865"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:15:27.075252 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:27.075163    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39b7fdab-b4bc-4a31-b557-c9f911502865-kube-api-access-hphbl" (OuterVolumeSpecName: "kube-api-access-hphbl") pod "39b7fdab-b4bc-4a31-b557-c9f911502865" (UID: "39b7fdab-b4bc-4a31-b557-c9f911502865"). InnerVolumeSpecName "kube-api-access-hphbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:15:27.103116 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:27.103095    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw"
Jul 20 17:15:27.103216 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:27.103126    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw" event={"ID":"2e9de5be-5e31-4a00-be51-4ca60d36a8af","Type":"ContainerDied","Data":"61f103e0c7e734185c1a8f4586887bb0c2f56411aeb0922fded237a62a5c62de"}
Jul 20 17:15:27.103216 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:27.103177    2577 scope.go:117] "RemoveContainer" containerID="cb93245712f56c058ffa009c4aeb83a3f0d29d1923c10e837bb15a29d88a1c8b"
Jul 20 17:15:27.104474 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:27.104449    2577 generic.go:356] "Generic (PLEG): container finished" podID="39b7fdab-b4bc-4a31-b557-c9f911502865" containerID="1cb855280c9da04617c9bb589a2db9208b5e60508134a815811abe047680b724" exitCode=0
Jul 20 17:15:27.104580 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:27.104488    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh"
Jul 20 17:15:27.104580 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:27.104536    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh" event={"ID":"39b7fdab-b4bc-4a31-b557-c9f911502865","Type":"ContainerDied","Data":"1cb855280c9da04617c9bb589a2db9208b5e60508134a815811abe047680b724"}
Jul 20 17:15:27.104580 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:27.104577    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh" event={"ID":"39b7fdab-b4bc-4a31-b557-c9f911502865","Type":"ContainerDied","Data":"71d204dc44327db1a4f904fd1bd9262667bef8301fe1f105535dfd6ec501dfbc"}
Jul 20 17:15:27.113156 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:27.113141    2577 scope.go:117] "RemoveContainer" containerID="1cb855280c9da04617c9bb589a2db9208b5e60508134a815811abe047680b724"
Jul 20 17:15:27.120645 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:27.120629    2577 scope.go:117] "RemoveContainer" containerID="1cb855280c9da04617c9bb589a2db9208b5e60508134a815811abe047680b724"
Jul 20 17:15:27.120890 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:15:27.120875    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cb855280c9da04617c9bb589a2db9208b5e60508134a815811abe047680b724\": container with ID starting with 1cb855280c9da04617c9bb589a2db9208b5e60508134a815811abe047680b724 not found: ID does not exist" containerID="1cb855280c9da04617c9bb589a2db9208b5e60508134a815811abe047680b724"
Jul 20 17:15:27.120936 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:27.120895    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cb855280c9da04617c9bb589a2db9208b5e60508134a815811abe047680b724"} err="failed to get container status \"1cb855280c9da04617c9bb589a2db9208b5e60508134a815811abe047680b724\": rpc error: code = NotFound desc = could not find container \"1cb855280c9da04617c9bb589a2db9208b5e60508134a815811abe047680b724\": container with ID starting with 1cb855280c9da04617c9bb589a2db9208b5e60508134a815811abe047680b724 not found: ID does not exist"
Jul 20 17:15:27.127502 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:27.127478    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw"]
Jul 20 17:15:27.131088 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:27.131069    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-a16895/sub-test-model-a16895-kserve-696b7f75f5-kf6lw"]
Jul 20 17:15:27.141610 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:27.141588    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh"]
Jul 20 17:15:27.145708 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:27.145687    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-e142d5/sub-test-model-e142d5-kserve-ff9d49c9c-shwxh"]
Jul 20 17:15:27.174002 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:27.173984    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39b7fdab-b4bc-4a31-b557-c9f911502865-model-cache\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:15:27.174002 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:27.174002    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hphbl\" (UniqueName: \"kubernetes.io/projected/39b7fdab-b4bc-4a31-b557-c9f911502865-kube-api-access-hphbl\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:15:27.174106 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:27.174012    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39b7fdab-b4bc-4a31-b557-c9f911502865-dshm\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:15:27.174106 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:27.174021    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39b7fdab-b4bc-4a31-b557-c9f911502865-tls-certs\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:15:27.211427 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:27.211405    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e9de5be-5e31-4a00-be51-4ca60d36a8af" path="/var/lib/kubelet/pods/2e9de5be-5e31-4a00-be51-4ca60d36a8af/volumes"
Jul 20 17:15:27.211793 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:27.211780    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39b7fdab-b4bc-4a31-b557-c9f911502865" path="/var/lib/kubelet/pods/39b7fdab-b4bc-4a31-b557-c9f911502865/volumes"
Jul 20 17:15:29.914455 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:29.914424    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz"]
Jul 20 17:15:29.914948 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:29.914927    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e9de5be-5e31-4a00-be51-4ca60d36a8af" containerName="main"
Jul 20 17:15:29.915034 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:29.914951    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e9de5be-5e31-4a00-be51-4ca60d36a8af" containerName="main"
Jul 20 17:15:29.915034 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:29.914978    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39b7fdab-b4bc-4a31-b557-c9f911502865" containerName="main"
Jul 20 17:15:29.915034 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:29.914989    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="39b7fdab-b4bc-4a31-b557-c9f911502865" containerName="main"
Jul 20 17:15:29.915181 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:29.915068    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="39b7fdab-b4bc-4a31-b557-c9f911502865" containerName="main"
Jul 20 17:15:29.915181 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:29.915084    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e9de5be-5e31-4a00-be51-4ca60d36a8af" containerName="main"
Jul 20 17:15:29.919560 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:29.919538    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz"
Jul 20 17:15:29.925010 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:29.924963    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a16895\"/\"default-dockercfg-z4cmd\""
Jul 20 17:15:29.925010 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:29.924992    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a16895\"/\"rate-test-model-a16895-kserve-self-signed-certs\""
Jul 20 17:15:29.925184 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:29.925112    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a16895\"/\"kube-root-ca.crt\""
Jul 20 17:15:29.925278 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:29.925260    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a16895\"/\"openshift-service-ca.crt\""
Jul 20 17:15:29.940322 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:29.940282    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz"]
Jul 20 17:15:29.967604 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:29.967571    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:15:30.095599 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:30.095565    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62071b75-40ea-4e31-a3b8-12a39bbec3fc-tmp-dir\") pod \"rate-test-model-a16895-kserve-6d4976f67f-c76vz\" (UID: \"62071b75-40ea-4e31-a3b8-12a39bbec3fc\") " pod="ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz"
Jul 20 17:15:30.095716 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:30.095608    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62071b75-40ea-4e31-a3b8-12a39bbec3fc-home\") pod \"rate-test-model-a16895-kserve-6d4976f67f-c76vz\" (UID: \"62071b75-40ea-4e31-a3b8-12a39bbec3fc\") " pod="ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz"
Jul 20 17:15:30.095716 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:30.095681    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62071b75-40ea-4e31-a3b8-12a39bbec3fc-model-cache\") pod \"rate-test-model-a16895-kserve-6d4976f67f-c76vz\" (UID: \"62071b75-40ea-4e31-a3b8-12a39bbec3fc\") " pod="ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz"
Jul 20 17:15:30.095813 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:30.095749    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62071b75-40ea-4e31-a3b8-12a39bbec3fc-dshm\") pod \"rate-test-model-a16895-kserve-6d4976f67f-c76vz\" (UID: \"62071b75-40ea-4e31-a3b8-12a39bbec3fc\") " pod="ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz"
Jul 20 17:15:30.095813 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:30.095805    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62071b75-40ea-4e31-a3b8-12a39bbec3fc-tls-certs\") pod \"rate-test-model-a16895-kserve-6d4976f67f-c76vz\" (UID: \"62071b75-40ea-4e31-a3b8-12a39bbec3fc\") " pod="ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz"
Jul 20 17:15:30.095886 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:30.095832    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzq77\" (UniqueName: \"kubernetes.io/projected/62071b75-40ea-4e31-a3b8-12a39bbec3fc-kube-api-access-wzq77\") pod \"rate-test-model-a16895-kserve-6d4976f67f-c76vz\" (UID: \"62071b75-40ea-4e31-a3b8-12a39bbec3fc\") " pod="ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz"
Jul 20 17:15:30.196695 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:30.196613    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62071b75-40ea-4e31-a3b8-12a39bbec3fc-dshm\") pod \"rate-test-model-a16895-kserve-6d4976f67f-c76vz\" (UID: \"62071b75-40ea-4e31-a3b8-12a39bbec3fc\") " pod="ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz"
Jul 20 17:15:30.196695 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:30.196670    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62071b75-40ea-4e31-a3b8-12a39bbec3fc-tls-certs\") pod \"rate-test-model-a16895-kserve-6d4976f67f-c76vz\" (UID: \"62071b75-40ea-4e31-a3b8-12a39bbec3fc\") " pod="ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz"
Jul 20 17:15:30.196695 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:30.196694    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzq77\" (UniqueName: \"kubernetes.io/projected/62071b75-40ea-4e31-a3b8-12a39bbec3fc-kube-api-access-wzq77\") pod \"rate-test-model-a16895-kserve-6d4976f67f-c76vz\" (UID: \"62071b75-40ea-4e31-a3b8-12a39bbec3fc\") " pod="ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz"
Jul 20 17:15:30.196936 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:30.196720    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62071b75-40ea-4e31-a3b8-12a39bbec3fc-tmp-dir\") pod \"rate-test-model-a16895-kserve-6d4976f67f-c76vz\" (UID: \"62071b75-40ea-4e31-a3b8-12a39bbec3fc\") " pod="ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz"
Jul 20 17:15:30.196936 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:30.196742    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62071b75-40ea-4e31-a3b8-12a39bbec3fc-home\") pod \"rate-test-model-a16895-kserve-6d4976f67f-c76vz\" (UID: \"62071b75-40ea-4e31-a3b8-12a39bbec3fc\") " pod="ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz"
Jul 20 17:15:30.196936 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:30.196899    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62071b75-40ea-4e31-a3b8-12a39bbec3fc-model-cache\") pod \"rate-test-model-a16895-kserve-6d4976f67f-c76vz\" (UID: \"62071b75-40ea-4e31-a3b8-12a39bbec3fc\") " pod="ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz"
Jul 20 17:15:30.197099 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:30.197085    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62071b75-40ea-4e31-a3b8-12a39bbec3fc-home\") pod \"rate-test-model-a16895-kserve-6d4976f67f-c76vz\" (UID: \"62071b75-40ea-4e31-a3b8-12a39bbec3fc\") " pod="ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz"
Jul 20 17:15:30.197200 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:30.197180    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62071b75-40ea-4e31-a3b8-12a39bbec3fc-tmp-dir\") pod \"rate-test-model-a16895-kserve-6d4976f67f-c76vz\" (UID: \"62071b75-40ea-4e31-a3b8-12a39bbec3fc\") " pod="ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz"
Jul 20 17:15:30.197259 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:30.197233    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62071b75-40ea-4e31-a3b8-12a39bbec3fc-model-cache\") pod \"rate-test-model-a16895-kserve-6d4976f67f-c76vz\" (UID: \"62071b75-40ea-4e31-a3b8-12a39bbec3fc\") " pod="ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz"
Jul 20 17:15:30.198763 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:30.198742    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62071b75-40ea-4e31-a3b8-12a39bbec3fc-dshm\") pod \"rate-test-model-a16895-kserve-6d4976f67f-c76vz\" (UID: \"62071b75-40ea-4e31-a3b8-12a39bbec3fc\") " pod="ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz"
Jul 20 17:15:30.199101 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:30.199081    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62071b75-40ea-4e31-a3b8-12a39bbec3fc-tls-certs\") pod \"rate-test-model-a16895-kserve-6d4976f67f-c76vz\" (UID: \"62071b75-40ea-4e31-a3b8-12a39bbec3fc\") " pod="ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz"
Jul 20 17:15:30.210455 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:30.210433    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzq77\" (UniqueName: \"kubernetes.io/projected/62071b75-40ea-4e31-a3b8-12a39bbec3fc-kube-api-access-wzq77\") pod \"rate-test-model-a16895-kserve-6d4976f67f-c76vz\" (UID: \"62071b75-40ea-4e31-a3b8-12a39bbec3fc\") " pod="ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz"
Jul 20 17:15:30.228293 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:30.228272    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz"
Jul 20 17:15:30.360266 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:30.360237    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz"]
Jul 20 17:15:30.361272 ip-10-0-130-85 kubenswrapper[2577]: W0720 17:15:30.361244    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod62071b75_40ea_4e31_a3b8_12a39bbec3fc.slice/crio-400988f74c0467b2d148100b1bc7ceb4b2c8abe5e380acf9decdddd4f13a9612 WatchSource:0}: Error finding container 400988f74c0467b2d148100b1bc7ceb4b2c8abe5e380acf9decdddd4f13a9612: Status 404 returned error can't find the container with id 400988f74c0467b2d148100b1bc7ceb4b2c8abe5e380acf9decdddd4f13a9612
Jul 20 17:15:31.126094 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:31.126058    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz" event={"ID":"62071b75-40ea-4e31-a3b8-12a39bbec3fc","Type":"ContainerStarted","Data":"a73e83d0dd1ce37701f2f18075edb67c45821e84c46b4a1a14c52101da64dad2"}
Jul 20 17:15:31.126094 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:31.126098    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz" event={"ID":"62071b75-40ea-4e31-a3b8-12a39bbec3fc","Type":"ContainerStarted","Data":"400988f74c0467b2d148100b1bc7ceb4b2c8abe5e380acf9decdddd4f13a9612"}
Jul 20 17:15:31.151867 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:31.151808    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz" podStartSLOduration=2.151792426 podStartE2EDuration="2.151792426s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 17:15:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:15:31.149170655 +0000 UTC m=+2754.398664673" watchObservedRunningTime="2026-07-20 17:15:31.151792426 +0000 UTC m=+2754.401286444"
Jul 20 17:15:37.450127 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:37.450095    2577 scope.go:117] "RemoveContainer" containerID="ce47e93545c38810efd8204b2ed0cff0c2bd78d463fb81247040af8c7384d093"
Jul 20 17:15:40.229257 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:40.229226    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz"
Jul 20 17:15:40.229708 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:40.229269    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz"
Jul 20 17:15:40.241755 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:40.241731    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz"
Jul 20 17:15:41.179288 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:41.179258    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz"
Jul 20 17:15:46.473505 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:46.473468    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:15:46.550261 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:46.550235    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:15:52.251314 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:52.251253    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:15:52.351858 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:15:52.351832    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:16:00.351069 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:16:00.350990    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:16:00.453111 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:16:00.453075    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:16:20.532735 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:16:20.532700    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:05.992569 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:05.992533    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz"]
Jul 20 17:17:06.022377 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:06.022343    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:06.835197 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:06.835162    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:21.012011 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:21.011965    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz" podUID="62071b75-40ea-4e31-a3b8-12a39bbec3fc" containerName="main" containerID="cri-o://a73e83d0dd1ce37701f2f18075edb67c45821e84c46b4a1a14c52101da64dad2" gracePeriod=45
Jul 20 17:17:21.169814 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:21.169778    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz" podUID="62071b75-40ea-4e31-a3b8-12a39bbec3fc" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": dial tcp 10.133.0.59:8000: connect: connection refused"
Jul 20 17:17:21.262658 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:21.262601    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz"
Jul 20 17:17:21.437997 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:21.437965    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62071b75-40ea-4e31-a3b8-12a39bbec3fc-dshm\") pod \"62071b75-40ea-4e31-a3b8-12a39bbec3fc\" (UID: \"62071b75-40ea-4e31-a3b8-12a39bbec3fc\") "
Jul 20 17:17:21.438188 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:21.438014    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62071b75-40ea-4e31-a3b8-12a39bbec3fc-model-cache\") pod \"62071b75-40ea-4e31-a3b8-12a39bbec3fc\" (UID: \"62071b75-40ea-4e31-a3b8-12a39bbec3fc\") "
Jul 20 17:17:21.438188 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:21.438071    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62071b75-40ea-4e31-a3b8-12a39bbec3fc-tmp-dir\") pod \"62071b75-40ea-4e31-a3b8-12a39bbec3fc\" (UID: \"62071b75-40ea-4e31-a3b8-12a39bbec3fc\") "
Jul 20 17:17:21.438188 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:21.438098    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzq77\" (UniqueName: \"kubernetes.io/projected/62071b75-40ea-4e31-a3b8-12a39bbec3fc-kube-api-access-wzq77\") pod \"62071b75-40ea-4e31-a3b8-12a39bbec3fc\" (UID: \"62071b75-40ea-4e31-a3b8-12a39bbec3fc\") "
Jul 20 17:17:21.438188 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:21.438128    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62071b75-40ea-4e31-a3b8-12a39bbec3fc-home\") pod \"62071b75-40ea-4e31-a3b8-12a39bbec3fc\" (UID: \"62071b75-40ea-4e31-a3b8-12a39bbec3fc\") "
Jul 20 17:17:21.438188 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:21.438143    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62071b75-40ea-4e31-a3b8-12a39bbec3fc-tls-certs\") pod \"62071b75-40ea-4e31-a3b8-12a39bbec3fc\" (UID: \"62071b75-40ea-4e31-a3b8-12a39bbec3fc\") "
Jul 20 17:17:21.438479 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:21.438399    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62071b75-40ea-4e31-a3b8-12a39bbec3fc-model-cache" (OuterVolumeSpecName: "model-cache") pod "62071b75-40ea-4e31-a3b8-12a39bbec3fc" (UID: "62071b75-40ea-4e31-a3b8-12a39bbec3fc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:17:21.438479 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:21.438410    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62071b75-40ea-4e31-a3b8-12a39bbec3fc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "62071b75-40ea-4e31-a3b8-12a39bbec3fc" (UID: "62071b75-40ea-4e31-a3b8-12a39bbec3fc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:17:21.438479 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:21.438421    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62071b75-40ea-4e31-a3b8-12a39bbec3fc-home" (OuterVolumeSpecName: "home") pod "62071b75-40ea-4e31-a3b8-12a39bbec3fc" (UID: "62071b75-40ea-4e31-a3b8-12a39bbec3fc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:17:21.440207 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:21.440172    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62071b75-40ea-4e31-a3b8-12a39bbec3fc-dshm" (OuterVolumeSpecName: "dshm") pod "62071b75-40ea-4e31-a3b8-12a39bbec3fc" (UID: "62071b75-40ea-4e31-a3b8-12a39bbec3fc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:17:21.440358 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:21.440250    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62071b75-40ea-4e31-a3b8-12a39bbec3fc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "62071b75-40ea-4e31-a3b8-12a39bbec3fc" (UID: "62071b75-40ea-4e31-a3b8-12a39bbec3fc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:17:21.440358 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:21.440253    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62071b75-40ea-4e31-a3b8-12a39bbec3fc-kube-api-access-wzq77" (OuterVolumeSpecName: "kube-api-access-wzq77") pod "62071b75-40ea-4e31-a3b8-12a39bbec3fc" (UID: "62071b75-40ea-4e31-a3b8-12a39bbec3fc"). InnerVolumeSpecName "kube-api-access-wzq77". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:17:21.539176 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:21.539103    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzq77\" (UniqueName: \"kubernetes.io/projected/62071b75-40ea-4e31-a3b8-12a39bbec3fc-kube-api-access-wzq77\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:17:21.539176 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:21.539129    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62071b75-40ea-4e31-a3b8-12a39bbec3fc-home\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:17:21.539176 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:21.539142    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62071b75-40ea-4e31-a3b8-12a39bbec3fc-tls-certs\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:17:21.539176 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:21.539151    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62071b75-40ea-4e31-a3b8-12a39bbec3fc-dshm\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:17:21.539176 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:21.539159    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62071b75-40ea-4e31-a3b8-12a39bbec3fc-model-cache\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:17:21.539176 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:21.539167    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62071b75-40ea-4e31-a3b8-12a39bbec3fc-tmp-dir\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:17:21.627920 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:21.627887    2577 generic.go:356] "Generic (PLEG): container finished" podID="62071b75-40ea-4e31-a3b8-12a39bbec3fc" containerID="a73e83d0dd1ce37701f2f18075edb67c45821e84c46b4a1a14c52101da64dad2" exitCode=0
Jul 20 17:17:21.628060 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:21.627925    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz"
Jul 20 17:17:21.628060 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:21.627963    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz" event={"ID":"62071b75-40ea-4e31-a3b8-12a39bbec3fc","Type":"ContainerDied","Data":"a73e83d0dd1ce37701f2f18075edb67c45821e84c46b4a1a14c52101da64dad2"}
Jul 20 17:17:21.628060 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:21.627997    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz" event={"ID":"62071b75-40ea-4e31-a3b8-12a39bbec3fc","Type":"ContainerDied","Data":"400988f74c0467b2d148100b1bc7ceb4b2c8abe5e380acf9decdddd4f13a9612"}
Jul 20 17:17:21.628060 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:21.628015    2577 scope.go:117] "RemoveContainer" containerID="a73e83d0dd1ce37701f2f18075edb67c45821e84c46b4a1a14c52101da64dad2"
Jul 20 17:17:21.637366 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:21.637344    2577 scope.go:117] "RemoveContainer" containerID="a73e83d0dd1ce37701f2f18075edb67c45821e84c46b4a1a14c52101da64dad2"
Jul 20 17:17:21.637603 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:17:21.637584    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a73e83d0dd1ce37701f2f18075edb67c45821e84c46b4a1a14c52101da64dad2\": container with ID starting with a73e83d0dd1ce37701f2f18075edb67c45821e84c46b4a1a14c52101da64dad2 not found: ID does not exist" containerID="a73e83d0dd1ce37701f2f18075edb67c45821e84c46b4a1a14c52101da64dad2"
Jul 20 17:17:21.637649 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:21.637616    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a73e83d0dd1ce37701f2f18075edb67c45821e84c46b4a1a14c52101da64dad2"} err="failed to get container status \"a73e83d0dd1ce37701f2f18075edb67c45821e84c46b4a1a14c52101da64dad2\": rpc error: code = NotFound desc = could not find container \"a73e83d0dd1ce37701f2f18075edb67c45821e84c46b4a1a14c52101da64dad2\": container with ID starting with a73e83d0dd1ce37701f2f18075edb67c45821e84c46b4a1a14c52101da64dad2 not found: ID does not exist"
Jul 20 17:17:21.662381 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:21.662357    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz"]
Jul 20 17:17:21.670043 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:21.670022    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-a16895/rate-test-model-a16895-kserve-6d4976f67f-c76vz"]
Jul 20 17:17:23.212059 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:23.212024    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62071b75-40ea-4e31-a3b8-12a39bbec3fc" path="/var/lib/kubelet/pods/62071b75-40ea-4e31-a3b8-12a39bbec3fc/volumes"
Jul 20 17:17:27.271042 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:27.271010    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:27.368149 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:27.368116    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:37.097873 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:37.097797    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a16895-56cb9799b-9n96c"]
Jul 20 17:17:37.098275 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:37.098053    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a16895-56cb9799b-9n96c" podUID="161ff40d-6008-4ffe-8e60-0d5b53567e3c" containerName="maas-api" containerID="cri-o://85d8a53d8fa34760fdeda53059cf4394db57ec773c045d34bf22862333dd4719" gracePeriod=30
Jul 20 17:17:37.147824 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:37.147792    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:37.538229 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:37.538203    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a16895-56cb9799b-9n96c"
Jul 20 17:17:37.671224 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:37.671136    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lcfht\" (UniqueName: \"kubernetes.io/projected/161ff40d-6008-4ffe-8e60-0d5b53567e3c-kube-api-access-lcfht\") pod \"161ff40d-6008-4ffe-8e60-0d5b53567e3c\" (UID: \"161ff40d-6008-4ffe-8e60-0d5b53567e3c\") "
Jul 20 17:17:37.671224 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:37.671208    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/161ff40d-6008-4ffe-8e60-0d5b53567e3c-openshift-service-ca\") pod \"161ff40d-6008-4ffe-8e60-0d5b53567e3c\" (UID: \"161ff40d-6008-4ffe-8e60-0d5b53567e3c\") "
Jul 20 17:17:37.671466 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:37.671401    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/161ff40d-6008-4ffe-8e60-0d5b53567e3c-maas-api-tls\") pod \"161ff40d-6008-4ffe-8e60-0d5b53567e3c\" (UID: \"161ff40d-6008-4ffe-8e60-0d5b53567e3c\") "
Jul 20 17:17:37.671559 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:37.671538    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/161ff40d-6008-4ffe-8e60-0d5b53567e3c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "161ff40d-6008-4ffe-8e60-0d5b53567e3c" (UID: "161ff40d-6008-4ffe-8e60-0d5b53567e3c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 17:17:37.671751 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:37.671726    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/161ff40d-6008-4ffe-8e60-0d5b53567e3c-openshift-service-ca\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:17:37.673168 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:37.673144    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/161ff40d-6008-4ffe-8e60-0d5b53567e3c-kube-api-access-lcfht" (OuterVolumeSpecName: "kube-api-access-lcfht") pod "161ff40d-6008-4ffe-8e60-0d5b53567e3c" (UID: "161ff40d-6008-4ffe-8e60-0d5b53567e3c"). InnerVolumeSpecName "kube-api-access-lcfht". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:17:37.673406 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:37.673383    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/161ff40d-6008-4ffe-8e60-0d5b53567e3c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "161ff40d-6008-4ffe-8e60-0d5b53567e3c" (UID: "161ff40d-6008-4ffe-8e60-0d5b53567e3c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 17:17:37.698383 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:37.698348    2577 generic.go:356] "Generic (PLEG): container finished" podID="161ff40d-6008-4ffe-8e60-0d5b53567e3c" containerID="85d8a53d8fa34760fdeda53059cf4394db57ec773c045d34bf22862333dd4719" exitCode=0
Jul 20 17:17:37.698479 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:37.698389    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a16895-56cb9799b-9n96c" event={"ID":"161ff40d-6008-4ffe-8e60-0d5b53567e3c","Type":"ContainerDied","Data":"85d8a53d8fa34760fdeda53059cf4394db57ec773c045d34bf22862333dd4719"}
Jul 20 17:17:37.698479 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:37.698400    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a16895-56cb9799b-9n96c"
Jul 20 17:17:37.698479 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:37.698441    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-a16895-56cb9799b-9n96c" event={"ID":"161ff40d-6008-4ffe-8e60-0d5b53567e3c","Type":"ContainerDied","Data":"6fff5e23fe5111cb39d8cbd8000488a6c6ef4aed95c09d08ac563cb5bb682744"}
Jul 20 17:17:37.698479 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:37.698471    2577 scope.go:117] "RemoveContainer" containerID="85d8a53d8fa34760fdeda53059cf4394db57ec773c045d34bf22862333dd4719"
Jul 20 17:17:37.707474 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:37.707460    2577 scope.go:117] "RemoveContainer" containerID="85d8a53d8fa34760fdeda53059cf4394db57ec773c045d34bf22862333dd4719"
Jul 20 17:17:37.707716 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:17:37.707694    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85d8a53d8fa34760fdeda53059cf4394db57ec773c045d34bf22862333dd4719\": container with ID starting with 85d8a53d8fa34760fdeda53059cf4394db57ec773c045d34bf22862333dd4719 not found: ID does not exist" containerID="85d8a53d8fa34760fdeda53059cf4394db57ec773c045d34bf22862333dd4719"
Jul 20 17:17:37.707779 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:37.707730    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85d8a53d8fa34760fdeda53059cf4394db57ec773c045d34bf22862333dd4719"} err="failed to get container status \"85d8a53d8fa34760fdeda53059cf4394db57ec773c045d34bf22862333dd4719\": rpc error: code = NotFound desc = could not find container \"85d8a53d8fa34760fdeda53059cf4394db57ec773c045d34bf22862333dd4719\": container with ID starting with 85d8a53d8fa34760fdeda53059cf4394db57ec773c045d34bf22862333dd4719 not found: ID does not exist"
Jul 20 17:17:37.720534 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:37.720503    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a16895-56cb9799b-9n96c"]
Jul 20 17:17:37.728249 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:37.728231    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-a16895-56cb9799b-9n96c"]
Jul 20 17:17:37.773014 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:37.772992    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lcfht\" (UniqueName: \"kubernetes.io/projected/161ff40d-6008-4ffe-8e60-0d5b53567e3c-kube-api-access-lcfht\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:17:37.773014 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:37.773011    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/161ff40d-6008-4ffe-8e60-0d5b53567e3c-maas-api-tls\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:17:39.212359 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:39.212324    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="161ff40d-6008-4ffe-8e60-0d5b53567e3c" path="/var/lib/kubelet/pods/161ff40d-6008-4ffe-8e60-0d5b53567e3c/volumes"
Jul 20 17:17:41.675339 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:41.675287    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:42.095885 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:42.095857    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:48.431990 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:48.431955    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:48.506863 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:48.506821    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:48.746337 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:48.746219    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"]
Jul 20 17:17:48.746619 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:48.746561    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4" podUID="1893d618-2628-4baa-a0de-803c6b15be38" containerName="istio-proxy" containerID="cri-o://09c665d4873ddd07e12064f7b7544453d667df544c12317b6d26773b43213797" gracePeriod=30
Jul 20 17:17:48.779108 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:48.779071    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:49.068990 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:49.068954    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:49.858718 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:49.858683    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:50.279352 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:50.279252    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:17:53.994637 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:53.994613    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:17:54.107150 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.107125    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1893d618-2628-4baa-a0de-803c6b15be38-istio-envoy\") pod \"1893d618-2628-4baa-a0de-803c6b15be38\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") "
Jul 20 17:17:54.107150 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.107156    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1893d618-2628-4baa-a0de-803c6b15be38-istio-token\") pod \"1893d618-2628-4baa-a0de-803c6b15be38\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") "
Jul 20 17:17:54.107393 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.107180    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1893d618-2628-4baa-a0de-803c6b15be38-istio-podinfo\") pod \"1893d618-2628-4baa-a0de-803c6b15be38\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") "
Jul 20 17:17:54.107393 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.107205    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1893d618-2628-4baa-a0de-803c6b15be38-istio-data\") pod \"1893d618-2628-4baa-a0de-803c6b15be38\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") "
Jul 20 17:17:54.107393 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.107228    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rlp9m\" (UniqueName: \"kubernetes.io/projected/1893d618-2628-4baa-a0de-803c6b15be38-kube-api-access-rlp9m\") pod \"1893d618-2628-4baa-a0de-803c6b15be38\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") "
Jul 20 17:17:54.107393 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.107252    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1893d618-2628-4baa-a0de-803c6b15be38-istiod-ca-cert\") pod \"1893d618-2628-4baa-a0de-803c6b15be38\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") "
Jul 20 17:17:54.107393 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.107339    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1893d618-2628-4baa-a0de-803c6b15be38-workload-socket\") pod \"1893d618-2628-4baa-a0de-803c6b15be38\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") "
Jul 20 17:17:54.107393 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.107375    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1893d618-2628-4baa-a0de-803c6b15be38-workload-certs\") pod \"1893d618-2628-4baa-a0de-803c6b15be38\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") "
Jul 20 17:17:54.107709 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.107432    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1893d618-2628-4baa-a0de-803c6b15be38-credential-socket\") pod \"1893d618-2628-4baa-a0de-803c6b15be38\" (UID: \"1893d618-2628-4baa-a0de-803c6b15be38\") "
Jul 20 17:17:54.107776 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.107698    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1893d618-2628-4baa-a0de-803c6b15be38-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1893d618-2628-4baa-a0de-803c6b15be38" (UID: "1893d618-2628-4baa-a0de-803c6b15be38"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 17:17:54.108112 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.107858    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1893d618-2628-4baa-a0de-803c6b15be38-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1893d618-2628-4baa-a0de-803c6b15be38" (UID: "1893d618-2628-4baa-a0de-803c6b15be38"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:17:54.108112 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.107868    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1893d618-2628-4baa-a0de-803c6b15be38-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1893d618-2628-4baa-a0de-803c6b15be38" (UID: "1893d618-2628-4baa-a0de-803c6b15be38"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:17:54.108112 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.108075    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1893d618-2628-4baa-a0de-803c6b15be38-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1893d618-2628-4baa-a0de-803c6b15be38" (UID: "1893d618-2628-4baa-a0de-803c6b15be38"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:17:54.108362 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.108124    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1893d618-2628-4baa-a0de-803c6b15be38-istio-data" (OuterVolumeSpecName: "istio-data") pod "1893d618-2628-4baa-a0de-803c6b15be38" (UID: "1893d618-2628-4baa-a0de-803c6b15be38"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:17:54.109443 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.109422    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1893d618-2628-4baa-a0de-803c6b15be38-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1893d618-2628-4baa-a0de-803c6b15be38" (UID: "1893d618-2628-4baa-a0de-803c6b15be38"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 20 17:17:54.109621 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.109599    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1893d618-2628-4baa-a0de-803c6b15be38-kube-api-access-rlp9m" (OuterVolumeSpecName: "kube-api-access-rlp9m") pod "1893d618-2628-4baa-a0de-803c6b15be38" (UID: "1893d618-2628-4baa-a0de-803c6b15be38"). InnerVolumeSpecName "kube-api-access-rlp9m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:17:54.109841 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.109824    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1893d618-2628-4baa-a0de-803c6b15be38-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1893d618-2628-4baa-a0de-803c6b15be38" (UID: "1893d618-2628-4baa-a0de-803c6b15be38"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:17:54.109891 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.109831    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1893d618-2628-4baa-a0de-803c6b15be38-istio-token" (OuterVolumeSpecName: "istio-token") pod "1893d618-2628-4baa-a0de-803c6b15be38" (UID: "1893d618-2628-4baa-a0de-803c6b15be38"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:17:54.208722 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.208696    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1893d618-2628-4baa-a0de-803c6b15be38-istio-envoy\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:17:54.208722 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.208720    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1893d618-2628-4baa-a0de-803c6b15be38-istio-token\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:17:54.208722 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.208729    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1893d618-2628-4baa-a0de-803c6b15be38-istio-podinfo\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:17:54.208939 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.208736    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1893d618-2628-4baa-a0de-803c6b15be38-istio-data\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:17:54.208939 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.208745    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlp9m\" (UniqueName: \"kubernetes.io/projected/1893d618-2628-4baa-a0de-803c6b15be38-kube-api-access-rlp9m\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:17:54.208939 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.208753    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1893d618-2628-4baa-a0de-803c6b15be38-istiod-ca-cert\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:17:54.208939 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.208761    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1893d618-2628-4baa-a0de-803c6b15be38-workload-socket\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:17:54.208939 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.208769    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1893d618-2628-4baa-a0de-803c6b15be38-workload-certs\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:17:54.208939 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.208777    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1893d618-2628-4baa-a0de-803c6b15be38-credential-socket\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:17:54.775548 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.775509    2577 generic.go:356] "Generic (PLEG): container finished" podID="1893d618-2628-4baa-a0de-803c6b15be38" containerID="09c665d4873ddd07e12064f7b7544453d667df544c12317b6d26773b43213797" exitCode=0
Jul 20 17:17:54.775718 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.775563    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"
Jul 20 17:17:54.775718 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.775566    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4" event={"ID":"1893d618-2628-4baa-a0de-803c6b15be38","Type":"ContainerDied","Data":"09c665d4873ddd07e12064f7b7544453d667df544c12317b6d26773b43213797"}
Jul 20 17:17:54.775718 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.775629    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4" event={"ID":"1893d618-2628-4baa-a0de-803c6b15be38","Type":"ContainerDied","Data":"db6b6ca71d8e3ca13f3ecea6e1c838beb5274c908912631e26b1d36e3e04ea67"}
Jul 20 17:17:54.775718 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.775659    2577 scope.go:117] "RemoveContainer" containerID="09c665d4873ddd07e12064f7b7544453d667df544c12317b6d26773b43213797"
Jul 20 17:17:54.786666 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.786641    2577 scope.go:117] "RemoveContainer" containerID="0f966d33fedf2be859a0d310fc6d20bc4a14c61fa73d7d66a20fe32431b49272"
Jul 20 17:17:54.796942 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.796922    2577 scope.go:117] "RemoveContainer" containerID="09c665d4873ddd07e12064f7b7544453d667df544c12317b6d26773b43213797"
Jul 20 17:17:54.797252 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:17:54.797232    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09c665d4873ddd07e12064f7b7544453d667df544c12317b6d26773b43213797\": container with ID starting with 09c665d4873ddd07e12064f7b7544453d667df544c12317b6d26773b43213797 not found: ID does not exist" containerID="09c665d4873ddd07e12064f7b7544453d667df544c12317b6d26773b43213797"
Jul 20 17:17:54.797333 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.797259    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09c665d4873ddd07e12064f7b7544453d667df544c12317b6d26773b43213797"} err="failed to get container status \"09c665d4873ddd07e12064f7b7544453d667df544c12317b6d26773b43213797\": rpc error: code = NotFound desc = could not find container \"09c665d4873ddd07e12064f7b7544453d667df544c12317b6d26773b43213797\": container with ID starting with 09c665d4873ddd07e12064f7b7544453d667df544c12317b6d26773b43213797 not found: ID does not exist"
Jul 20 17:17:54.797333 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.797274    2577 scope.go:117] "RemoveContainer" containerID="0f966d33fedf2be859a0d310fc6d20bc4a14c61fa73d7d66a20fe32431b49272"
Jul 20 17:17:54.797555 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:17:54.797539    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f966d33fedf2be859a0d310fc6d20bc4a14c61fa73d7d66a20fe32431b49272\": container with ID starting with 0f966d33fedf2be859a0d310fc6d20bc4a14c61fa73d7d66a20fe32431b49272 not found: ID does not exist" containerID="0f966d33fedf2be859a0d310fc6d20bc4a14c61fa73d7d66a20fe32431b49272"
Jul 20 17:17:54.797605 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.797559    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f966d33fedf2be859a0d310fc6d20bc4a14c61fa73d7d66a20fe32431b49272"} err="failed to get container status \"0f966d33fedf2be859a0d310fc6d20bc4a14c61fa73d7d66a20fe32431b49272\": rpc error: code = NotFound desc = could not find container \"0f966d33fedf2be859a0d310fc6d20bc4a14c61fa73d7d66a20fe32431b49272\": container with ID starting with 0f966d33fedf2be859a0d310fc6d20bc4a14c61fa73d7d66a20fe32431b49272 not found: ID does not exist"
Jul 20 17:17:54.802675 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.802652    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"]
Jul 20 17:17:54.809839 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:54.809817    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-a16895-openshift-default-7f88fdbf95-r2hh4"]
Jul 20 17:17:55.211562 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:55.211531    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1893d618-2628-4baa-a0de-803c6b15be38" path="/var/lib/kubelet/pods/1893d618-2628-4baa-a0de-803c6b15be38/volumes"
Jul 20 17:17:59.808436 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:17:59.808402    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:18:04.353745 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:04.353709    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:18:04.795459 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:04.795432    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:18:11.449604 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:11.449569    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:18:11.482639 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:11.482599    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"]
Jul 20 17:18:11.482860 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:11.482838    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg" podUID="f9073619-7c5a-46a3-890b-e5fd42e7d35f" containerName="istio-proxy" containerID="cri-o://7decc791be6dae83e4d2752cc97517a4fb831b506758f68d630961fb5c33d0ae" gracePeriod=30
Jul 20 17:18:11.550253 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:11.550215    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:18:11.755998 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:11.755917    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:18:13.183733 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:13.183694    2577 patch_prober.go:28] interesting pod/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 20 17:18:13.184117 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:13.183757    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg" podUID="f9073619-7c5a-46a3-890b-e5fd42e7d35f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 20 17:18:16.726466 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.726434    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:18:16.806404 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.806379    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9073619-7c5a-46a3-890b-e5fd42e7d35f-istio-podinfo\") pod \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") "
Jul 20 17:18:16.806523 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.806430    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9073619-7c5a-46a3-890b-e5fd42e7d35f-workload-certs\") pod \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") "
Jul 20 17:18:16.806523 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.806456    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8n4q\" (UniqueName: \"kubernetes.io/projected/f9073619-7c5a-46a3-890b-e5fd42e7d35f-kube-api-access-l8n4q\") pod \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") "
Jul 20 17:18:16.806523 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.806479    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9073619-7c5a-46a3-890b-e5fd42e7d35f-istio-envoy\") pod \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") "
Jul 20 17:18:16.806523 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.806506    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9073619-7c5a-46a3-890b-e5fd42e7d35f-istio-data\") pod \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") "
Jul 20 17:18:16.806746 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.806564    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9073619-7c5a-46a3-890b-e5fd42e7d35f-workload-socket\") pod \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") "
Jul 20 17:18:16.806746 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.806620    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9073619-7c5a-46a3-890b-e5fd42e7d35f-istiod-ca-cert\") pod \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") "
Jul 20 17:18:16.806746 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.806652    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9073619-7c5a-46a3-890b-e5fd42e7d35f-istio-token\") pod \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") "
Jul 20 17:18:16.806746 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.806699    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9073619-7c5a-46a3-890b-e5fd42e7d35f-credential-socket\") pod \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\" (UID: \"f9073619-7c5a-46a3-890b-e5fd42e7d35f\") "
Jul 20 17:18:16.807020 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.806817    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9073619-7c5a-46a3-890b-e5fd42e7d35f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f9073619-7c5a-46a3-890b-e5fd42e7d35f" (UID: "f9073619-7c5a-46a3-890b-e5fd42e7d35f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:18:16.807020 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.806872    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9073619-7c5a-46a3-890b-e5fd42e7d35f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f9073619-7c5a-46a3-890b-e5fd42e7d35f" (UID: "f9073619-7c5a-46a3-890b-e5fd42e7d35f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:18:16.807140 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.807085    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9073619-7c5a-46a3-890b-e5fd42e7d35f-workload-certs\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:18:16.807140 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.807109    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9073619-7c5a-46a3-890b-e5fd42e7d35f-workload-socket\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:18:16.807242 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.807168    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9073619-7c5a-46a3-890b-e5fd42e7d35f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f9073619-7c5a-46a3-890b-e5fd42e7d35f" (UID: "f9073619-7c5a-46a3-890b-e5fd42e7d35f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 17:18:16.807359 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.807334    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9073619-7c5a-46a3-890b-e5fd42e7d35f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f9073619-7c5a-46a3-890b-e5fd42e7d35f" (UID: "f9073619-7c5a-46a3-890b-e5fd42e7d35f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:18:16.807766 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.807737    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9073619-7c5a-46a3-890b-e5fd42e7d35f-istio-data" (OuterVolumeSpecName: "istio-data") pod "f9073619-7c5a-46a3-890b-e5fd42e7d35f" (UID: "f9073619-7c5a-46a3-890b-e5fd42e7d35f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:18:16.808677 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.808646    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f9073619-7c5a-46a3-890b-e5fd42e7d35f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f9073619-7c5a-46a3-890b-e5fd42e7d35f" (UID: "f9073619-7c5a-46a3-890b-e5fd42e7d35f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 20 17:18:16.808677 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.808659    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9073619-7c5a-46a3-890b-e5fd42e7d35f-kube-api-access-l8n4q" (OuterVolumeSpecName: "kube-api-access-l8n4q") pod "f9073619-7c5a-46a3-890b-e5fd42e7d35f" (UID: "f9073619-7c5a-46a3-890b-e5fd42e7d35f"). InnerVolumeSpecName "kube-api-access-l8n4q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:18:16.808818 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.808726    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9073619-7c5a-46a3-890b-e5fd42e7d35f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f9073619-7c5a-46a3-890b-e5fd42e7d35f" (UID: "f9073619-7c5a-46a3-890b-e5fd42e7d35f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 17:18:16.808942 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.808923    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9073619-7c5a-46a3-890b-e5fd42e7d35f-istio-token" (OuterVolumeSpecName: "istio-token") pod "f9073619-7c5a-46a3-890b-e5fd42e7d35f" (UID: "f9073619-7c5a-46a3-890b-e5fd42e7d35f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 17:18:16.870695 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.870667    2577 generic.go:356] "Generic (PLEG): container finished" podID="f9073619-7c5a-46a3-890b-e5fd42e7d35f" containerID="7decc791be6dae83e4d2752cc97517a4fb831b506758f68d630961fb5c33d0ae" exitCode=0
Jul 20 17:18:16.870790 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.870710    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"
Jul 20 17:18:16.870790 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.870726    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg" event={"ID":"f9073619-7c5a-46a3-890b-e5fd42e7d35f","Type":"ContainerDied","Data":"7decc791be6dae83e4d2752cc97517a4fb831b506758f68d630961fb5c33d0ae"}
Jul 20 17:18:16.870790 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.870768    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg" event={"ID":"f9073619-7c5a-46a3-890b-e5fd42e7d35f","Type":"ContainerDied","Data":"9c7a37bd10ced21d41e71602a006d13dc5a5b6e8382f569656cb0e8bbb9361b8"}
Jul 20 17:18:16.870790 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.870789    2577 scope.go:117] "RemoveContainer" containerID="7decc791be6dae83e4d2752cc97517a4fb831b506758f68d630961fb5c33d0ae"
Jul 20 17:18:16.879610 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.879593    2577 scope.go:117] "RemoveContainer" containerID="e4ba2da396da97e6e9a1b2eaa39e4c7024930204ecb2809875db5d82347d492d"
Jul 20 17:18:16.886807 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.886789    2577 scope.go:117] "RemoveContainer" containerID="7decc791be6dae83e4d2752cc97517a4fb831b506758f68d630961fb5c33d0ae"
Jul 20 17:18:16.887066 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:18:16.887047    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7decc791be6dae83e4d2752cc97517a4fb831b506758f68d630961fb5c33d0ae\": container with ID starting with 7decc791be6dae83e4d2752cc97517a4fb831b506758f68d630961fb5c33d0ae not found: ID does not exist" containerID="7decc791be6dae83e4d2752cc97517a4fb831b506758f68d630961fb5c33d0ae"
Jul 20 17:18:16.887114 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.887075    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7decc791be6dae83e4d2752cc97517a4fb831b506758f68d630961fb5c33d0ae"} err="failed to get container status \"7decc791be6dae83e4d2752cc97517a4fb831b506758f68d630961fb5c33d0ae\": rpc error: code = NotFound desc = could not find container \"7decc791be6dae83e4d2752cc97517a4fb831b506758f68d630961fb5c33d0ae\": container with ID starting with 7decc791be6dae83e4d2752cc97517a4fb831b506758f68d630961fb5c33d0ae not found: ID does not exist"
Jul 20 17:18:16.887114 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.887092    2577 scope.go:117] "RemoveContainer" containerID="e4ba2da396da97e6e9a1b2eaa39e4c7024930204ecb2809875db5d82347d492d"
Jul 20 17:18:16.887336 ip-10-0-130-85 kubenswrapper[2577]: E0720 17:18:16.887315    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4ba2da396da97e6e9a1b2eaa39e4c7024930204ecb2809875db5d82347d492d\": container with ID starting with e4ba2da396da97e6e9a1b2eaa39e4c7024930204ecb2809875db5d82347d492d not found: ID does not exist" containerID="e4ba2da396da97e6e9a1b2eaa39e4c7024930204ecb2809875db5d82347d492d"
Jul 20 17:18:16.887417 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.887339    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4ba2da396da97e6e9a1b2eaa39e4c7024930204ecb2809875db5d82347d492d"} err="failed to get container status \"e4ba2da396da97e6e9a1b2eaa39e4c7024930204ecb2809875db5d82347d492d\": rpc error: code = NotFound desc = could not find container \"e4ba2da396da97e6e9a1b2eaa39e4c7024930204ecb2809875db5d82347d492d\": container with ID starting with e4ba2da396da97e6e9a1b2eaa39e4c7024930204ecb2809875db5d82347d492d not found: ID does not exist"
Jul 20 17:18:16.893701 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.893680    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"]
Jul 20 17:18:16.899479 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.899459    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-e142d5-openshift-default-675dd5894c-dtmlg"]
Jul 20 17:18:16.908238 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.908219    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8n4q\" (UniqueName: \"kubernetes.io/projected/f9073619-7c5a-46a3-890b-e5fd42e7d35f-kube-api-access-l8n4q\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:18:16.908238 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.908239    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9073619-7c5a-46a3-890b-e5fd42e7d35f-istio-envoy\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:18:16.908392 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.908248    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9073619-7c5a-46a3-890b-e5fd42e7d35f-istio-data\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:18:16.908392 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.908256    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9073619-7c5a-46a3-890b-e5fd42e7d35f-istiod-ca-cert\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:18:16.908392 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.908266    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9073619-7c5a-46a3-890b-e5fd42e7d35f-istio-token\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:18:16.908392 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.908274    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9073619-7c5a-46a3-890b-e5fd42e7d35f-credential-socket\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:18:16.908392 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:16.908283    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9073619-7c5a-46a3-890b-e5fd42e7d35f-istio-podinfo\") on node \"ip-10-0-130-85.ec2.internal\" DevicePath \"\""
Jul 20 17:18:17.213581 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:17.213481    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9073619-7c5a-46a3-890b-e5fd42e7d35f" path="/var/lib/kubelet/pods/f9073619-7c5a-46a3-890b-e5fd42e7d35f/volumes"
Jul 20 17:18:19.607033 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:19.607003    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9d55867bb-nsh92_dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b/copy-manifests/0.log"
Jul 20 17:18:19.620833 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:19.620807    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9d55867bb-nsh92_dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b/manager/0.log"
Jul 20 17:18:20.205533 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:20.205505    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29742765-7x5tj_8bfc237b-41c2-40b3-82f9-596643dcb4eb/cleanup/0.log"
Jul 20 17:18:20.621136 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:20.621104    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-qhrgc_826b421a-e9dc-44c8-bfe1-5ebffaf759a6/postgres/0.log"
Jul 20 17:18:22.598201 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:22.598172    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-t79zm_cea7bc8f-75a2-4f2b-a14d-8ead22da6a5d/kuadrant-console-plugin/0.log"
Jul 20 17:18:24.031184 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:24.031154    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-745b54bffd-zgzf9_7f988b64-f729-4c51-afee-366220e6ffcf/kube-auth-proxy/0.log"
Jul 20 17:18:24.441503 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:24.441464    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-64flz_750883b9-48a6-4050-b1ee-28f3d7d9eda8/payload-processing/0.log"
Jul 20 17:18:24.981040 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:24.981007    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-4qnxs_3e53b8d7-5bb7-4d2e-8524-a755abd79661/main/0.log"
Jul 20 17:18:25.594778 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:25.594745    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-gqsvg_e12a1a47-4c70-4eb3-ad22-10711533dc83/main/0.log"
Jul 20 17:18:31.753786 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:31.753740    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:18:31.939921 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:31.939888    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log" limit=1024
Jul 20 17:18:35.133959 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:35.133920    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4957k_1230abfc-e6e1-432a-b99e-2d700df05d26/global-pull-secret-syncer/0.log"
Jul 20 17:18:35.383329 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:35.383186    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-82fz4_dc7c0532-d1ad-4a0d-b63b-5a952a7cae21/konnectivity-agent/0.log"
Jul 20 17:18:35.593052 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:35.593023    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-85.ec2.internal_786ae2d571f496bce3302e9386a601e9/haproxy/0.log"
Jul 20 17:18:41.001537 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:41.001515    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-t79zm_cea7bc8f-75a2-4f2b-a14d-8ead22da6a5d/kuadrant-console-plugin/0.log"
Jul 20 17:18:43.528274 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:43.528182    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-mn2jt_57efcb32-98cb-44fd-9044-1647dfb1f735/kube-state-metrics/0.log"
Jul 20 17:18:43.558382 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:43.558358    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-mn2jt_57efcb32-98cb-44fd-9044-1647dfb1f735/kube-rbac-proxy-main/0.log"
Jul 20 17:18:43.586724 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:43.586678    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-mn2jt_57efcb32-98cb-44fd-9044-1647dfb1f735/kube-rbac-proxy-self/0.log"
Jul 20 17:18:43.769068 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:43.769034    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-drtfq_112aad28-f9b6-415e-ac47-3334d4c2a15e/node-exporter/0.log"
Jul 20 17:18:43.797822 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:43.797804    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-drtfq_112aad28-f9b6-415e-ac47-3334d4c2a15e/kube-rbac-proxy/0.log"
Jul 20 17:18:43.825712 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:43.825687    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-drtfq_112aad28-f9b6-415e-ac47-3334d4c2a15e/init-textfile/0.log"
Jul 20 17:18:44.064437 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:44.064366    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e8dd70e-cbec-4e9a-869a-0a8b56f0b257/prometheus/0.log"
Jul 20 17:18:44.096616 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:44.096588    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e8dd70e-cbec-4e9a-869a-0a8b56f0b257/config-reloader/0.log"
Jul 20 17:18:44.129903 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:44.129868    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e8dd70e-cbec-4e9a-869a-0a8b56f0b257/thanos-sidecar/0.log"
Jul 20 17:18:44.166216 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:44.166182    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e8dd70e-cbec-4e9a-869a-0a8b56f0b257/kube-rbac-proxy-web/0.log"
Jul 20 17:18:44.192667 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:44.192645    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e8dd70e-cbec-4e9a-869a-0a8b56f0b257/kube-rbac-proxy/0.log"
Jul 20 17:18:44.221110 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:44.221088    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e8dd70e-cbec-4e9a-869a-0a8b56f0b257/kube-rbac-proxy-thanos/0.log"
Jul 20 17:18:44.253882 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:44.253859    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e8dd70e-cbec-4e9a-869a-0a8b56f0b257/init-config-reloader/0.log"
Jul 20 17:18:45.163931 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.163893    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-m6c6l/perf-node-gather-daemonset-q6wzn"]
Jul 20 17:18:45.164399 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.164339    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1893d618-2628-4baa-a0de-803c6b15be38" containerName="istio-proxy"
Jul 20 17:18:45.164399 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.164357    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1893d618-2628-4baa-a0de-803c6b15be38" containerName="istio-proxy"
Jul 20 17:18:45.164399 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.164389    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62071b75-40ea-4e31-a3b8-12a39bbec3fc" containerName="main"
Jul 20 17:18:45.164399 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.164398    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="62071b75-40ea-4e31-a3b8-12a39bbec3fc" containerName="main"
Jul 20 17:18:45.164623 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.164412    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9073619-7c5a-46a3-890b-e5fd42e7d35f" containerName="istio-proxy"
Jul 20 17:18:45.164623 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.164420    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9073619-7c5a-46a3-890b-e5fd42e7d35f" containerName="istio-proxy"
Jul 20 17:18:45.164623 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.164435    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9073619-7c5a-46a3-890b-e5fd42e7d35f" containerName="istio-proxy"
Jul 20 17:18:45.164623 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.164444    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9073619-7c5a-46a3-890b-e5fd42e7d35f" containerName="istio-proxy"
Jul 20 17:18:45.164623 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.164459    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="161ff40d-6008-4ffe-8e60-0d5b53567e3c" containerName="maas-api"
Jul 20 17:18:45.164623 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.164468    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="161ff40d-6008-4ffe-8e60-0d5b53567e3c" containerName="maas-api"
Jul 20 17:18:45.164623 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.164540    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1893d618-2628-4baa-a0de-803c6b15be38" containerName="istio-proxy"
Jul 20 17:18:45.164623 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.164555    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1893d618-2628-4baa-a0de-803c6b15be38" containerName="istio-proxy"
Jul 20 17:18:45.164623 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.164565    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9073619-7c5a-46a3-890b-e5fd42e7d35f" containerName="istio-proxy"
Jul 20 17:18:45.164623 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.164575    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="62071b75-40ea-4e31-a3b8-12a39bbec3fc" containerName="main"
Jul 20 17:18:45.164623 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.164584    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="161ff40d-6008-4ffe-8e60-0d5b53567e3c" containerName="maas-api"
Jul 20 17:18:45.169509 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.169490    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-m6c6l/perf-node-gather-daemonset-q6wzn"
Jul 20 17:18:45.173360 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.173340    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-m6c6l\"/\"default-dockercfg-szprv\""
Jul 20 17:18:45.174013 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.173999    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-m6c6l\"/\"kube-root-ca.crt\""
Jul 20 17:18:45.174074 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.174020    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-m6c6l\"/\"openshift-service-ca.crt\""
Jul 20 17:18:45.188309 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.188274    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-m6c6l/perf-node-gather-daemonset-q6wzn"]
Jul 20 17:18:45.246837 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.246810    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/247fad6a-20b6-409e-a259-d550fb53b6fe-lib-modules\") pod \"perf-node-gather-daemonset-q6wzn\" (UID: \"247fad6a-20b6-409e-a259-d550fb53b6fe\") " pod="openshift-must-gather-m6c6l/perf-node-gather-daemonset-q6wzn"
Jul 20 17:18:45.247006 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.246953    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8nfx\" (UniqueName: \"kubernetes.io/projected/247fad6a-20b6-409e-a259-d550fb53b6fe-kube-api-access-x8nfx\") pod \"perf-node-gather-daemonset-q6wzn\" (UID: \"247fad6a-20b6-409e-a259-d550fb53b6fe\") " pod="openshift-must-gather-m6c6l/perf-node-gather-daemonset-q6wzn"
Jul 20 17:18:45.247006 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.247001    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/247fad6a-20b6-409e-a259-d550fb53b6fe-sys\") pod \"perf-node-gather-daemonset-q6wzn\" (UID: \"247fad6a-20b6-409e-a259-d550fb53b6fe\") " pod="openshift-must-gather-m6c6l/perf-node-gather-daemonset-q6wzn"
Jul 20 17:18:45.247088 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.247068    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/247fad6a-20b6-409e-a259-d550fb53b6fe-proc\") pod \"perf-node-gather-daemonset-q6wzn\" (UID: \"247fad6a-20b6-409e-a259-d550fb53b6fe\") " pod="openshift-must-gather-m6c6l/perf-node-gather-daemonset-q6wzn"
Jul 20 17:18:45.247176 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.247157    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/247fad6a-20b6-409e-a259-d550fb53b6fe-podres\") pod \"perf-node-gather-daemonset-q6wzn\" (UID: \"247fad6a-20b6-409e-a259-d550fb53b6fe\") " pod="openshift-must-gather-m6c6l/perf-node-gather-daemonset-q6wzn"
Jul 20 17:18:45.347785 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.347748    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/247fad6a-20b6-409e-a259-d550fb53b6fe-podres\") pod \"perf-node-gather-daemonset-q6wzn\" (UID: \"247fad6a-20b6-409e-a259-d550fb53b6fe\") " pod="openshift-must-gather-m6c6l/perf-node-gather-daemonset-q6wzn"
Jul 20 17:18:45.347957 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.347795    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/247fad6a-20b6-409e-a259-d550fb53b6fe-lib-modules\") pod \"perf-node-gather-daemonset-q6wzn\" (UID: \"247fad6a-20b6-409e-a259-d550fb53b6fe\") " pod="openshift-must-gather-m6c6l/perf-node-gather-daemonset-q6wzn"
Jul 20 17:18:45.347957 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.347831    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8nfx\" (UniqueName: \"kubernetes.io/projected/247fad6a-20b6-409e-a259-d550fb53b6fe-kube-api-access-x8nfx\") pod \"perf-node-gather-daemonset-q6wzn\" (UID: \"247fad6a-20b6-409e-a259-d550fb53b6fe\") " pod="openshift-must-gather-m6c6l/perf-node-gather-daemonset-q6wzn"
Jul 20 17:18:45.347957 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.347864    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/247fad6a-20b6-409e-a259-d550fb53b6fe-sys\") pod \"perf-node-gather-daemonset-q6wzn\" (UID: \"247fad6a-20b6-409e-a259-d550fb53b6fe\") " pod="openshift-must-gather-m6c6l/perf-node-gather-daemonset-q6wzn"
Jul 20 17:18:45.347957 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.347893    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/247fad6a-20b6-409e-a259-d550fb53b6fe-proc\") pod \"perf-node-gather-daemonset-q6wzn\" (UID: \"247fad6a-20b6-409e-a259-d550fb53b6fe\") " pod="openshift-must-gather-m6c6l/perf-node-gather-daemonset-q6wzn"
Jul 20 17:18:45.347957 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.347906    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/247fad6a-20b6-409e-a259-d550fb53b6fe-podres\") pod \"perf-node-gather-daemonset-q6wzn\" (UID: \"247fad6a-20b6-409e-a259-d550fb53b6fe\") " pod="openshift-must-gather-m6c6l/perf-node-gather-daemonset-q6wzn"
Jul 20 17:18:45.347957 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.347958    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/247fad6a-20b6-409e-a259-d550fb53b6fe-proc\") pod \"perf-node-gather-daemonset-q6wzn\" (UID: \"247fad6a-20b6-409e-a259-d550fb53b6fe\") " pod="openshift-must-gather-m6c6l/perf-node-gather-daemonset-q6wzn"
Jul 20 17:18:45.348240 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.347961    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/247fad6a-20b6-409e-a259-d550fb53b6fe-sys\") pod \"perf-node-gather-daemonset-q6wzn\" (UID: \"247fad6a-20b6-409e-a259-d550fb53b6fe\") " pod="openshift-must-gather-m6c6l/perf-node-gather-daemonset-q6wzn"
Jul 20 17:18:45.348240 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.347982    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/247fad6a-20b6-409e-a259-d550fb53b6fe-lib-modules\") pod \"perf-node-gather-daemonset-q6wzn\" (UID: \"247fad6a-20b6-409e-a259-d550fb53b6fe\") " pod="openshift-must-gather-m6c6l/perf-node-gather-daemonset-q6wzn"
Jul 20 17:18:45.357241 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.357214    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8nfx\" (UniqueName: \"kubernetes.io/projected/247fad6a-20b6-409e-a259-d550fb53b6fe-kube-api-access-x8nfx\") pod \"perf-node-gather-daemonset-q6wzn\" (UID: \"247fad6a-20b6-409e-a259-d550fb53b6fe\") " pod="openshift-must-gather-m6c6l/perf-node-gather-daemonset-q6wzn"
Jul 20 17:18:45.478780 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.478689    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-m6c6l/perf-node-gather-daemonset-q6wzn"
Jul 20 17:18:45.607877 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.607849    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-m6c6l/perf-node-gather-daemonset-q6wzn"]
Jul 20 17:18:45.610481 ip-10-0-130-85 kubenswrapper[2577]: W0720 17:18:45.610454    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod247fad6a_20b6_409e_a259_d550fb53b6fe.slice/crio-6339f6e5d15d64ca2bce82620b6218a569fb7bdf1464ac126de4a568866ff5d7 WatchSource:0}: Error finding container 6339f6e5d15d64ca2bce82620b6218a569fb7bdf1464ac126de4a568866ff5d7: Status 404 returned error can't find the container with id 6339f6e5d15d64ca2bce82620b6218a569fb7bdf1464ac126de4a568866ff5d7
Jul 20 17:18:45.999430 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.999334    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-m6c6l/perf-node-gather-daemonset-q6wzn" event={"ID":"247fad6a-20b6-409e-a259-d550fb53b6fe","Type":"ContainerStarted","Data":"bbd022ba406a1d6712a8fdc3c60aa00e14c91b0750eb108b0bc469979aebe8c9"}
Jul 20 17:18:45.999430 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.999392    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-m6c6l/perf-node-gather-daemonset-q6wzn"
Jul 20 17:18:45.999430 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:45.999412    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-m6c6l/perf-node-gather-daemonset-q6wzn" event={"ID":"247fad6a-20b6-409e-a259-d550fb53b6fe","Type":"ContainerStarted","Data":"6339f6e5d15d64ca2bce82620b6218a569fb7bdf1464ac126de4a568866ff5d7"}
Jul 20 17:18:46.021028 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:46.020979    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-m6c6l/perf-node-gather-daemonset-q6wzn" podStartSLOduration=1.020965492 podStartE2EDuration="1.020965492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 17:18:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 17:18:46.018697119 +0000 UTC m=+2949.268191138" watchObservedRunningTime="2026-07-20 17:18:46.020965492 +0000 UTC m=+2949.270459511"
Jul 20 17:18:48.906397 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:48.906366    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jzd8m_41c8f82b-bd64-40ff-a6c2-ce412321d3fa/dns/0.log"
Jul 20 17:18:48.929984 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:48.929959    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jzd8m_41c8f82b-bd64-40ff-a6c2-ce412321d3fa/kube-rbac-proxy/0.log"
Jul 20 17:18:49.047979 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:49.047955    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-h568x_650ecc59-8e80-424e-b8fd-1e285a0cda1a/dns-node-resolver/0.log"
Jul 20 17:18:49.760541 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:49.760509    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-c685fb89f-tgvnt_5498a318-5a5f-487f-a788-17bb93a8bd4e/registry/0.log"
Jul 20 17:18:49.816820 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:49.816792    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qq95v_d8418755-37c8-4c02-bdcd-1ac4c903c6ab/node-ca/0.log"
Jul 20 17:18:51.139941 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:51.139913    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-745b54bffd-zgzf9_7f988b64-f729-4c51-afee-366220e6ffcf/kube-auth-proxy/0.log"
Jul 20 17:18:51.309415 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:51.309377    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-64flz_750883b9-48a6-4050-b1ee-28f3d7d9eda8/payload-processing/0.log"
Jul 20 17:18:52.012258 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:52.012232    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-m6c6l/perf-node-gather-daemonset-q6wzn"
Jul 20 17:18:52.084537 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:52.084512    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kv4gz_93c45db6-1931-4880-9920-621adf7376a9/serve-healthcheck-canary/0.log"
Jul 20 17:18:52.688601 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:52.688567    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bd59c_3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8/kube-rbac-proxy/0.log"
Jul 20 17:18:52.723719 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:52.723695    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bd59c_3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8/exporter/0.log"
Jul 20 17:18:52.763360 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:52.763330    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bd59c_3f4098ce-46e2-4f61-8ae3-3a3f71b3abc8/extractor/0.log"
Jul 20 17:18:56.187336 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:56.187292    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-745b54bffd-zgzf9_7f988b64-f729-4c51-afee-366220e6ffcf/kube-auth-proxy/0.log"
Jul 20 17:18:56.344587 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:56.344555    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-64flz_750883b9-48a6-4050-b1ee-28f3d7d9eda8/payload-processing/0.log"
Jul 20 17:18:56.524524 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:56.524453    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9d55867bb-nsh92_dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b/manager/0.log"
Jul 20 17:18:56.548930 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:56.548903    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9d55867bb-nsh92_dab53aaf-8e6f-44a2-9e2c-668cc09bbb8b/copy-manifests/0.log"
Jul 20 17:18:57.836526 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:57.836488    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-745b54bffd-zgzf9_7f988b64-f729-4c51-afee-366220e6ffcf/kube-auth-proxy/0.log"
Jul 20 17:18:57.851393 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:57.851368    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6775c44bcb-msq95_a9c37908-6c71-4dae-a2ca-2c097a3022a2/manager/0.log"
Jul 20 17:18:57.977783 ip-10-0-130-85 kubenswrapper[2577]: I0720 17:18:57.977752    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-64flz_750883b9-48a6-4050-b1ee-28f3d7d9eda8/payload-processing/0.log"