Sep 15 10:18:04.664713 ip-10-0-141-235 systemd[1]: Starting Kubernetes Kubelet...
Sep 15 10:18:05.062954 ip-10-0-141-235 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 15 10:18:05.062954 ip-10-0-141-235 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 15 10:18:05.062954 ip-10-0-141-235 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 15 10:18:05.062954 ip-10-0-141-235 kubenswrapper[2583]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 15 10:18:05.062954 ip-10-0-141-235 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 15 10:18:05.064149 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.063940    2583 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 15 10:18:05.068231 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068217    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 10:18:05.068231 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068231    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 10:18:05.068298 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068235    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 10:18:05.068298 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068238    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 10:18:05.068298 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068241    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 10:18:05.068298 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068244    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 10:18:05.068298 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068250    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 10:18:05.068298 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068255    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 10:18:05.068298 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068258    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 10:18:05.068298 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068260    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 10:18:05.068298 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068263    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 10:18:05.068298 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068266    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 10:18:05.068298 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068268    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 10:18:05.068298 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068271    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 10:18:05.068298 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068274    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 10:18:05.068298 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068276    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 10:18:05.068298 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068279    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 10:18:05.068298 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068281    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 10:18:05.068298 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068284    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 10:18:05.068298 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068286    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 10:18:05.068298 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068289    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 10:18:05.068298 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068291    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 10:18:05.068876 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068294    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 10:18:05.068876 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068297    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 10:18:05.068876 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068300    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 10:18:05.068876 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068302    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 10:18:05.068876 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068305    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 10:18:05.068876 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068308    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 10:18:05.068876 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068311    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 10:18:05.068876 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068314    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 10:18:05.068876 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068317    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 10:18:05.068876 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068319    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 10:18:05.068876 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068322    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 10:18:05.068876 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068324    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 10:18:05.068876 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068326    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 10:18:05.068876 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068329    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 10:18:05.068876 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068332    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 10:18:05.068876 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068335    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 10:18:05.068876 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068338    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 10:18:05.068876 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068340    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 10:18:05.068876 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068343    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 10:18:05.068876 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068345    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 10:18:05.069385 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068348    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 10:18:05.069385 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068350    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 10:18:05.069385 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068353    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 10:18:05.069385 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068355    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 10:18:05.069385 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068357    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 10:18:05.069385 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068360    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 10:18:05.069385 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068362    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 10:18:05.069385 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068365    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 10:18:05.069385 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068369    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 10:18:05.069385 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068373    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 10:18:05.069385 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068376    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 10:18:05.069385 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068379    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 10:18:05.069385 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068382    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 10:18:05.069385 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068385    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 10:18:05.069385 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068388    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 10:18:05.069385 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068391    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 10:18:05.069385 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068394    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 10:18:05.069385 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068397    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 10:18:05.069385 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068400    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 10:18:05.069385 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068403    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 10:18:05.069892 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068405    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 10:18:05.069892 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068408    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 10:18:05.069892 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068412    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 10:18:05.069892 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068414    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 10:18:05.069892 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068417    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 10:18:05.069892 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068419    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 10:18:05.069892 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068422    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 10:18:05.069892 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068424    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 10:18:05.069892 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068427    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 10:18:05.069892 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068430    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 10:18:05.069892 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068433    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 10:18:05.069892 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068436    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 10:18:05.069892 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068452    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 10:18:05.069892 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068455    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 10:18:05.069892 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068459    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 10:18:05.069892 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068461    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 10:18:05.069892 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068464    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 10:18:05.069892 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068466    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 10:18:05.069892 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068469    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 10:18:05.070363 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068472    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 10:18:05.070363 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068474    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 10:18:05.070363 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068478    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 10:18:05.070363 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068481    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 10:18:05.070363 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068484    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 10:18:05.070363 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068488    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 10:18:05.070363 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068491    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 10:18:05.070363 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068496    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 10:18:05.070363 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068500    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 10:18:05.070363 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068503    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 10:18:05.070363 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068506    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 10:18:05.070363 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068509    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 10:18:05.070363 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068512    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 10:18:05.070363 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068514    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 10:18:05.070363 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068517    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 10:18:05.070363 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068519    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 10:18:05.070363 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068522    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 10:18:05.070363 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.068524    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 10:18:05.070363 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069206    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 10:18:05.070839 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069212    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 10:18:05.070839 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069215    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 10:18:05.070839 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069218    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 10:18:05.070839 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069221    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 10:18:05.070839 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069224    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 10:18:05.070839 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069227    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 10:18:05.070839 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069230    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 10:18:05.070839 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069232    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 10:18:05.070839 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069235    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 10:18:05.070839 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069238    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 10:18:05.070839 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069240    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 10:18:05.070839 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069243    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 10:18:05.070839 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069245    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 10:18:05.070839 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069248    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 10:18:05.070839 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069251    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 10:18:05.070839 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069253    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 10:18:05.070839 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069256    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 10:18:05.070839 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069259    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 10:18:05.070839 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069264    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 10:18:05.070839 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069266    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 10:18:05.071370 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069269    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 10:18:05.071370 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069272    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 10:18:05.071370 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069275    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 10:18:05.071370 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069278    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 10:18:05.071370 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069280    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 10:18:05.071370 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069283    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 10:18:05.071370 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069286    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 10:18:05.071370 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069288    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 10:18:05.071370 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069291    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 10:18:05.071370 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069303    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 10:18:05.071370 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069306    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 10:18:05.071370 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069309    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 10:18:05.071370 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069311    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 10:18:05.071370 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069314    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 10:18:05.071370 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069316    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 10:18:05.071370 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069319    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 10:18:05.071370 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069321    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 10:18:05.071370 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069324    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 10:18:05.071370 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069327    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 10:18:05.071370 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069330    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 10:18:05.071878 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069333    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 10:18:05.071878 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069335    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 10:18:05.071878 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069338    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 10:18:05.071878 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069340    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 10:18:05.071878 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069343    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 10:18:05.071878 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069345    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 10:18:05.071878 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069348    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 10:18:05.071878 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069351    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 10:18:05.071878 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069354    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 10:18:05.071878 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069358    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 10:18:05.071878 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069363    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 10:18:05.071878 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069367    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 10:18:05.071878 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069370    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 10:18:05.071878 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069373    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 10:18:05.071878 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069376    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 10:18:05.071878 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069379    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 10:18:05.071878 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069382    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 10:18:05.071878 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069385    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 10:18:05.071878 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069387    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 10:18:05.072355 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069390    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 10:18:05.072355 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069394    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 10:18:05.072355 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069396    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 10:18:05.072355 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069399    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 10:18:05.072355 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069402    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 10:18:05.072355 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069405    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 10:18:05.072355 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069407    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 10:18:05.072355 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069410    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 10:18:05.072355 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069412    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 10:18:05.072355 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069415    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 10:18:05.072355 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069418    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 10:18:05.072355 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069421    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 10:18:05.072355 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069424    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 10:18:05.072355 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069426    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 10:18:05.072355 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069429    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 10:18:05.072355 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069431    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 10:18:05.072355 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069434    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 10:18:05.072355 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069436    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 10:18:05.072355 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069452    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 10:18:05.072355 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069455    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 10:18:05.072867 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069458    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 10:18:05.072867 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069461    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 10:18:05.072867 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069463    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 10:18:05.072867 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069466    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 10:18:05.072867 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069468    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 10:18:05.072867 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069471    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 10:18:05.072867 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069473    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 10:18:05.072867 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069476    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 10:18:05.072867 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069479    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 10:18:05.072867 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069483    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 10:18:05.072867 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069486    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 10:18:05.072867 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069488    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 10:18:05.072867 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069491    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 10:18:05.072867 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069496    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 10:18:05.072867 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069499    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 10:18:05.072867 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069502    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 10:18:05.072867 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069505    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 10:18:05.072867 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069508    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 10:18:05.072867 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069511    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 10:18:05.073348 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069854    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 10:18:05.073348 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069860    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 10:18:05.073348 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069863    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 10:18:05.073348 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069866    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 10:18:05.073348 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069869    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 10:18:05.073348 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069872    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 10:18:05.073348 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069875    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 10:18:05.073348 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069877    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 10:18:05.073348 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069880    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 10:18:05.073348 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069882    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 10:18:05.073348 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069885    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 10:18:05.073348 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069888    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 10:18:05.073348 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069891    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 10:18:05.073348 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069893    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 10:18:05.073348 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069896    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 10:18:05.073348 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069899    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 10:18:05.073348 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069902    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 10:18:05.073348 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069904    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 10:18:05.073348 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069907    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 10:18:05.073348 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069909    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 10:18:05.073860 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069912    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 10:18:05.073860 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069915    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 10:18:05.073860 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069917    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 10:18:05.073860 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069920    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 10:18:05.073860 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069922    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 10:18:05.073860 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069925    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 10:18:05.073860 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069927    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 10:18:05.073860 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069930    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 10:18:05.073860 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069932    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 10:18:05.073860 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069935    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 10:18:05.073860 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069938    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 10:18:05.073860 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069941    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 10:18:05.073860 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069943    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 10:18:05.073860 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069946    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 10:18:05.073860 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069948    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 10:18:05.073860 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069951    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 10:18:05.073860 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069953    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 10:18:05.073860 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069956    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 10:18:05.073860 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069959    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 10:18:05.073860 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069961    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 10:18:05.074357 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069964    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 10:18:05.074357 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069966    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 10:18:05.074357 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069969    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 10:18:05.074357 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069971    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 10:18:05.074357 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069974    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 10:18:05.074357 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069977    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 10:18:05.074357 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069979    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 10:18:05.074357 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069982    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 10:18:05.074357 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069984    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 10:18:05.074357 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069987    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 10:18:05.074357 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069989    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 10:18:05.074357 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069991    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 10:18:05.074357 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069994    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 10:18:05.074357 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069996    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 10:18:05.074357 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.069999    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 10:18:05.074357 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070002    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 10:18:05.074357 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070004    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 10:18:05.074357 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070007    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 10:18:05.074357 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070009    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 10:18:05.074357 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070012    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 10:18:05.074873 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070014    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 10:18:05.074873 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070017    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 10:18:05.074873 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070019    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 10:18:05.074873 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070022    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 10:18:05.074873 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070024    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 10:18:05.074873 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070027    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 10:18:05.074873 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070029    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 10:18:05.074873 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070032    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 10:18:05.074873 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070034    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 10:18:05.074873 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070037    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 10:18:05.074873 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070039    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 10:18:05.074873 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070043    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 10:18:05.074873 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070047    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 10:18:05.074873 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070051    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 10:18:05.074873 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070054    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 10:18:05.074873 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070057    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 10:18:05.074873 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070060    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 10:18:05.074873 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070063    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 10:18:05.074873 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070066    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 10:18:05.075333 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070069    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 10:18:05.075333 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070071    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 10:18:05.075333 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070074    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 10:18:05.075333 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070076    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 10:18:05.075333 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070079    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 10:18:05.075333 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070081    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 10:18:05.075333 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070084    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 10:18:05.075333 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070086    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 10:18:05.075333 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070089    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 10:18:05.075333 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070091    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 10:18:05.075333 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070094    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 10:18:05.075333 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070097    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 10:18:05.075333 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070099    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 10:18:05.075333 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070102    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 10:18:05.075333 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070107    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 10:18:05.075333 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070109    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 10:18:05.075333 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070112    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 10:18:05.075333 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070114    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 10:18:05.075333 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070117    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 10:18:05.075333 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070119    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 10:18:05.075858 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070189    2583 flags.go:64] FLAG: --address="0.0.0.0"
Sep 15 10:18:05.075858 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070218    2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 15 10:18:05.075858 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070229    2583 flags.go:64] FLAG: --anonymous-auth="true"
Sep 15 10:18:05.075858 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070233    2583 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 15 10:18:05.075858 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070237    2583 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 15 10:18:05.075858 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070241    2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 15 10:18:05.075858 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070245    2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 15 10:18:05.075858 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070249    2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 15 10:18:05.075858 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070253    2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 15 10:18:05.075858 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070256    2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 15 10:18:05.075858 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070259    2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 15 10:18:05.075858 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070262    2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 15 10:18:05.075858 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070266    2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 15 10:18:05.075858 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070270    2583 flags.go:64] FLAG: --cgroup-root=""
Sep 15 10:18:05.075858 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070273    2583 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 15 10:18:05.075858 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070276    2583 flags.go:64] FLAG: --client-ca-file=""
Sep 15 10:18:05.075858 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070278    2583 flags.go:64] FLAG: --cloud-provider="external"
Sep 15 10:18:05.075858 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070281    2583 flags.go:64] FLAG: --cluster-dns="[]"
Sep 15 10:18:05.075858 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070284    2583 flags.go:64] FLAG: --cluster-domain=""
Sep 15 10:18:05.075858 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070287    2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 15 10:18:05.075858 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070291    2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 15 10:18:05.075858 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070294    2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 15 10:18:05.075858 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070297    2583 flags.go:64] FLAG: --container-log-max-files="5"
Sep 15 10:18:05.075858 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070301    2583 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 15 10:18:05.076460 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070304    2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 15 10:18:05.076460 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070307    2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 15 10:18:05.076460 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070313    2583 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 15 10:18:05.076460 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070316    2583 flags.go:64] FLAG: --contention-profiling="false"
Sep 15 10:18:05.076460 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070319    2583 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 15 10:18:05.076460 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070322    2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 15 10:18:05.076460 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070325    2583 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 15 10:18:05.076460 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070329    2583 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 15 10:18:05.076460 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070332    2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 15 10:18:05.076460 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070336    2583 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 15 10:18:05.076460 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070339    2583 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 15 10:18:05.076460 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070341    2583 flags.go:64] FLAG: --enable-load-reader="false"
Sep 15 10:18:05.076460 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070344    2583 flags.go:64] FLAG: --enable-server="true"
Sep 15 10:18:05.076460 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070348    2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 15 10:18:05.076460 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070352    2583 flags.go:64] FLAG: --event-burst="100"
Sep 15 10:18:05.076460 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070355    2583 flags.go:64] FLAG: --event-qps="50"
Sep 15 10:18:05.076460 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070358    2583 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 15 10:18:05.076460 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070361    2583 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 15 10:18:05.076460 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070364    2583 flags.go:64] FLAG: --eviction-hard=""
Sep 15 10:18:05.076460 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070368    2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 15 10:18:05.076460 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070371    2583 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 15 10:18:05.076460 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070375    2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 15 10:18:05.076460 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070378    2583 flags.go:64] FLAG: --eviction-soft=""
Sep 15 10:18:05.076460 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070381    2583 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 15 10:18:05.076460 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070383    2583 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 15 10:18:05.077088 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070386    2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 15 10:18:05.077088 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070389    2583 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 15 10:18:05.077088 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070392    2583 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 15 10:18:05.077088 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070395    2583 flags.go:64] FLAG: --fail-swap-on="true"
Sep 15 10:18:05.077088 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070398    2583 flags.go:64] FLAG: --feature-gates=""
Sep 15 10:18:05.077088 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070401    2583 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 15 10:18:05.077088 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070405    2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 15 10:18:05.077088 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070407    2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 15 10:18:05.077088 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070411    2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 15 10:18:05.077088 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070415    2583 flags.go:64] FLAG: --healthz-port="10248"
Sep 15 10:18:05.077088 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070418    2583 flags.go:64] FLAG: --help="false"
Sep 15 10:18:05.077088 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070421    2583 flags.go:64] FLAG: --hostname-override="ip-10-0-141-235.ec2.internal"
Sep 15 10:18:05.077088 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070424    2583 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 15 10:18:05.077088 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070427    2583 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 15 10:18:05.077088 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070430    2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 15 10:18:05.077088 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070433    2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 15 10:18:05.077088 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070436    2583 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 15 10:18:05.077088 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070451    2583 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 15 10:18:05.077088 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070455    2583 flags.go:64] FLAG: --image-service-endpoint=""
Sep 15 10:18:05.077088 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070458    2583 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 15 10:18:05.077088 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070461    2583 flags.go:64] FLAG: --kube-api-burst="100"
Sep 15 10:18:05.077088 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070464    2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 15 10:18:05.077088 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070468    2583 flags.go:64] FLAG: --kube-api-qps="50"
Sep 15 10:18:05.077088 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070473    2583 flags.go:64] FLAG: --kube-reserved=""
Sep 15 10:18:05.077715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070476    2583 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 15 10:18:05.077715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070479    2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 15 10:18:05.077715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070483    2583 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 15 10:18:05.077715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070486    2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 15 10:18:05.077715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070488    2583 flags.go:64] FLAG: --lock-file=""
Sep 15 10:18:05.077715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070491    2583 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 15 10:18:05.077715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070494    2583 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 15 10:18:05.077715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070497    2583 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 15 10:18:05.077715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070502    2583 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 15 10:18:05.077715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070505    2583 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 15 10:18:05.077715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070509    2583 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 15 10:18:05.077715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070511    2583 flags.go:64] FLAG: --logging-format="text"
Sep 15 10:18:05.077715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070514    2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 15 10:18:05.077715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070518    2583 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 15 10:18:05.077715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070521    2583 flags.go:64] FLAG: --manifest-url=""
Sep 15 10:18:05.077715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070524    2583 flags.go:64] FLAG: --manifest-url-header=""
Sep 15 10:18:05.077715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070528    2583 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 15 10:18:05.077715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070532    2583 flags.go:64] FLAG: --max-open-files="1000000"
Sep 15 10:18:05.077715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070536    2583 flags.go:64] FLAG: --max-pods="110"
Sep 15 10:18:05.077715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070539    2583 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 15 10:18:05.077715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070542    2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 15 10:18:05.077715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070545    2583 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 15 10:18:05.077715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070548    2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 15 10:18:05.077715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070551    2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 15 10:18:05.077715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070554    2583 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 15 10:18:05.078321 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070557    2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 15 10:18:05.078321 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070564    2583 flags.go:64] FLAG: --node-status-max-images="50"
Sep 15 10:18:05.078321 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070567    2583 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 15 10:18:05.078321 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070570    2583 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 15 10:18:05.078321 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070573    2583 flags.go:64] FLAG: --pod-cidr=""
Sep 15 10:18:05.078321 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070577    2583 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 15 10:18:05.078321 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070582    2583 flags.go:64] FLAG: --pod-manifest-path=""
Sep 15 10:18:05.078321 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070586    2583 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 15 10:18:05.078321 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070589    2583 flags.go:64] FLAG: --pods-per-core="0"
Sep 15 10:18:05.078321 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070592    2583 flags.go:64] FLAG: --port="10250"
Sep 15 10:18:05.078321 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070595    2583 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 15 10:18:05.078321 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070598    2583 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01f48519bd0177704"
Sep 15 10:18:05.078321 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070601    2583 flags.go:64] FLAG: --qos-reserved=""
Sep 15 10:18:05.078321 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070604    2583 flags.go:64] FLAG: --read-only-port="10255"
Sep 15 10:18:05.078321 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070607    2583 flags.go:64] FLAG: --register-node="true"
Sep 15 10:18:05.078321 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070610    2583 flags.go:64] FLAG: --register-with-taints=""
Sep 15 10:18:05.078321 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070614    2583 flags.go:64] FLAG: --registry-burst="10"
Sep 15 10:18:05.078321 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070616    2583 flags.go:64] FLAG: --registry-qps="5"
Sep 15 10:18:05.078321 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070619    2583 flags.go:64] FLAG: --reserved-cpus=""
Sep 15 10:18:05.078321 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070622    2583 flags.go:64] FLAG: --reserved-memory=""
Sep 15 10:18:05.078321 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070626    2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 15 10:18:05.078321 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070629    2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 15 10:18:05.078321 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070631    2583 flags.go:64] FLAG: --rotate-certificates="false"
Sep 15 10:18:05.078321 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070634    2583 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 15 10:18:05.078321 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070639    2583 flags.go:64] FLAG: --runonce="false"
Sep 15 10:18:05.078963 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070641    2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 15 10:18:05.078963 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070645    2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 15 10:18:05.078963 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070648    2583 flags.go:64] FLAG: --seccomp-default="false"
Sep 15 10:18:05.078963 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070651    2583 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 15 10:18:05.078963 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070654    2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 15 10:18:05.078963 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070657    2583 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 15 10:18:05.078963 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070660    2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 15 10:18:05.078963 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070663    2583 flags.go:64] FLAG: --storage-driver-password="root"
Sep 15 10:18:05.078963 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070666    2583 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 15 10:18:05.078963 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070669    2583 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 15 10:18:05.078963 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070672    2583 flags.go:64] FLAG: --storage-driver-user="root"
Sep 15 10:18:05.078963 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070674    2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 15 10:18:05.078963 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070677    2583 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 15 10:18:05.078963 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070681    2583 flags.go:64] FLAG: --system-cgroups=""
Sep 15 10:18:05.078963 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070685    2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 15 10:18:05.078963 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070691    2583 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 15 10:18:05.078963 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070694    2583 flags.go:64] FLAG: --tls-cert-file=""
Sep 15 10:18:05.078963 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070697    2583 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 15 10:18:05.078963 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070700    2583 flags.go:64] FLAG: --tls-min-version=""
Sep 15 10:18:05.078963 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070703    2583 flags.go:64] FLAG: --tls-private-key-file=""
Sep 15 10:18:05.078963 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070705    2583 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 15 10:18:05.078963 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070708    2583 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 15 10:18:05.078963 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070711    2583 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 15 10:18:05.078963 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070714    2583 flags.go:64] FLAG: --v="2"
Sep 15 10:18:05.078963 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070718    2583 flags.go:64] FLAG: --version="false"
Sep 15 10:18:05.079896 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070721    2583 flags.go:64] FLAG: --vmodule=""
Sep 15 10:18:05.079896 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070725    2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 15 10:18:05.079896 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.070728    2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 15 10:18:05.079896 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070837    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 10:18:05.079896 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070841    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 10:18:05.079896 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070844    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 10:18:05.079896 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070849    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 10:18:05.079896 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070851    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 10:18:05.079896 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070854    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 10:18:05.079896 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070856    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 10:18:05.079896 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070859    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 10:18:05.079896 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070862    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 10:18:05.079896 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070864    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 10:18:05.079896 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070867    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 10:18:05.079896 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070869    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 10:18:05.079896 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070872    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 10:18:05.079896 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070875    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 10:18:05.079896 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070877    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 10:18:05.079896 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070880    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 10:18:05.079896 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070882    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 10:18:05.080623 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070887    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 10:18:05.080623 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070892    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 10:18:05.080623 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070895    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 10:18:05.080623 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070898    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 10:18:05.080623 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070900    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 10:18:05.080623 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070903    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 10:18:05.080623 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070906    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 10:18:05.080623 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070908    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 10:18:05.080623 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070911    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 10:18:05.080623 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070914    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 10:18:05.080623 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070916    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 10:18:05.080623 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070919    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 10:18:05.080623 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070921    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 10:18:05.080623 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070924    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 10:18:05.080623 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070926    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 10:18:05.080623 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070929    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 10:18:05.080623 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070932    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 10:18:05.080623 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070934    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 10:18:05.080623 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070939    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 10:18:05.081115 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070941    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 10:18:05.081115 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070944    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 10:18:05.081115 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070946    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 10:18:05.081115 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070949    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 10:18:05.081115 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070952    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 10:18:05.081115 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070955    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 10:18:05.081115 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070957    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 10:18:05.081115 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070959    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 10:18:05.081115 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070962    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 10:18:05.081115 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070965    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 10:18:05.081115 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070967    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 10:18:05.081115 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070971    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 10:18:05.081115 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070974    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 10:18:05.081115 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070977    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 10:18:05.081115 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070980    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 10:18:05.081115 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070983    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 10:18:05.081115 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070986    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 10:18:05.081115 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070989    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 10:18:05.081115 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070992    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 10:18:05.081612 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070994    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 10:18:05.081612 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070997    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 10:18:05.081612 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.070999    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 10:18:05.081612 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071002    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 10:18:05.081612 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071005    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 10:18:05.081612 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071007    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 10:18:05.081612 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071010    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 10:18:05.081612 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071012    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 10:18:05.081612 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071015    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 10:18:05.081612 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071017    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 10:18:05.081612 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071020    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 10:18:05.081612 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071022    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 10:18:05.081612 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071027    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 10:18:05.081612 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071029    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 10:18:05.081612 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071032    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 10:18:05.081612 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071034    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 10:18:05.081612 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071037    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 10:18:05.081612 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071039    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 10:18:05.081612 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071042    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 10:18:05.082080 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071046    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 10:18:05.082080 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071049    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 10:18:05.082080 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071052    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 10:18:05.082080 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071055    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 10:18:05.082080 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071057    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 10:18:05.082080 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071060    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 10:18:05.082080 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071062    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 10:18:05.082080 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071065    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 10:18:05.082080 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071069    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 10:18:05.082080 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071072    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 10:18:05.082080 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071075    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 10:18:05.082080 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071077    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 10:18:05.082080 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071080    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 10:18:05.082080 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071083    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 10:18:05.082080 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071085    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 10:18:05.082080 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071088    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 10:18:05.082080 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071090    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 10:18:05.082080 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071093    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 10:18:05.082080 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071096    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 10:18:05.082080 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071098    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 10:18:05.082595 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071101    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 10:18:05.082595 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071103    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 10:18:05.082595 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071106    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 10:18:05.082595 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071109    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 10:18:05.082595 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.071111    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 10:18:05.082595 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.071907    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 15 10:18:05.082595 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.081036    2583 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 15 10:18:05.082595 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.081051    2583 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 15 10:18:05.082799 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.082788    2583 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 15 10:18:05.082799 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.082795    2583 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 15 10:18:05.082855 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082852    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 10:18:05.082889 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082857    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 10:18:05.082889 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082861    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 10:18:05.082889 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082864    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 10:18:05.082889 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082866    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 10:18:05.082889 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082869    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 10:18:05.082889 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082872    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 10:18:05.082889 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082874    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 10:18:05.082889 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082877    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 10:18:05.082889 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082880    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 10:18:05.082889 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082882    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 10:18:05.082889 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082885    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 10:18:05.082889 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082887    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 10:18:05.082889 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082890    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 10:18:05.082889 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082893    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 10:18:05.083237 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082897    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 10:18:05.083237 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082900    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 10:18:05.083237 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082903    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 10:18:05.083237 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082905    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 10:18:05.083237 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082908    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 10:18:05.083237 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082910    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 10:18:05.083237 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082913    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 10:18:05.083237 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082921    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 10:18:05.083237 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082924    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 10:18:05.083237 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082926    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 10:18:05.083237 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082929    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 10:18:05.083237 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082932    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 10:18:05.083237 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082934    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 10:18:05.083237 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082937    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 10:18:05.083237 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082940    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 10:18:05.083237 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082942    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 10:18:05.083237 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082945    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 10:18:05.083237 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082948    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 10:18:05.083237 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082950    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 10:18:05.083820 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082953    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 10:18:05.083820 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082956    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 10:18:05.083820 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082958    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 10:18:05.083820 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082961    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 10:18:05.083820 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082963    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 10:18:05.083820 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082966    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 10:18:05.083820 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082968    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 10:18:05.083820 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082971    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 10:18:05.083820 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082973    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 10:18:05.083820 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082976    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 10:18:05.083820 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082979    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 10:18:05.083820 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082982    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 10:18:05.083820 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082984    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 10:18:05.083820 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082988    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 10:18:05.083820 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082991    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 10:18:05.083820 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082994    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 10:18:05.083820 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082997    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 10:18:05.083820 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.082999    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 10:18:05.083820 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083002    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 10:18:05.084279 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083006    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 10:18:05.084279 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083010    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 10:18:05.084279 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083013    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 10:18:05.084279 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083016    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 10:18:05.084279 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083019    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 10:18:05.084279 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083022    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 10:18:05.084279 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083026    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 10:18:05.084279 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083029    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 10:18:05.084279 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083032    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 10:18:05.084279 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083035    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 10:18:05.084279 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083038    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 10:18:05.084279 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083040    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 10:18:05.084279 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083043    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 10:18:05.084279 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083045    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 10:18:05.084279 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083048    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 10:18:05.084279 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083051    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 10:18:05.084279 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083054    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 10:18:05.084279 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083056    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 10:18:05.084279 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083059    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 10:18:05.084767 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083061    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 10:18:05.084767 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083064    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 10:18:05.084767 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083066    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 10:18:05.084767 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083069    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 10:18:05.084767 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083072    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 10:18:05.084767 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083075    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 10:18:05.084767 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083077    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 10:18:05.084767 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083082    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 10:18:05.084767 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083086    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 10:18:05.084767 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083089    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 10:18:05.084767 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083091    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 10:18:05.084767 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083094    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 10:18:05.084767 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083097    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 10:18:05.084767 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083100    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 10:18:05.084767 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083102    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 10:18:05.084767 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083105    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 10:18:05.084767 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083108    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 10:18:05.084767 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083110    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 10:18:05.084767 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083113    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 10:18:05.084767 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083116    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 10:18:05.085250 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083118    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 10:18:05.085250 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083121    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 10:18:05.085250 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083124    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 10:18:05.085250 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083126    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 10:18:05.085250 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083129    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 10:18:05.085250 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083132    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 10:18:05.085250 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083134    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 10:18:05.085250 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.083139    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 15 10:18:05.085250 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083258    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 15 10:18:05.085250 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083264    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 15 10:18:05.085250 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083267    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 15 10:18:05.085250 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083271    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 15 10:18:05.085250 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083274    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 15 10:18:05.085250 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083277    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 15 10:18:05.085250 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083280    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 15 10:18:05.085635 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083283    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 15 10:18:05.085635 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083285    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 15 10:18:05.085635 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083288    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 15 10:18:05.085635 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083291    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 15 10:18:05.085635 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083294    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 15 10:18:05.085635 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083297    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 15 10:18:05.085635 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083300    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 15 10:18:05.085635 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083303    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 15 10:18:05.085635 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083306    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 15 10:18:05.085635 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083309    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 15 10:18:05.085635 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083311    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 15 10:18:05.085635 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083314    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 15 10:18:05.085635 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083317    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 15 10:18:05.085635 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083319    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 15 10:18:05.085635 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083322    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 15 10:18:05.085635 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083325    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 15 10:18:05.085635 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083327    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 15 10:18:05.085635 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083330    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 15 10:18:05.085635 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083333    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 15 10:18:05.085635 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083335    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 15 10:18:05.086148 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083338    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 15 10:18:05.086148 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083340    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 15 10:18:05.086148 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083343    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 15 10:18:05.086148 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083345    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 15 10:18:05.086148 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083348    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 15 10:18:05.086148 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083350    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 15 10:18:05.086148 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083353    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 15 10:18:05.086148 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083356    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 15 10:18:05.086148 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083360    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 15 10:18:05.086148 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083362    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 15 10:18:05.086148 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083365    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 15 10:18:05.086148 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083367    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 15 10:18:05.086148 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083370    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 15 10:18:05.086148 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083372    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 15 10:18:05.086148 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083376    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 15 10:18:05.086148 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083379    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 15 10:18:05.086148 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083381    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 15 10:18:05.086148 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083384    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 15 10:18:05.086148 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083387    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 15 10:18:05.086629 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083390    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 15 10:18:05.086629 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083393    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 15 10:18:05.086629 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083396    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 15 10:18:05.086629 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083400    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 15 10:18:05.086629 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083403    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 15 10:18:05.086629 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083406    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 15 10:18:05.086629 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083408    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 15 10:18:05.086629 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083411    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 15 10:18:05.086629 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083414    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 15 10:18:05.086629 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083417    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 15 10:18:05.086629 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083419    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 15 10:18:05.086629 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083421    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 15 10:18:05.086629 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083424    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 15 10:18:05.086629 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083426    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 15 10:18:05.086629 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083429    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 15 10:18:05.086629 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083431    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 15 10:18:05.086629 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083434    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 15 10:18:05.086629 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083437    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 15 10:18:05.086629 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083455    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 15 10:18:05.086629 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083458    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 15 10:18:05.087112 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083461    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 15 10:18:05.087112 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083464    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 15 10:18:05.087112 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083466    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 15 10:18:05.087112 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083469    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 15 10:18:05.087112 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083471    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 15 10:18:05.087112 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083474    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 15 10:18:05.087112 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083476    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 15 10:18:05.087112 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083479    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 15 10:18:05.087112 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083482    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 15 10:18:05.087112 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083484    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 15 10:18:05.087112 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083487    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 15 10:18:05.087112 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083490    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 15 10:18:05.087112 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083494    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 15 10:18:05.087112 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083497    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 15 10:18:05.087112 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083500    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 15 10:18:05.087112 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083503    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 15 10:18:05.087112 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083506    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 15 10:18:05.087112 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083509    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 15 10:18:05.087112 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083512    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 15 10:18:05.087626 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083515    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 15 10:18:05.087626 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083518    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 15 10:18:05.087626 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083521    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 15 10:18:05.087626 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083523    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 15 10:18:05.087626 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083526    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 15 10:18:05.087626 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083529    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 15 10:18:05.087626 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083532    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 15 10:18:05.087626 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083535    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 15 10:18:05.087626 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083538    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 15 10:18:05.087626 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083540    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 15 10:18:05.087626 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083543    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 15 10:18:05.087626 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083546    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 15 10:18:05.087626 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083548    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 15 10:18:05.087626 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:05.083551    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 15 10:18:05.087626 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.083556    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 15 10:18:05.088006 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.083662    2583 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 15 10:18:05.088006 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.087051    2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 15 10:18:05.088115 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.088104    2583 server.go:1019] "Starting client certificate rotation"
Sep 15 10:18:05.088229 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.088212    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 15 10:18:05.088268 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.088244    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 15 10:18:05.090952 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.090939    2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 15 10:18:05.094642 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.094626    2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 15 10:18:05.108453 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.108418    2583 log.go:25] "Validated CRI v1 runtime API"
Sep 15 10:18:05.111846 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.111831    2583 log.go:25] "Validated CRI v1 image API"
Sep 15 10:18:05.112680 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.112665    2583 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 15 10:18:05.116394 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.116374    2583 fs.go:135] Filesystem UUIDs: map[420e77d5-4141-4c43-92de-08d48ef1427a:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 bee92179-667f-4820-a313-38be1228792d:/dev/nvme0n1p3]
Sep 15 10:18:05.116489 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.116391    2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 15 10:18:05.119110 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.119094    2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 15 10:18:05.122425 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.122312    2583 manager.go:216] Machine: {Timestamp:2026-09-15 10:18:05.119929624 +0000 UTC m=+0.366896584 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3092304 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2549fd6af96c76669ae9bad3209a91 SystemUUID:ec2549fd-6af9-6c76-669a-e9bad3209a91 BootID:68bbed23-1185-4ea9-9402-8268897a440c Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:d5:33:30:19 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:d5:33:30:19 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fe:6c:20:c8:01:b1 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 15 10:18:05.122425 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.122419    2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 15 10:18:05.122550 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.122537    2583 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 15 10:18:05.126372 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.126350    2583 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 15 10:18:05.126526 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.126374    2583 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-235.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 15 10:18:05.127926 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.127916    2583 topology_manager.go:138] "Creating topology manager with none policy"
Sep 15 10:18:05.127964 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.127928    2583 container_manager_linux.go:309] "Creating device plugin manager"
Sep 15 10:18:05.127964 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.127940    2583 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 15 10:18:05.128776 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.128766    2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 15 10:18:05.130040 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.130031    2583 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 15 10:18:05.134668 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.134648    2583 state_mem.go:36] "Initialized new in-memory state store"
Sep 15 10:18:05.134795 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.134786    2583 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 15 10:18:05.134830 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.134821    2583 kubelet.go:486] "Attempting to sync node with API server"
Sep 15 10:18:05.134859 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.134837    2583 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 15 10:18:05.134897 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.134863    2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 15 10:18:05.134897 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.134876    2583 kubelet.go:398] "Adding apiserver pod source"
Sep 15 10:18:05.134897 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.134888    2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 15 10:18:05.135316 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.135301    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 15 10:18:05.135375 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.135319    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 15 10:18:05.139172 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.139149    2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hk8zz"
Sep 15 10:18:05.139955 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.139927    2583 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 15 10:18:05.142778 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.142763    2583 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 15 10:18:05.142828 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.142797    2583 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 15 10:18:05.144261 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:05.144239    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 15 10:18:05.144310 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:05.144290    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 15 10:18:05.145421 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.145409    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 15 10:18:05.145481 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.145426    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 15 10:18:05.145481 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.145432    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 15 10:18:05.145481 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.145437    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 15 10:18:05.145481 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.145458    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 15 10:18:05.145481 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.145463    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 15 10:18:05.145481 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.145476    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 15 10:18:05.145481 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.145481    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 15 10:18:05.145665 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.145488    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 15 10:18:05.145665 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.145493    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 15 10:18:05.145665 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.145501    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 15 10:18:05.145665 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.145506    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 15 10:18:05.146636 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.146618    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hk8zz"
Sep 15 10:18:05.147146 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.147137    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 15 10:18:05.147181 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.147148    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 15 10:18:05.150840 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.150812    2583 server.go:1268] "Started kubelet"
Sep 15 10:18:05.150927 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.150841    2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 15 10:18:05.150993 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.150926    2583 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 15 10:18:05.151059 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.151036    2583 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 15 10:18:05.151580 ip-10-0-141-235 systemd[1]: Started Kubernetes Kubelet.
Sep 15 10:18:05.152418 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.152390    2583 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 15 10:18:05.153156 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.153143    2583 server.go:310] "Adding debug handlers to kubelet server"
Sep 15 10:18:05.156045 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:05.156030    2583 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 15 10:18:05.160639 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.160622    2583 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 15 10:18:05.160729 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.160643    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 15 10:18:05.160729 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:05.160692    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-235.ec2.internal\" not found"
Sep 15 10:18:05.161418 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.161401    2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 15 10:18:05.161418 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.161407    2583 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 15 10:18:05.161557 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.161428    2583 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 15 10:18:05.161557 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.161544    2583 reconstruct.go:97] "Volume reconstruction finished"
Sep 15 10:18:05.161557 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.161553    2583 reconciler.go:29] "Reconciler: start to sync state"
Sep 15 10:18:05.161698 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.161565    2583 factory.go:55] Registering systemd factory
Sep 15 10:18:05.161698 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.161580    2583 factory.go:223] Registration of the systemd container factory successfully
Sep 15 10:18:05.161812 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.161798    2583 factory.go:153] Registering CRI-O factory
Sep 15 10:18:05.161861 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.161815    2583 factory.go:223] Registration of the crio container factory successfully
Sep 15 10:18:05.161912 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.161862    2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 15 10:18:05.161912 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.161881    2583 factory.go:103] Registering Raw factory
Sep 15 10:18:05.161912 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.161896    2583 manager.go:1196] Started watching for new ooms in manager
Sep 15 10:18:05.162297 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.162283    2583 manager.go:349] Starting recovery of all containers
Sep 15 10:18:05.163181 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.163163    2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 10:18:05.163352 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.163338    2583 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-141-235.ec2.internal" not found
Sep 15 10:18:05.165540 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:05.165489    2583 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-141-235.ec2.internal\" not found" node="ip-10-0-141-235.ec2.internal"
Sep 15 10:18:05.172192 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.172171    2583 manager.go:354] Recovery completed
Sep 15 10:18:05.174243 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.174218    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 15 10:18:05.178251 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.178154    2583 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-141-235.ec2.internal" not found
Sep 15 10:18:05.178683 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.178528    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 10:18:05.180390 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.180369    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-235.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 10:18:05.180503 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.180408    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-235.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 10:18:05.180503 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.180425    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-235.ec2.internal" event="NodeHasSufficientPID"
Sep 15 10:18:05.181074 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.181055    2583 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 15 10:18:05.181074 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.181061    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 10:18:05.181074 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.181066    2583 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 15 10:18:05.181199 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.181082    2583 state_mem.go:36] "Initialized new in-memory state store"
Sep 15 10:18:05.181731 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.181717    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-235.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 10:18:05.181783 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.181742    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-235.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 10:18:05.181783 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.181755    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-235.ec2.internal" event="NodeHasSufficientPID"
Sep 15 10:18:05.183243 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.183232    2583 policy_none.go:49] "None policy: Start"
Sep 15 10:18:05.183284 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.183247    2583 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 15 10:18:05.183284 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.183257    2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 15 10:18:05.185230 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.185219    2583 policy_none.go:47] "Start"
Sep 15 10:18:05.209992 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.209976    2583 manager.go:337] "Starting Device Plugin manager"
Sep 15 10:18:05.210080 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:05.210015    2583 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 15 10:18:05.210080 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.210028    2583 server.go:87] "Starting device plugin registration server"
Sep 15 10:18:05.210265 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.210251    2583 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 15 10:18:05.210326 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.210266    2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 15 10:18:05.210746 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.210422    2583 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 15 10:18:05.210746 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.210509    2583 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 15 10:18:05.210746 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.210518    2583 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 15 10:18:05.212699 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:05.212541    2583 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 15 10:18:05.212699 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:05.212589    2583 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-235.ec2.internal\" not found"
Sep 15 10:18:05.238202 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.238184    2583 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-141-235.ec2.internal" not found
Sep 15 10:18:05.293301 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.293273    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 15 10:18:05.293387 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.293325    2583 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 15 10:18:05.293387 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.293346    2583 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 15 10:18:05.293387 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:05.293382    2583 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 15 10:18:05.295621 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.295599    2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 10:18:05.310359 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.310336    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 10:18:05.311104 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.311089    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-235.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 10:18:05.311163 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.311119    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-235.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 10:18:05.311163 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.311133    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-235.ec2.internal" event="NodeHasSufficientPID"
Sep 15 10:18:05.311163 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.311154    2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-235.ec2.internal"
Sep 15 10:18:05.318935 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.318866    2583 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-235.ec2.internal"
Sep 15 10:18:05.318935 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:05.318888    2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-235.ec2.internal\": node \"ip-10-0-141-235.ec2.internal\" not found"
Sep 15 10:18:05.328341 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:05.328319    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-235.ec2.internal\" not found"
Sep 15 10:18:05.394302 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.394279    2583 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-235.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-235.ec2.internal"]
Sep 15 10:18:05.394387 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.394342    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 10:18:05.395151 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.395136    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-235.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 10:18:05.395204 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.395168    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-235.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 10:18:05.395204 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.395179    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-235.ec2.internal" event="NodeHasSufficientPID"
Sep 15 10:18:05.397274 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.397263    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 10:18:05.397375 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.397361    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-235.ec2.internal"
Sep 15 10:18:05.397421 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.397390    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 10:18:05.397968 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.397951    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-235.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 10:18:05.397968 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.397959    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-235.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 10:18:05.398086 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.397978    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-235.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 10:18:05.398086 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.397983    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-235.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 10:18:05.398086 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.397991    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-235.ec2.internal" event="NodeHasSufficientPID"
Sep 15 10:18:05.398086 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.397996    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-235.ec2.internal" event="NodeHasSufficientPID"
Sep 15 10:18:05.399977 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.399961    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-235.ec2.internal"
Sep 15 10:18:05.400031 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.399996    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 15 10:18:05.400640 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.400625    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-235.ec2.internal" event="NodeHasSufficientMemory"
Sep 15 10:18:05.400723 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.400655    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-235.ec2.internal" event="NodeHasNoDiskPressure"
Sep 15 10:18:05.400723 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.400672    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-235.ec2.internal" event="NodeHasSufficientPID"
Sep 15 10:18:05.428390 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:05.428368    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-235.ec2.internal\" not found"
Sep 15 10:18:05.432099 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:05.432087    2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-235.ec2.internal\" not found" node="ip-10-0-141-235.ec2.internal"
Sep 15 10:18:05.436580 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:05.436565    2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-235.ec2.internal\" not found" node="ip-10-0-141-235.ec2.internal"
Sep 15 10:18:05.464023 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.464002    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/93a9cd305dd93bbe9f79180cbc0e6eaa-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-235.ec2.internal\" (UID: \"93a9cd305dd93bbe9f79180cbc0e6eaa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-235.ec2.internal"
Sep 15 10:18:05.464104 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.464029    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/93a9cd305dd93bbe9f79180cbc0e6eaa-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-235.ec2.internal\" (UID: \"93a9cd305dd93bbe9f79180cbc0e6eaa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-235.ec2.internal"
Sep 15 10:18:05.464104 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.464055    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2b3d6833f74b07abf73dd136dadede80-config\") pod \"kube-apiserver-proxy-ip-10-0-141-235.ec2.internal\" (UID: \"2b3d6833f74b07abf73dd136dadede80\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-235.ec2.internal"
Sep 15 10:18:05.529169 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:05.529150    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-235.ec2.internal\" not found"
Sep 15 10:18:05.564563 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.564539    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/93a9cd305dd93bbe9f79180cbc0e6eaa-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-235.ec2.internal\" (UID: \"93a9cd305dd93bbe9f79180cbc0e6eaa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-235.ec2.internal"
Sep 15 10:18:05.564638 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.564573    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/93a9cd305dd93bbe9f79180cbc0e6eaa-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-235.ec2.internal\" (UID: \"93a9cd305dd93bbe9f79180cbc0e6eaa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-235.ec2.internal"
Sep 15 10:18:05.564638 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.564588    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2b3d6833f74b07abf73dd136dadede80-config\") pod \"kube-apiserver-proxy-ip-10-0-141-235.ec2.internal\" (UID: \"2b3d6833f74b07abf73dd136dadede80\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-235.ec2.internal"
Sep 15 10:18:05.564638 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.564611    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2b3d6833f74b07abf73dd136dadede80-config\") pod \"kube-apiserver-proxy-ip-10-0-141-235.ec2.internal\" (UID: \"2b3d6833f74b07abf73dd136dadede80\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-235.ec2.internal"
Sep 15 10:18:05.564638 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.564544    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/93a9cd305dd93bbe9f79180cbc0e6eaa-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-235.ec2.internal\" (UID: \"93a9cd305dd93bbe9f79180cbc0e6eaa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-235.ec2.internal"
Sep 15 10:18:05.564773 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.564642    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/93a9cd305dd93bbe9f79180cbc0e6eaa-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-235.ec2.internal\" (UID: \"93a9cd305dd93bbe9f79180cbc0e6eaa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-235.ec2.internal"
Sep 15 10:18:05.629785 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:05.629744    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-235.ec2.internal\" not found"
Sep 15 10:18:05.730417 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:05.730393    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-235.ec2.internal\" not found"
Sep 15 10:18:05.734550 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.734536    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-235.ec2.internal"
Sep 15 10:18:05.739218 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:05.738892    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-235.ec2.internal"
Sep 15 10:18:05.830974 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:05.830945    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-235.ec2.internal\" not found"
Sep 15 10:18:05.933859 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:05.933802    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-235.ec2.internal\" not found"
Sep 15 10:18:06.035354 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:06.035331    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-235.ec2.internal\" not found"
Sep 15 10:18:06.088138 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:06.088114    2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 15 10:18:06.088547 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:06.088258    2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 15 10:18:06.088547 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:06.088277    2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 15 10:18:06.136294 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:06.136269    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-235.ec2.internal\" not found"
Sep 15 10:18:06.152646 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:06.152615    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-14 10:13:06 +0000 UTC" deadline="2028-02-20 04:02:50.163174454 +0000 UTC"
Sep 15 10:18:06.152646 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:06.152645    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12545h44m44.010533404s"
Sep 15 10:18:06.237010 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:06.236951    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-235.ec2.internal\" not found"
Sep 15 10:18:06.242563 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:06.242546    2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 10:18:06.244736 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:06.244719    2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 10:18:06.261622 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:06.261603    2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-235.ec2.internal"
Sep 15 10:18:06.272718 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:06.272702    2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 15 10:18:06.273412 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:06.273397    2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-235.ec2.internal"
Sep 15 10:18:06.281799 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:06.281783    2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 15 10:18:06.445547 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:06.445372    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93a9cd305dd93bbe9f79180cbc0e6eaa.slice/crio-dc5070c4b121442edc5ee965ecf634d35c62ec229364f48444b45826641e10b5 WatchSource:0}: Error finding container dc5070c4b121442edc5ee965ecf634d35c62ec229364f48444b45826641e10b5: Status 404 returned error can't find the container with id dc5070c4b121442edc5ee965ecf634d35c62ec229364f48444b45826641e10b5
Sep 15 10:18:06.449717 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:06.449703    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 10:18:06.568676 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:06.568655    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b3d6833f74b07abf73dd136dadede80.slice/crio-6055a8d51da4917fe066e8a0a33bfa8f916b60bfcb234051a237c23900ab2217 WatchSource:0}: Error finding container 6055a8d51da4917fe066e8a0a33bfa8f916b60bfcb234051a237c23900ab2217: Status 404 returned error can't find the container with id 6055a8d51da4917fe066e8a0a33bfa8f916b60bfcb234051a237c23900ab2217
Sep 15 10:18:07.135413 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.135381    2583 apiserver.go:52] "Watching apiserver"
Sep 15 10:18:07.141654 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.141619    2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 10:18:07.142712 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.142687    2583 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 15 10:18:07.144493 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.144457    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-mw292","openshift-image-registry/node-ca-5gw48","openshift-multus/multus-n5sdr","openshift-network-operator/iptables-alerter-w2qnt","openshift-ovn-kubernetes/ovnkube-node-pkmxf","kube-system/konnectivity-agent-8hkxj","kube-system/kube-apiserver-proxy-ip-10-0-141-235.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4sn5","openshift-cluster-node-tuning-operator/tuned-vf2xj","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-235.ec2.internal","openshift-multus/multus-additional-cni-plugins-58dk4","openshift-multus/network-metrics-daemon-gsxvv","openshift-network-diagnostics/network-check-target-rxb8p","kube-system/global-pull-secret-syncer-qjcwx"]
Sep 15 10:18:07.147386 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.147361    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:18:07.147550 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:07.147486    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxb8p" podUID="88d49ac3-3a21-43ee-b74b-dfbd6239292e"
Sep 15 10:18:07.149853 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.149821    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5gw48"
Sep 15 10:18:07.152510 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.152187    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 15 10:18:07.152510 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.152238    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 15 10:18:07.152510 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.152268    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.152510 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.152187    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nvkmf\""
Sep 15 10:18:07.152510 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.152187    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 15 10:18:07.154789 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.154558    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-w2qnt"
Sep 15 10:18:07.154965 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.154939    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 15 10:18:07.155120 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.154997    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ngz5l\""
Sep 15 10:18:07.155267 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.155251    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 15 10:18:07.155267 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.155258    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 15 10:18:07.155396 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.155254    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 15 10:18:07.156819 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.156798    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-ncv85\""
Sep 15 10:18:07.157084 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.157053    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 15 10:18:07.157192 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.157090    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 15 10:18:07.157192 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.157053    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 15 10:18:07.157192 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.157187    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.159372 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.159191    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 15 10:18:07.159372 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.159323    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 15 10:18:07.159695 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.159676    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8hkxj"
Sep 15 10:18:07.160710 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.160335    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 15 10:18:07.160710 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.160344    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pccfn\""
Sep 15 10:18:07.160710 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.160361    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 15 10:18:07.160710 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.160375    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 15 10:18:07.160710 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.160540    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 15 10:18:07.161410 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.161390    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 15 10:18:07.161811 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.161778    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 15 10:18:07.162102 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.162032    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mw292"
Sep 15 10:18:07.162102 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.162073    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 15 10:18:07.162102 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.162090    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-fb6pf\""
Sep 15 10:18:07.164082 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.164060    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-k8jwj\""
Sep 15 10:18:07.164182 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.164103    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 15 10:18:07.164346 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.164330    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 15 10:18:07.165128 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.164841    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4sn5"
Sep 15 10:18:07.166924 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.166901    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 15 10:18:07.166924 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.166901    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5tv4m\""
Sep 15 10:18:07.167074 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.166948    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 15 10:18:07.167074 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.166972    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 15 10:18:07.167401 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.167382    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.169961 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.169775    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-c7bqh\""
Sep 15 10:18:07.169961 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.169799    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 15 10:18:07.170121 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.170062    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 15 10:18:07.170121 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.170081    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:07.170233 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:07.170155    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjcwx" podUID="d9306059-36e0-4eae-8dc8-3ce075dc9b21"
Sep 15 10:18:07.172523 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.172502    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-58dk4"
Sep 15 10:18:07.173876 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.173852    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-282z4\" (UniqueName: \"kubernetes.io/projected/88f8318b-0faa-4708-afc6-de7f7ae3cee3-kube-api-access-282z4\") pod \"node-resolver-mw292\" (UID: \"88f8318b-0faa-4708-afc6-de7f7ae3cee3\") " pod="openshift-dns/node-resolver-mw292"
Sep 15 10:18:07.173976 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.173888    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-multus-socket-dir-parent\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.173976 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.173915    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-host-run-netns\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.173976 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.173961    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-hostroot\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.174147 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.173996    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c512fef3-9326-49ca-a58e-4fa39b7d9814-host-slash\") pod \"iptables-alerter-w2qnt\" (UID: \"c512fef3-9326-49ca-a58e-4fa39b7d9814\") " pod="openshift-network-operator/iptables-alerter-w2qnt"
Sep 15 10:18:07.174147 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.174026    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-host-kubelet\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.174147 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.174053    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-systemd-units\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.174147 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.174079    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-host-run-netns\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.174147 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.174104    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-run-systemd\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.174147 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.174129    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-system-cni-dir\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.174385 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.174190    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t6hb\" (UniqueName: \"kubernetes.io/projected/c512fef3-9326-49ca-a58e-4fa39b7d9814-kube-api-access-6t6hb\") pod \"iptables-alerter-w2qnt\" (UID: \"c512fef3-9326-49ca-a58e-4fa39b7d9814\") " pod="openshift-network-operator/iptables-alerter-w2qnt"
Sep 15 10:18:07.174385 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.174225    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-host-run-k8s-cni-cncf-io\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.174385 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.174253    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqtl9\" (UniqueName: \"kubernetes.io/projected/88d49ac3-3a21-43ee-b74b-dfbd6239292e-kube-api-access-dqtl9\") pod \"network-check-target-rxb8p\" (UID: \"88d49ac3-3a21-43ee-b74b-dfbd6239292e\") " pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:18:07.174385 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.174293    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh7jp\" (UniqueName: \"kubernetes.io/projected/b05ac5f2-3cd6-48f4-857a-08403194adbe-kube-api-access-qh7jp\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.174385 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.174323    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-multus-cni-dir\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.174385 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.174348    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-host-var-lib-cni-bin\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.174385 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.174373    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-host-run-multus-certs\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.174739 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.174414    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-etc-kubernetes\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.174739 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.174472    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/17509e5d-d9dc-4f53-a111-4929185b0a3c-konnectivity-ca\") pod \"konnectivity-agent-8hkxj\" (UID: \"17509e5d-d9dc-4f53-a111-4929185b0a3c\") " pod="kube-system/konnectivity-agent-8hkxj"
Sep 15 10:18:07.174739 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.174507    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-run-openvswitch\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.174739 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.174545    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.174739 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.174571    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b05ac5f2-3cd6-48f4-857a-08403194adbe-ovnkube-config\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.174739 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.174592    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b05ac5f2-3cd6-48f4-857a-08403194adbe-env-overrides\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.175279 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.175096    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 15 10:18:07.175377 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.175320    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:07.175658 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.175601    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/572e0352-78c0-4269-a50f-4d6e86d06f46-cni-binary-copy\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.175823 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:07.175784    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gsxvv" podUID="c5d3dc08-8291-4828-87d1-f0da01fd24c8"
Sep 15 10:18:07.176016 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.175919    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91c14855-f9ab-4882-8185-8289a2bd8269-host\") pod \"node-ca-5gw48\" (UID: \"91c14855-f9ab-4882-8185-8289a2bd8269\") " pod="openshift-image-registry/node-ca-5gw48"
Sep 15 10:18:07.176016 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.175964    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/91c14855-f9ab-4882-8185-8289a2bd8269-serviceca\") pod \"node-ca-5gw48\" (UID: \"91c14855-f9ab-4882-8185-8289a2bd8269\") " pod="openshift-image-registry/node-ca-5gw48"
Sep 15 10:18:07.176016 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.175992    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-etc-openvswitch\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.176199 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.176162    2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 15 10:18:07.176437 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.176029    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-host-cni-netd\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.176502 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.176471    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88f8318b-0faa-4708-afc6-de7f7ae3cee3-tmp-dir\") pod \"node-resolver-mw292\" (UID: \"88f8318b-0faa-4708-afc6-de7f7ae3cee3\") " pod="openshift-dns/node-resolver-mw292"
Sep 15 10:18:07.176569 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.176519    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9c9rd\""
Sep 15 10:18:07.176569 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.176529    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6g87\" (UniqueName: \"kubernetes.io/projected/91c14855-f9ab-4882-8185-8289a2bd8269-kube-api-access-n6g87\") pod \"node-ca-5gw48\" (UID: \"91c14855-f9ab-4882-8185-8289a2bd8269\") " pod="openshift-image-registry/node-ca-5gw48"
Sep 15 10:18:07.176682 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.176566    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-var-lib-openvswitch\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.176682 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.176588    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 15 10:18:07.176682 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.176615    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-host-run-ovn-kubernetes\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.176823 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.176789    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-host-cni-bin\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.176878 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.176822    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-cnibin\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.176984 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.176960    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-host-var-lib-cni-multus\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.177042 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.177008    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-host-var-lib-kubelet\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.177097 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.177051    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thp2x\" (UniqueName: \"kubernetes.io/projected/572e0352-78c0-4269-a50f-4d6e86d06f46-kube-api-access-thp2x\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.177149 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.177097    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-host-slash\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.177201 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.177169    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-run-ovn\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.177252 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.177235    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b05ac5f2-3cd6-48f4-857a-08403194adbe-ovn-node-metrics-cert\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.178045 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.177383    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b05ac5f2-3cd6-48f4-857a-08403194adbe-ovnkube-script-lib\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.178045 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.177421    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-multus-conf-dir\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.178045 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.177571    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/572e0352-78c0-4269-a50f-4d6e86d06f46-multus-daemon-config\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.178045 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.177654    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/17509e5d-d9dc-4f53-a111-4929185b0a3c-agent-certs\") pod \"konnectivity-agent-8hkxj\" (UID: \"17509e5d-d9dc-4f53-a111-4929185b0a3c\") " pod="kube-system/konnectivity-agent-8hkxj"
Sep 15 10:18:07.178045 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.177693    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c512fef3-9326-49ca-a58e-4fa39b7d9814-iptables-alerter-script\") pod \"iptables-alerter-w2qnt\" (UID: \"c512fef3-9326-49ca-a58e-4fa39b7d9814\") " pod="openshift-network-operator/iptables-alerter-w2qnt"
Sep 15 10:18:07.178045 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.177782    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-node-log\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.178045 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.177830    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-log-socket\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.178045 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.177866    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/88f8318b-0faa-4708-afc6-de7f7ae3cee3-hosts-file\") pod \"node-resolver-mw292\" (UID: \"88f8318b-0faa-4708-afc6-de7f7ae3cee3\") " pod="openshift-dns/node-resolver-mw292"
Sep 15 10:18:07.196579 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.196554    2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-bs7qb"
Sep 15 10:18:07.204877 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.204842    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bs7qb"
Sep 15 10:18:07.263438 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.263405    2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 15 10:18:07.278637 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.278588    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-node-log\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.278637 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.278632    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/88f8318b-0faa-4708-afc6-de7f7ae3cee3-hosts-file\") pod \"node-resolver-mw292\" (UID: \"88f8318b-0faa-4708-afc6-de7f7ae3cee3\") " pod="openshift-dns/node-resolver-mw292"
Sep 15 10:18:07.278972 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.278659    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-282z4\" (UniqueName: \"kubernetes.io/projected/88f8318b-0faa-4708-afc6-de7f7ae3cee3-kube-api-access-282z4\") pod \"node-resolver-mw292\" (UID: \"88f8318b-0faa-4708-afc6-de7f7ae3cee3\") " pod="openshift-dns/node-resolver-mw292"
Sep 15 10:18:07.278972 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.278675    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-host-run-netns\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.278972 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.278700    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-hostroot\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.278972 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.278729    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b64b5ead-1ece-4aff-8975-5c59302a42d6-system-cni-dir\") pod \"multus-additional-cni-plugins-58dk4\" (UID: \"b64b5ead-1ece-4aff-8975-5c59302a42d6\") " pod="openshift-multus/multus-additional-cni-plugins-58dk4"
Sep 15 10:18:07.278972 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.278756    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-host-run-netns\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.278972 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.278725    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-node-log\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.278972 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.278760    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11f3ddfa-e976-47c6-abab-60db632f8e64-sys\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.278972 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.278778    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-hostroot\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.278972 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.278759    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/88f8318b-0faa-4708-afc6-de7f7ae3cee3-hosts-file\") pod \"node-resolver-mw292\" (UID: \"88f8318b-0faa-4708-afc6-de7f7ae3cee3\") " pod="openshift-dns/node-resolver-mw292"
Sep 15 10:18:07.278972 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.278815    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11f3ddfa-e976-47c6-abab-60db632f8e64-tmp\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.278972 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.278851    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b64b5ead-1ece-4aff-8975-5c59302a42d6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-58dk4\" (UID: \"b64b5ead-1ece-4aff-8975-5c59302a42d6\") " pod="openshift-multus/multus-additional-cni-plugins-58dk4"
Sep 15 10:18:07.278972 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.278886    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-host-run-k8s-cni-cncf-io\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.278972 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.278911    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/11f3ddfa-e976-47c6-abab-60db632f8e64-etc-sysconfig\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.278972 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.278937    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvhwf\" (UniqueName: \"kubernetes.io/projected/11f3ddfa-e976-47c6-abab-60db632f8e64-kube-api-access-wvhwf\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.278972 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.278963    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b64b5ead-1ece-4aff-8975-5c59302a42d6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-58dk4\" (UID: \"b64b5ead-1ece-4aff-8975-5c59302a42d6\") " pod="openshift-multus/multus-additional-cni-plugins-58dk4"
Sep 15 10:18:07.280945 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.278992    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-multus-cni-dir\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.280945 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.278989    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-host-run-k8s-cni-cncf-io\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.280945 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279019    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-etc-kubernetes\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.280945 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279062    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-etc-kubernetes\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.280945 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279085    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/11f3ddfa-e976-47c6-abab-60db632f8e64-run\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.280945 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279111    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11f3ddfa-e976-47c6-abab-60db632f8e64-lib-modules\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.280945 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279148    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-run-openvswitch\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.280945 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279174    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b05ac5f2-3cd6-48f4-857a-08403194adbe-ovnkube-config\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.280945 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279173    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-multus-cni-dir\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.280945 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279223    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-run-openvswitch\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.280945 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279229    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b05ac5f2-3cd6-48f4-857a-08403194adbe-env-overrides\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.280945 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279277    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/91c14855-f9ab-4882-8185-8289a2bd8269-serviceca\") pod \"node-ca-5gw48\" (UID: \"91c14855-f9ab-4882-8185-8289a2bd8269\") " pod="openshift-image-registry/node-ca-5gw48"
Sep 15 10:18:07.280945 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279323    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-host-cni-netd\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.280945 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279355    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/11f3ddfa-e976-47c6-abab-60db632f8e64-etc-modprobe-d\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.280945 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279390    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-host-run-ovn-kubernetes\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.280945 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279416    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-cnibin\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.280945 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279440    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/11f3ddfa-e976-47c6-abab-60db632f8e64-etc-systemd\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.280945 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279486    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11f3ddfa-e976-47c6-abab-60db632f8e64-host\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.281766 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279514    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b64b5ead-1ece-4aff-8975-5c59302a42d6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-58dk4\" (UID: \"b64b5ead-1ece-4aff-8975-5c59302a42d6\") " pod="openshift-multus/multus-additional-cni-plugins-58dk4"
Sep 15 10:18:07.281766 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279544    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5d3dc08-8291-4828-87d1-f0da01fd24c8-metrics-certs\") pod \"network-metrics-daemon-gsxvv\" (UID: \"c5d3dc08-8291-4828-87d1-f0da01fd24c8\") " pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:07.281766 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279569    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhj9v\" (UniqueName: \"kubernetes.io/projected/36de6650-259d-4ed2-9f48-43caefc90a18-kube-api-access-bhj9v\") pod \"aws-ebs-csi-driver-node-t4sn5\" (UID: \"36de6650-259d-4ed2-9f48-43caefc90a18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4sn5"
Sep 15 10:18:07.281766 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279597    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-run-ovn\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.281766 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279621    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b05ac5f2-3cd6-48f4-857a-08403194adbe-ovn-node-metrics-cert\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.281766 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279620    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-cnibin\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.281766 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279645    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b05ac5f2-3cd6-48f4-857a-08403194adbe-ovnkube-script-lib\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.281766 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279670    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11f3ddfa-e976-47c6-abab-60db632f8e64-etc-kubernetes\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.281766 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279681    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-host-cni-netd\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.281766 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279697    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/36de6650-259d-4ed2-9f48-43caefc90a18-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t4sn5\" (UID: \"36de6650-259d-4ed2-9f48-43caefc90a18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4sn5"
Sep 15 10:18:07.281766 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279721    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/36de6650-259d-4ed2-9f48-43caefc90a18-registration-dir\") pod \"aws-ebs-csi-driver-node-t4sn5\" (UID: \"36de6650-259d-4ed2-9f48-43caefc90a18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4sn5"
Sep 15 10:18:07.281766 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279737    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-host-run-ovn-kubernetes\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.281766 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279748    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c512fef3-9326-49ca-a58e-4fa39b7d9814-iptables-alerter-script\") pod \"iptables-alerter-w2qnt\" (UID: \"c512fef3-9326-49ca-a58e-4fa39b7d9814\") " pod="openshift-network-operator/iptables-alerter-w2qnt"
Sep 15 10:18:07.281766 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279764    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b05ac5f2-3cd6-48f4-857a-08403194adbe-env-overrides\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.281766 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279774    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-log-socket\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.281766 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279788    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-run-ovn\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.281766 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279805    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-multus-socket-dir-parent\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.282798 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279838    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/91c14855-f9ab-4882-8185-8289a2bd8269-serviceca\") pod \"node-ca-5gw48\" (UID: \"91c14855-f9ab-4882-8185-8289a2bd8269\") " pod="openshift-image-registry/node-ca-5gw48"
Sep 15 10:18:07.282798 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279830    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b64b5ead-1ece-4aff-8975-5c59302a42d6-cnibin\") pod \"multus-additional-cni-plugins-58dk4\" (UID: \"b64b5ead-1ece-4aff-8975-5c59302a42d6\") " pod="openshift-multus/multus-additional-cni-plugins-58dk4"
Sep 15 10:18:07.282798 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279864    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b05ac5f2-3cd6-48f4-857a-08403194adbe-ovnkube-config\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.282798 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279887    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/36de6650-259d-4ed2-9f48-43caefc90a18-device-dir\") pod \"aws-ebs-csi-driver-node-t4sn5\" (UID: \"36de6650-259d-4ed2-9f48-43caefc90a18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4sn5"
Sep 15 10:18:07.282798 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279918    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-log-socket\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.282798 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279923    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-multus-socket-dir-parent\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.282798 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279959    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d9306059-36e0-4eae-8dc8-3ce075dc9b21-original-pull-secret\") pod \"global-pull-secret-syncer-qjcwx\" (UID: \"d9306059-36e0-4eae-8dc8-3ce075dc9b21\") " pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:07.282798 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.279991    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c512fef3-9326-49ca-a58e-4fa39b7d9814-host-slash\") pod \"iptables-alerter-w2qnt\" (UID: \"c512fef3-9326-49ca-a58e-4fa39b7d9814\") " pod="openshift-network-operator/iptables-alerter-w2qnt"
Sep 15 10:18:07.282798 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.280020    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-host-kubelet\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.282798 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.280072    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-systemd-units\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.282798 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.280099    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-host-run-netns\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.282798 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.280102    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c512fef3-9326-49ca-a58e-4fa39b7d9814-host-slash\") pod \"iptables-alerter-w2qnt\" (UID: \"c512fef3-9326-49ca-a58e-4fa39b7d9814\") " pod="openshift-network-operator/iptables-alerter-w2qnt"
Sep 15 10:18:07.282798 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.280104    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-host-kubelet\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.282798 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.280149    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-run-systemd\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.282798 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.280164    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-systemd-units\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.282798 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.280181    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-system-cni-dir\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.282798 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.280206    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-host-run-netns\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.283710 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.280212    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-run-systemd\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.283710 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.280210    2583 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 15 10:18:07.283710 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.280211    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b64b5ead-1ece-4aff-8975-5c59302a42d6-cni-binary-copy\") pod \"multus-additional-cni-plugins-58dk4\" (UID: \"b64b5ead-1ece-4aff-8975-5c59302a42d6\") " pod="openshift-multus/multus-additional-cni-plugins-58dk4"
Sep 15 10:18:07.283710 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.280263    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcrzb\" (UniqueName: \"kubernetes.io/projected/b64b5ead-1ece-4aff-8975-5c59302a42d6-kube-api-access-mcrzb\") pod \"multus-additional-cni-plugins-58dk4\" (UID: \"b64b5ead-1ece-4aff-8975-5c59302a42d6\") " pod="openshift-multus/multus-additional-cni-plugins-58dk4"
Sep 15 10:18:07.283710 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.280295    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t6hb\" (UniqueName: \"kubernetes.io/projected/c512fef3-9326-49ca-a58e-4fa39b7d9814-kube-api-access-6t6hb\") pod \"iptables-alerter-w2qnt\" (UID: \"c512fef3-9326-49ca-a58e-4fa39b7d9814\") " pod="openshift-network-operator/iptables-alerter-w2qnt"
Sep 15 10:18:07.283710 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.280328    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/11f3ddfa-e976-47c6-abab-60db632f8e64-etc-sysctl-d\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.283710 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.280329    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-system-cni-dir\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.283710 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.280366    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/36de6650-259d-4ed2-9f48-43caefc90a18-etc-selinux\") pod \"aws-ebs-csi-driver-node-t4sn5\" (UID: \"36de6650-259d-4ed2-9f48-43caefc90a18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4sn5"
Sep 15 10:18:07.283710 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.280399    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d9306059-36e0-4eae-8dc8-3ce075dc9b21-kubelet-config\") pod \"global-pull-secret-syncer-qjcwx\" (UID: \"d9306059-36e0-4eae-8dc8-3ce075dc9b21\") " pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:07.283710 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.280404    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b05ac5f2-3cd6-48f4-857a-08403194adbe-ovnkube-script-lib\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.283710 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.280425    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d9306059-36e0-4eae-8dc8-3ce075dc9b21-dbus\") pod \"global-pull-secret-syncer-qjcwx\" (UID: \"d9306059-36e0-4eae-8dc8-3ce075dc9b21\") " pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:07.283710 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.280494    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqtl9\" (UniqueName: \"kubernetes.io/projected/88d49ac3-3a21-43ee-b74b-dfbd6239292e-kube-api-access-dqtl9\") pod \"network-check-target-rxb8p\" (UID: \"88d49ac3-3a21-43ee-b74b-dfbd6239292e\") " pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:18:07.283710 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.280504    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c512fef3-9326-49ca-a58e-4fa39b7d9814-iptables-alerter-script\") pod \"iptables-alerter-w2qnt\" (UID: \"c512fef3-9326-49ca-a58e-4fa39b7d9814\") " pod="openshift-network-operator/iptables-alerter-w2qnt"
Sep 15 10:18:07.283710 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.280534    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh7jp\" (UniqueName: \"kubernetes.io/projected/b05ac5f2-3cd6-48f4-857a-08403194adbe-kube-api-access-qh7jp\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.283710 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.280680    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-host-var-lib-cni-bin\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.283710 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.280794    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-host-run-multus-certs\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.283710 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.280829    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-host-var-lib-cni-bin\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.284388 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.280876    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/17509e5d-d9dc-4f53-a111-4929185b0a3c-konnectivity-ca\") pod \"konnectivity-agent-8hkxj\" (UID: \"17509e5d-d9dc-4f53-a111-4929185b0a3c\") " pod="kube-system/konnectivity-agent-8hkxj"
Sep 15 10:18:07.284388 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.280908    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.284388 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.280881    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-host-run-multus-certs\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.284388 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.280939    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/572e0352-78c0-4269-a50f-4d6e86d06f46-cni-binary-copy\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.284388 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.281567    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/17509e5d-d9dc-4f53-a111-4929185b0a3c-konnectivity-ca\") pod \"konnectivity-agent-8hkxj\" (UID: \"17509e5d-d9dc-4f53-a111-4929185b0a3c\") " pod="kube-system/konnectivity-agent-8hkxj"
Sep 15 10:18:07.284388 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.281576    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/572e0352-78c0-4269-a50f-4d6e86d06f46-cni-binary-copy\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.284388 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.280988    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.284388 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.281589    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91c14855-f9ab-4882-8185-8289a2bd8269-host\") pod \"node-ca-5gw48\" (UID: \"91c14855-f9ab-4882-8185-8289a2bd8269\") " pod="openshift-image-registry/node-ca-5gw48"
Sep 15 10:18:07.284388 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.281630    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-etc-openvswitch\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.284388 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.281629    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91c14855-f9ab-4882-8185-8289a2bd8269-host\") pod \"node-ca-5gw48\" (UID: \"91c14855-f9ab-4882-8185-8289a2bd8269\") " pod="openshift-image-registry/node-ca-5gw48"
Sep 15 10:18:07.284388 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.281668    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88f8318b-0faa-4708-afc6-de7f7ae3cee3-tmp-dir\") pod \"node-resolver-mw292\" (UID: \"88f8318b-0faa-4708-afc6-de7f7ae3cee3\") " pod="openshift-dns/node-resolver-mw292"
Sep 15 10:18:07.284388 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.281685    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-etc-openvswitch\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.284388 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.281698    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/11f3ddfa-e976-47c6-abab-60db632f8e64-etc-tuned\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.284388 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.281716    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6g87\" (UniqueName: \"kubernetes.io/projected/91c14855-f9ab-4882-8185-8289a2bd8269-kube-api-access-n6g87\") pod \"node-ca-5gw48\" (UID: \"91c14855-f9ab-4882-8185-8289a2bd8269\") " pod="openshift-image-registry/node-ca-5gw48"
Sep 15 10:18:07.284388 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.281737    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-var-lib-openvswitch\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.284388 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.281761    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-host-cni-bin\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.284388 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.281807    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-host-var-lib-cni-multus\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.285243 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.281823    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-host-var-lib-kubelet\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.285243 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.281839    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thp2x\" (UniqueName: \"kubernetes.io/projected/572e0352-78c0-4269-a50f-4d6e86d06f46-kube-api-access-thp2x\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.285243 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.281856    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k66tv\" (UniqueName: \"kubernetes.io/projected/c5d3dc08-8291-4828-87d1-f0da01fd24c8-kube-api-access-k66tv\") pod \"network-metrics-daemon-gsxvv\" (UID: \"c5d3dc08-8291-4828-87d1-f0da01fd24c8\") " pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:07.285243 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.281872    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/11f3ddfa-e976-47c6-abab-60db632f8e64-etc-sysctl-conf\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.285243 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.281891    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-host-slash\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.285243 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.281908    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-multus-conf-dir\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.285243 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.281924    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/572e0352-78c0-4269-a50f-4d6e86d06f46-multus-daemon-config\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.285243 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.281966    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/17509e5d-d9dc-4f53-a111-4929185b0a3c-agent-certs\") pod \"konnectivity-agent-8hkxj\" (UID: \"17509e5d-d9dc-4f53-a111-4929185b0a3c\") " pod="kube-system/konnectivity-agent-8hkxj"
Sep 15 10:18:07.285243 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.281983    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11f3ddfa-e976-47c6-abab-60db632f8e64-var-lib-kubelet\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.285243 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.282002    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/36de6650-259d-4ed2-9f48-43caefc90a18-socket-dir\") pod \"aws-ebs-csi-driver-node-t4sn5\" (UID: \"36de6650-259d-4ed2-9f48-43caefc90a18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4sn5"
Sep 15 10:18:07.285243 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.282018    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/36de6650-259d-4ed2-9f48-43caefc90a18-sys-fs\") pod \"aws-ebs-csi-driver-node-t4sn5\" (UID: \"36de6650-259d-4ed2-9f48-43caefc90a18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4sn5"
Sep 15 10:18:07.285243 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.282060    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88f8318b-0faa-4708-afc6-de7f7ae3cee3-tmp-dir\") pod \"node-resolver-mw292\" (UID: \"88f8318b-0faa-4708-afc6-de7f7ae3cee3\") " pod="openshift-dns/node-resolver-mw292"
Sep 15 10:18:07.285243 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.282164    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-var-lib-openvswitch\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.285243 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.282187    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-host-var-lib-kubelet\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.285243 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.282348    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-host-var-lib-cni-multus\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.285243 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.282459    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-host-slash\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.285243 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.282567    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/572e0352-78c0-4269-a50f-4d6e86d06f46-multus-conf-dir\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.285927 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.282701    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b05ac5f2-3cd6-48f4-857a-08403194adbe-host-cni-bin\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.285927 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.282722    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/572e0352-78c0-4269-a50f-4d6e86d06f46-multus-daemon-config\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.285927 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.285059    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b05ac5f2-3cd6-48f4-857a-08403194adbe-ovn-node-metrics-cert\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.285927 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.285138    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/17509e5d-d9dc-4f53-a111-4929185b0a3c-agent-certs\") pod \"konnectivity-agent-8hkxj\" (UID: \"17509e5d-d9dc-4f53-a111-4929185b0a3c\") " pod="kube-system/konnectivity-agent-8hkxj"
Sep 15 10:18:07.286707 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.286689    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-282z4\" (UniqueName: \"kubernetes.io/projected/88f8318b-0faa-4708-afc6-de7f7ae3cee3-kube-api-access-282z4\") pod \"node-resolver-mw292\" (UID: \"88f8318b-0faa-4708-afc6-de7f7ae3cee3\") " pod="openshift-dns/node-resolver-mw292"
Sep 15 10:18:07.291699 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:07.291675    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 10:18:07.291802 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:07.291717    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 10:18:07.291802 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:07.291732    2583 projected.go:196] Error preparing data for projected volume kube-api-access-dqtl9 for pod openshift-network-diagnostics/network-check-target-rxb8p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 10:18:07.291936 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:07.291818    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88d49ac3-3a21-43ee-b74b-dfbd6239292e-kube-api-access-dqtl9 podName:88d49ac3-3a21-43ee-b74b-dfbd6239292e nodeName:}" failed. No retries permitted until 2026-09-15 10:18:07.791794877 +0000 UTC m=+3.038761842 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-dqtl9" (UniqueName: "kubernetes.io/projected/88d49ac3-3a21-43ee-b74b-dfbd6239292e-kube-api-access-dqtl9") pod "network-check-target-rxb8p" (UID: "88d49ac3-3a21-43ee-b74b-dfbd6239292e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 10:18:07.292222 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.292198    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t6hb\" (UniqueName: \"kubernetes.io/projected/c512fef3-9326-49ca-a58e-4fa39b7d9814-kube-api-access-6t6hb\") pod \"iptables-alerter-w2qnt\" (UID: \"c512fef3-9326-49ca-a58e-4fa39b7d9814\") " pod="openshift-network-operator/iptables-alerter-w2qnt"
Sep 15 10:18:07.294288 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.294263    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thp2x\" (UniqueName: \"kubernetes.io/projected/572e0352-78c0-4269-a50f-4d6e86d06f46-kube-api-access-thp2x\") pod \"multus-n5sdr\" (UID: \"572e0352-78c0-4269-a50f-4d6e86d06f46\") " pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.294401 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.294374    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6g87\" (UniqueName: \"kubernetes.io/projected/91c14855-f9ab-4882-8185-8289a2bd8269-kube-api-access-n6g87\") pod \"node-ca-5gw48\" (UID: \"91c14855-f9ab-4882-8185-8289a2bd8269\") " pod="openshift-image-registry/node-ca-5gw48"
Sep 15 10:18:07.294797 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.294779    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh7jp\" (UniqueName: \"kubernetes.io/projected/b05ac5f2-3cd6-48f4-857a-08403194adbe-kube-api-access-qh7jp\") pod \"ovnkube-node-pkmxf\" (UID: \"b05ac5f2-3cd6-48f4-857a-08403194adbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.299386 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.299339    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-235.ec2.internal" event={"ID":"2b3d6833f74b07abf73dd136dadede80","Type":"ContainerStarted","Data":"6055a8d51da4917fe066e8a0a33bfa8f916b60bfcb234051a237c23900ab2217"}
Sep 15 10:18:07.300707 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.300679    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-235.ec2.internal" event={"ID":"93a9cd305dd93bbe9f79180cbc0e6eaa","Type":"ContainerStarted","Data":"dc5070c4b121442edc5ee965ecf634d35c62ec229364f48444b45826641e10b5"}
Sep 15 10:18:07.382588 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.382544    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b64b5ead-1ece-4aff-8975-5c59302a42d6-cni-binary-copy\") pod \"multus-additional-cni-plugins-58dk4\" (UID: \"b64b5ead-1ece-4aff-8975-5c59302a42d6\") " pod="openshift-multus/multus-additional-cni-plugins-58dk4"
Sep 15 10:18:07.382798 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.382599    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcrzb\" (UniqueName: \"kubernetes.io/projected/b64b5ead-1ece-4aff-8975-5c59302a42d6-kube-api-access-mcrzb\") pod \"multus-additional-cni-plugins-58dk4\" (UID: \"b64b5ead-1ece-4aff-8975-5c59302a42d6\") " pod="openshift-multus/multus-additional-cni-plugins-58dk4"
Sep 15 10:18:07.382798 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.382717    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/11f3ddfa-e976-47c6-abab-60db632f8e64-etc-sysctl-d\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.382798 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.382762    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/36de6650-259d-4ed2-9f48-43caefc90a18-etc-selinux\") pod \"aws-ebs-csi-driver-node-t4sn5\" (UID: \"36de6650-259d-4ed2-9f48-43caefc90a18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4sn5"
Sep 15 10:18:07.382798 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.382789    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d9306059-36e0-4eae-8dc8-3ce075dc9b21-kubelet-config\") pod \"global-pull-secret-syncer-qjcwx\" (UID: \"d9306059-36e0-4eae-8dc8-3ce075dc9b21\") " pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:07.383090 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.382815    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d9306059-36e0-4eae-8dc8-3ce075dc9b21-dbus\") pod \"global-pull-secret-syncer-qjcwx\" (UID: \"d9306059-36e0-4eae-8dc8-3ce075dc9b21\") " pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:07.383090 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.382868    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/11f3ddfa-e976-47c6-abab-60db632f8e64-etc-tuned\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.383090 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.382903    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k66tv\" (UniqueName: \"kubernetes.io/projected/c5d3dc08-8291-4828-87d1-f0da01fd24c8-kube-api-access-k66tv\") pod \"network-metrics-daemon-gsxvv\" (UID: \"c5d3dc08-8291-4828-87d1-f0da01fd24c8\") " pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:07.383090 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.382928    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/11f3ddfa-e976-47c6-abab-60db632f8e64-etc-sysctl-conf\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.383090 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.382960    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11f3ddfa-e976-47c6-abab-60db632f8e64-var-lib-kubelet\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.383090 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.382985    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/36de6650-259d-4ed2-9f48-43caefc90a18-socket-dir\") pod \"aws-ebs-csi-driver-node-t4sn5\" (UID: \"36de6650-259d-4ed2-9f48-43caefc90a18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4sn5"
Sep 15 10:18:07.383090 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.382990    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/36de6650-259d-4ed2-9f48-43caefc90a18-etc-selinux\") pod \"aws-ebs-csi-driver-node-t4sn5\" (UID: \"36de6650-259d-4ed2-9f48-43caefc90a18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4sn5"
Sep 15 10:18:07.383090 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.383013    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/36de6650-259d-4ed2-9f48-43caefc90a18-sys-fs\") pod \"aws-ebs-csi-driver-node-t4sn5\" (UID: \"36de6650-259d-4ed2-9f48-43caefc90a18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4sn5"
Sep 15 10:18:07.383090 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.383046    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b64b5ead-1ece-4aff-8975-5c59302a42d6-system-cni-dir\") pod \"multus-additional-cni-plugins-58dk4\" (UID: \"b64b5ead-1ece-4aff-8975-5c59302a42d6\") " pod="openshift-multus/multus-additional-cni-plugins-58dk4"
Sep 15 10:18:07.383090 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.383075    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11f3ddfa-e976-47c6-abab-60db632f8e64-sys\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.383615 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.383099    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11f3ddfa-e976-47c6-abab-60db632f8e64-tmp\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.383615 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.383113    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/11f3ddfa-e976-47c6-abab-60db632f8e64-etc-sysctl-d\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.383615 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.383124    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b64b5ead-1ece-4aff-8975-5c59302a42d6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-58dk4\" (UID: \"b64b5ead-1ece-4aff-8975-5c59302a42d6\") " pod="openshift-multus/multus-additional-cni-plugins-58dk4"
Sep 15 10:18:07.383615 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.383154    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/11f3ddfa-e976-47c6-abab-60db632f8e64-etc-sysconfig\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.383615 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.383165    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11f3ddfa-e976-47c6-abab-60db632f8e64-var-lib-kubelet\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.383615 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.383178    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvhwf\" (UniqueName: \"kubernetes.io/projected/11f3ddfa-e976-47c6-abab-60db632f8e64-kube-api-access-wvhwf\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.383615 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.383205    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b64b5ead-1ece-4aff-8975-5c59302a42d6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-58dk4\" (UID: \"b64b5ead-1ece-4aff-8975-5c59302a42d6\") " pod="openshift-multus/multus-additional-cni-plugins-58dk4"
Sep 15 10:18:07.383615 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.383211    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d9306059-36e0-4eae-8dc8-3ce075dc9b21-kubelet-config\") pod \"global-pull-secret-syncer-qjcwx\" (UID: \"d9306059-36e0-4eae-8dc8-3ce075dc9b21\") " pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:07.383615 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.383232    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/11f3ddfa-e976-47c6-abab-60db632f8e64-run\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.383615 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.383256    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11f3ddfa-e976-47c6-abab-60db632f8e64-lib-modules\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.383615 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.383291    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/11f3ddfa-e976-47c6-abab-60db632f8e64-etc-modprobe-d\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.383615 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.383319    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/11f3ddfa-e976-47c6-abab-60db632f8e64-etc-systemd\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.383615 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.383335    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b64b5ead-1ece-4aff-8975-5c59302a42d6-cni-binary-copy\") pod \"multus-additional-cni-plugins-58dk4\" (UID: \"b64b5ead-1ece-4aff-8975-5c59302a42d6\") " pod="openshift-multus/multus-additional-cni-plugins-58dk4"
Sep 15 10:18:07.383615 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.383414    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11f3ddfa-e976-47c6-abab-60db632f8e64-host\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.383615 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.383342    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11f3ddfa-e976-47c6-abab-60db632f8e64-host\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.383615 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.383463    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b64b5ead-1ece-4aff-8975-5c59302a42d6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-58dk4\" (UID: \"b64b5ead-1ece-4aff-8975-5c59302a42d6\") " pod="openshift-multus/multus-additional-cni-plugins-58dk4"
Sep 15 10:18:07.383615 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.383485    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/36de6650-259d-4ed2-9f48-43caefc90a18-socket-dir\") pod \"aws-ebs-csi-driver-node-t4sn5\" (UID: \"36de6650-259d-4ed2-9f48-43caefc90a18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4sn5"
Sep 15 10:18:07.384467 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.383494    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5d3dc08-8291-4828-87d1-f0da01fd24c8-metrics-certs\") pod \"network-metrics-daemon-gsxvv\" (UID: \"c5d3dc08-8291-4828-87d1-f0da01fd24c8\") " pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:07.384467 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.383555    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhj9v\" (UniqueName: \"kubernetes.io/projected/36de6650-259d-4ed2-9f48-43caefc90a18-kube-api-access-bhj9v\") pod \"aws-ebs-csi-driver-node-t4sn5\" (UID: \"36de6650-259d-4ed2-9f48-43caefc90a18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4sn5"
Sep 15 10:18:07.384467 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:07.383594    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 10:18:07.384467 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.383589    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11f3ddfa-e976-47c6-abab-60db632f8e64-etc-kubernetes\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.384467 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.383634    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/36de6650-259d-4ed2-9f48-43caefc90a18-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t4sn5\" (UID: \"36de6650-259d-4ed2-9f48-43caefc90a18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4sn5"
Sep 15 10:18:07.384467 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:07.383665    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5d3dc08-8291-4828-87d1-f0da01fd24c8-metrics-certs podName:c5d3dc08-8291-4828-87d1-f0da01fd24c8 nodeName:}" failed. No retries permitted until 2026-09-15 10:18:07.883645771 +0000 UTC m=+3.130612743 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5d3dc08-8291-4828-87d1-f0da01fd24c8-metrics-certs") pod "network-metrics-daemon-gsxvv" (UID: "c5d3dc08-8291-4828-87d1-f0da01fd24c8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 10:18:07.384467 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.383693    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/36de6650-259d-4ed2-9f48-43caefc90a18-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t4sn5\" (UID: \"36de6650-259d-4ed2-9f48-43caefc90a18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4sn5"
Sep 15 10:18:07.384467 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.383697    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/36de6650-259d-4ed2-9f48-43caefc90a18-registration-dir\") pod \"aws-ebs-csi-driver-node-t4sn5\" (UID: \"36de6650-259d-4ed2-9f48-43caefc90a18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4sn5"
Sep 15 10:18:07.384467 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.383750    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/36de6650-259d-4ed2-9f48-43caefc90a18-registration-dir\") pod \"aws-ebs-csi-driver-node-t4sn5\" (UID: \"36de6650-259d-4ed2-9f48-43caefc90a18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4sn5"
Sep 15 10:18:07.384467 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.383763    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/36de6650-259d-4ed2-9f48-43caefc90a18-sys-fs\") pod \"aws-ebs-csi-driver-node-t4sn5\" (UID: \"36de6650-259d-4ed2-9f48-43caefc90a18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4sn5"
Sep 15 10:18:07.384467 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.383792    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b64b5ead-1ece-4aff-8975-5c59302a42d6-cnibin\") pod \"multus-additional-cni-plugins-58dk4\" (UID: \"b64b5ead-1ece-4aff-8975-5c59302a42d6\") " pod="openshift-multus/multus-additional-cni-plugins-58dk4"
Sep 15 10:18:07.384467 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.383819    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/36de6650-259d-4ed2-9f48-43caefc90a18-device-dir\") pod \"aws-ebs-csi-driver-node-t4sn5\" (UID: \"36de6650-259d-4ed2-9f48-43caefc90a18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4sn5"
Sep 15 10:18:07.384467 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.383847    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d9306059-36e0-4eae-8dc8-3ce075dc9b21-original-pull-secret\") pod \"global-pull-secret-syncer-qjcwx\" (UID: \"d9306059-36e0-4eae-8dc8-3ce075dc9b21\") " pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:07.384467 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.383922    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b64b5ead-1ece-4aff-8975-5c59302a42d6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-58dk4\" (UID: \"b64b5ead-1ece-4aff-8975-5c59302a42d6\") " pod="openshift-multus/multus-additional-cni-plugins-58dk4"
Sep 15 10:18:07.384467 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.383340    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d9306059-36e0-4eae-8dc8-3ce075dc9b21-dbus\") pod \"global-pull-secret-syncer-qjcwx\" (UID: \"d9306059-36e0-4eae-8dc8-3ce075dc9b21\") " pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:07.384467 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.383985    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b64b5ead-1ece-4aff-8975-5c59302a42d6-system-cni-dir\") pod \"multus-additional-cni-plugins-58dk4\" (UID: \"b64b5ead-1ece-4aff-8975-5c59302a42d6\") " pod="openshift-multus/multus-additional-cni-plugins-58dk4"
Sep 15 10:18:07.384467 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:07.383996    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 10:18:07.385317 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.384044    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/11f3ddfa-e976-47c6-abab-60db632f8e64-etc-sysconfig\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.385317 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:07.384061    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9306059-36e0-4eae-8dc8-3ce075dc9b21-original-pull-secret podName:d9306059-36e0-4eae-8dc8-3ce075dc9b21 nodeName:}" failed. No retries permitted until 2026-09-15 10:18:07.884043003 +0000 UTC m=+3.131009975 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d9306059-36e0-4eae-8dc8-3ce075dc9b21-original-pull-secret") pod "global-pull-secret-syncer-qjcwx" (UID: "d9306059-36e0-4eae-8dc8-3ce075dc9b21") : object "kube-system"/"original-pull-secret" not registered
Sep 15 10:18:07.385317 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.384114    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/11f3ddfa-e976-47c6-abab-60db632f8e64-run\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.385317 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.384229    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11f3ddfa-e976-47c6-abab-60db632f8e64-lib-modules\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.385317 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.384236    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b64b5ead-1ece-4aff-8975-5c59302a42d6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-58dk4\" (UID: \"b64b5ead-1ece-4aff-8975-5c59302a42d6\") " pod="openshift-multus/multus-additional-cni-plugins-58dk4"
Sep 15 10:18:07.385317 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.384260    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11f3ddfa-e976-47c6-abab-60db632f8e64-sys\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.385317 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.384295    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/11f3ddfa-e976-47c6-abab-60db632f8e64-etc-systemd\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.385317 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.384353    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/11f3ddfa-e976-47c6-abab-60db632f8e64-etc-modprobe-d\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.385317 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.384400    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b64b5ead-1ece-4aff-8975-5c59302a42d6-cnibin\") pod \"multus-additional-cni-plugins-58dk4\" (UID: \"b64b5ead-1ece-4aff-8975-5c59302a42d6\") " pod="openshift-multus/multus-additional-cni-plugins-58dk4"
Sep 15 10:18:07.385317 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.384415    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/11f3ddfa-e976-47c6-abab-60db632f8e64-etc-sysctl-conf\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.385317 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.384445    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b64b5ead-1ece-4aff-8975-5c59302a42d6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-58dk4\" (UID: \"b64b5ead-1ece-4aff-8975-5c59302a42d6\") " pod="openshift-multus/multus-additional-cni-plugins-58dk4"
Sep 15 10:18:07.385317 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.384454    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11f3ddfa-e976-47c6-abab-60db632f8e64-etc-kubernetes\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.385317 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.384469    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/36de6650-259d-4ed2-9f48-43caefc90a18-device-dir\") pod \"aws-ebs-csi-driver-node-t4sn5\" (UID: \"36de6650-259d-4ed2-9f48-43caefc90a18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4sn5"
Sep 15 10:18:07.386837 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.386774    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/11f3ddfa-e976-47c6-abab-60db632f8e64-etc-tuned\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.386951 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.386931    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11f3ddfa-e976-47c6-abab-60db632f8e64-tmp\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.390680 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.390653    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcrzb\" (UniqueName: \"kubernetes.io/projected/b64b5ead-1ece-4aff-8975-5c59302a42d6-kube-api-access-mcrzb\") pod \"multus-additional-cni-plugins-58dk4\" (UID: \"b64b5ead-1ece-4aff-8975-5c59302a42d6\") " pod="openshift-multus/multus-additional-cni-plugins-58dk4"
Sep 15 10:18:07.393111 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.393072    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k66tv\" (UniqueName: \"kubernetes.io/projected/c5d3dc08-8291-4828-87d1-f0da01fd24c8-kube-api-access-k66tv\") pod \"network-metrics-daemon-gsxvv\" (UID: \"c5d3dc08-8291-4828-87d1-f0da01fd24c8\") " pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:07.393325 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.393298    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhj9v\" (UniqueName: \"kubernetes.io/projected/36de6650-259d-4ed2-9f48-43caefc90a18-kube-api-access-bhj9v\") pod \"aws-ebs-csi-driver-node-t4sn5\" (UID: \"36de6650-259d-4ed2-9f48-43caefc90a18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4sn5"
Sep 15 10:18:07.393494 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.393474    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvhwf\" (UniqueName: \"kubernetes.io/projected/11f3ddfa-e976-47c6-abab-60db632f8e64-kube-api-access-wvhwf\") pod \"tuned-vf2xj\" (UID: \"11f3ddfa-e976-47c6-abab-60db632f8e64\") " pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.463023 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.462984    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5gw48"
Sep 15 10:18:07.474548 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.474526    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-w2qnt"
Sep 15 10:18:07.475942 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:07.475912    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91c14855_f9ab_4882_8185_8289a2bd8269.slice/crio-93ecf85185abe166b61a9e816e1e03392f0f00edcaadf7e7559e4ae47b1688a9 WatchSource:0}: Error finding container 93ecf85185abe166b61a9e816e1e03392f0f00edcaadf7e7559e4ae47b1688a9: Status 404 returned error can't find the container with id 93ecf85185abe166b61a9e816e1e03392f0f00edcaadf7e7559e4ae47b1688a9
Sep 15 10:18:07.484636 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.484613    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-n5sdr"
Sep 15 10:18:07.486888 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:07.486862    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc512fef3_9326_49ca_a58e_4fa39b7d9814.slice/crio-be1279ddf919287ad7dd573dfc875514face579aa6a4235443b0df872bab561f WatchSource:0}: Error finding container be1279ddf919287ad7dd573dfc875514face579aa6a4235443b0df872bab561f: Status 404 returned error can't find the container with id be1279ddf919287ad7dd573dfc875514face579aa6a4235443b0df872bab561f
Sep 15 10:18:07.489432 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.489375    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:07.494344 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:07.494316    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod572e0352_78c0_4269_a50f_4d6e86d06f46.slice/crio-03eb77434d786a36ac39f12ba3b5220cdbe583ba5462cd5e571cd0258139fa76 WatchSource:0}: Error finding container 03eb77434d786a36ac39f12ba3b5220cdbe583ba5462cd5e571cd0258139fa76: Status 404 returned error can't find the container with id 03eb77434d786a36ac39f12ba3b5220cdbe583ba5462cd5e571cd0258139fa76
Sep 15 10:18:07.496028 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.496004    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8hkxj"
Sep 15 10:18:07.499859 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:07.499834    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb05ac5f2_3cd6_48f4_857a_08403194adbe.slice/crio-aa7d08d3d95cd89cf634ec00de665de86a7110f2838f5e05e9a97fc14a60b88e WatchSource:0}: Error finding container aa7d08d3d95cd89cf634ec00de665de86a7110f2838f5e05e9a97fc14a60b88e: Status 404 returned error can't find the container with id aa7d08d3d95cd89cf634ec00de665de86a7110f2838f5e05e9a97fc14a60b88e
Sep 15 10:18:07.504115 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.504088    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mw292"
Sep 15 10:18:07.506280 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:07.506252    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17509e5d_d9dc_4f53_a111_4929185b0a3c.slice/crio-1e739657176815fe657e8dcb3d2d2b994fc22ca2de4d21851450f9592f2e311b WatchSource:0}: Error finding container 1e739657176815fe657e8dcb3d2d2b994fc22ca2de4d21851450f9592f2e311b: Status 404 returned error can't find the container with id 1e739657176815fe657e8dcb3d2d2b994fc22ca2de4d21851450f9592f2e311b
Sep 15 10:18:07.511060 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.511037    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4sn5"
Sep 15 10:18:07.515515 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:07.515487    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88f8318b_0faa_4708_afc6_de7f7ae3cee3.slice/crio-86efd3ea4077d010edf89ed6ee58303fcaf9a802956032cc386b024e93bdb165 WatchSource:0}: Error finding container 86efd3ea4077d010edf89ed6ee58303fcaf9a802956032cc386b024e93bdb165: Status 404 returned error can't find the container with id 86efd3ea4077d010edf89ed6ee58303fcaf9a802956032cc386b024e93bdb165
Sep 15 10:18:07.517657 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.517637    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vf2xj"
Sep 15 10:18:07.521984 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:07.521959    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36de6650_259d_4ed2_9f48_43caefc90a18.slice/crio-dc70cfd8937d084a2774c7c2109c41efaba17c68cbc61c9e77ba4be18bbb7c63 WatchSource:0}: Error finding container dc70cfd8937d084a2774c7c2109c41efaba17c68cbc61c9e77ba4be18bbb7c63: Status 404 returned error can't find the container with id dc70cfd8937d084a2774c7c2109c41efaba17c68cbc61c9e77ba4be18bbb7c63
Sep 15 10:18:07.523219 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.523146    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-58dk4"
Sep 15 10:18:07.530346 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:07.530323    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11f3ddfa_e976_47c6_abab_60db632f8e64.slice/crio-a949b22a5724a5385e69c4ce1f639682246dba79269ef7b9448149c8e49eda37 WatchSource:0}: Error finding container a949b22a5724a5385e69c4ce1f639682246dba79269ef7b9448149c8e49eda37: Status 404 returned error can't find the container with id a949b22a5724a5385e69c4ce1f639682246dba79269ef7b9448149c8e49eda37
Sep 15 10:18:07.534245 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:07.534220    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb64b5ead_1ece_4aff_8975_5c59302a42d6.slice/crio-d79a1df7b26a704962eed6db3834d155e3e085d847a373343721503c184ba688 WatchSource:0}: Error finding container d79a1df7b26a704962eed6db3834d155e3e085d847a373343721503c184ba688: Status 404 returned error can't find the container with id d79a1df7b26a704962eed6db3834d155e3e085d847a373343721503c184ba688
Sep 15 10:18:07.687132 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.687041    2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 15 10:18:07.887760 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.887721    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5d3dc08-8291-4828-87d1-f0da01fd24c8-metrics-certs\") pod \"network-metrics-daemon-gsxvv\" (UID: \"c5d3dc08-8291-4828-87d1-f0da01fd24c8\") " pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:07.888217 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.887777    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d9306059-36e0-4eae-8dc8-3ce075dc9b21-original-pull-secret\") pod \"global-pull-secret-syncer-qjcwx\" (UID: \"d9306059-36e0-4eae-8dc8-3ce075dc9b21\") " pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:07.888217 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:07.887813    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqtl9\" (UniqueName: \"kubernetes.io/projected/88d49ac3-3a21-43ee-b74b-dfbd6239292e-kube-api-access-dqtl9\") pod \"network-check-target-rxb8p\" (UID: \"88d49ac3-3a21-43ee-b74b-dfbd6239292e\") " pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:18:07.888217 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:07.887855    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 10:18:07.888217 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:07.887930    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5d3dc08-8291-4828-87d1-f0da01fd24c8-metrics-certs podName:c5d3dc08-8291-4828-87d1-f0da01fd24c8 nodeName:}" failed. No retries permitted until 2026-09-15 10:18:08.887909127 +0000 UTC m=+4.134876081 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5d3dc08-8291-4828-87d1-f0da01fd24c8-metrics-certs") pod "network-metrics-daemon-gsxvv" (UID: "c5d3dc08-8291-4828-87d1-f0da01fd24c8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 10:18:07.888217 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:07.887959    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 10:18:07.888217 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:07.887974    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 10:18:07.888217 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:07.887983    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 10:18:07.888217 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:07.887996    2583 projected.go:196] Error preparing data for projected volume kube-api-access-dqtl9 for pod openshift-network-diagnostics/network-check-target-rxb8p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 10:18:07.888217 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:07.888028    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9306059-36e0-4eae-8dc8-3ce075dc9b21-original-pull-secret podName:d9306059-36e0-4eae-8dc8-3ce075dc9b21 nodeName:}" failed. No retries permitted until 2026-09-15 10:18:08.888011735 +0000 UTC m=+4.134978709 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d9306059-36e0-4eae-8dc8-3ce075dc9b21-original-pull-secret") pod "global-pull-secret-syncer-qjcwx" (UID: "d9306059-36e0-4eae-8dc8-3ce075dc9b21") : object "kube-system"/"original-pull-secret" not registered
Sep 15 10:18:07.888217 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:07.888056    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88d49ac3-3a21-43ee-b74b-dfbd6239292e-kube-api-access-dqtl9 podName:88d49ac3-3a21-43ee-b74b-dfbd6239292e nodeName:}" failed. No retries permitted until 2026-09-15 10:18:08.888036342 +0000 UTC m=+4.135003304 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-dqtl9" (UniqueName: "kubernetes.io/projected/88d49ac3-3a21-43ee-b74b-dfbd6239292e-kube-api-access-dqtl9") pod "network-check-target-rxb8p" (UID: "88d49ac3-3a21-43ee-b74b-dfbd6239292e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 10:18:08.208240 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:08.208193    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-14 10:13:08 +0000 UTC" deadline="2028-04-29 14:52:50.644855484 +0000 UTC"
Sep 15 10:18:08.208240 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:08.208238    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14212h34m42.436623671s"
Sep 15 10:18:08.293804 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:08.293721    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:08.293976 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:08.293858    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjcwx" podUID="d9306059-36e0-4eae-8dc8-3ce075dc9b21"
Sep 15 10:18:08.308091 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:08.308004    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vf2xj" event={"ID":"11f3ddfa-e976-47c6-abab-60db632f8e64","Type":"ContainerStarted","Data":"a949b22a5724a5385e69c4ce1f639682246dba79269ef7b9448149c8e49eda37"}
Sep 15 10:18:08.310244 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:08.310134    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4sn5" event={"ID":"36de6650-259d-4ed2-9f48-43caefc90a18","Type":"ContainerStarted","Data":"dc70cfd8937d084a2774c7c2109c41efaba17c68cbc61c9e77ba4be18bbb7c63"}
Sep 15 10:18:08.313000 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:08.312968    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf" event={"ID":"b05ac5f2-3cd6-48f4-857a-08403194adbe","Type":"ContainerStarted","Data":"aa7d08d3d95cd89cf634ec00de665de86a7110f2838f5e05e9a97fc14a60b88e"}
Sep 15 10:18:08.328007 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:08.327924    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-n5sdr" event={"ID":"572e0352-78c0-4269-a50f-4d6e86d06f46","Type":"ContainerStarted","Data":"03eb77434d786a36ac39f12ba3b5220cdbe583ba5462cd5e571cd0258139fa76"}
Sep 15 10:18:08.330535 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:08.330429    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5gw48" event={"ID":"91c14855-f9ab-4882-8185-8289a2bd8269","Type":"ContainerStarted","Data":"93ecf85185abe166b61a9e816e1e03392f0f00edcaadf7e7559e4ae47b1688a9"}
Sep 15 10:18:08.332774 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:08.332674    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-58dk4" event={"ID":"b64b5ead-1ece-4aff-8975-5c59302a42d6","Type":"ContainerStarted","Data":"d79a1df7b26a704962eed6db3834d155e3e085d847a373343721503c184ba688"}
Sep 15 10:18:08.340899 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:08.340868    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mw292" event={"ID":"88f8318b-0faa-4708-afc6-de7f7ae3cee3","Type":"ContainerStarted","Data":"86efd3ea4077d010edf89ed6ee58303fcaf9a802956032cc386b024e93bdb165"}
Sep 15 10:18:08.346763 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:08.346737    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8hkxj" event={"ID":"17509e5d-d9dc-4f53-a111-4929185b0a3c","Type":"ContainerStarted","Data":"1e739657176815fe657e8dcb3d2d2b994fc22ca2de4d21851450f9592f2e311b"}
Sep 15 10:18:08.349910 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:08.349881    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-w2qnt" event={"ID":"c512fef3-9326-49ca-a58e-4fa39b7d9814","Type":"ContainerStarted","Data":"be1279ddf919287ad7dd573dfc875514face579aa6a4235443b0df872bab561f"}
Sep 15 10:18:08.897840 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:08.897738    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5d3dc08-8291-4828-87d1-f0da01fd24c8-metrics-certs\") pod \"network-metrics-daemon-gsxvv\" (UID: \"c5d3dc08-8291-4828-87d1-f0da01fd24c8\") " pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:08.897840 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:08.897810    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d9306059-36e0-4eae-8dc8-3ce075dc9b21-original-pull-secret\") pod \"global-pull-secret-syncer-qjcwx\" (UID: \"d9306059-36e0-4eae-8dc8-3ce075dc9b21\") " pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:08.898169 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:08.897851    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqtl9\" (UniqueName: \"kubernetes.io/projected/88d49ac3-3a21-43ee-b74b-dfbd6239292e-kube-api-access-dqtl9\") pod \"network-check-target-rxb8p\" (UID: \"88d49ac3-3a21-43ee-b74b-dfbd6239292e\") " pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:18:08.898169 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:08.898030    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 10:18:08.898169 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:08.898053    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 10:18:08.898169 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:08.898067    2583 projected.go:196] Error preparing data for projected volume kube-api-access-dqtl9 for pod openshift-network-diagnostics/network-check-target-rxb8p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 10:18:08.898169 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:08.898133    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88d49ac3-3a21-43ee-b74b-dfbd6239292e-kube-api-access-dqtl9 podName:88d49ac3-3a21-43ee-b74b-dfbd6239292e nodeName:}" failed. No retries permitted until 2026-09-15 10:18:10.898111273 +0000 UTC m=+6.145078232 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-dqtl9" (UniqueName: "kubernetes.io/projected/88d49ac3-3a21-43ee-b74b-dfbd6239292e-kube-api-access-dqtl9") pod "network-check-target-rxb8p" (UID: "88d49ac3-3a21-43ee-b74b-dfbd6239292e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 10:18:08.898455 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:08.898205    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 10:18:08.898455 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:08.898241    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5d3dc08-8291-4828-87d1-f0da01fd24c8-metrics-certs podName:c5d3dc08-8291-4828-87d1-f0da01fd24c8 nodeName:}" failed. No retries permitted until 2026-09-15 10:18:10.898227987 +0000 UTC m=+6.145194953 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5d3dc08-8291-4828-87d1-f0da01fd24c8-metrics-certs") pod "network-metrics-daemon-gsxvv" (UID: "c5d3dc08-8291-4828-87d1-f0da01fd24c8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 10:18:08.898455 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:08.898299    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 10:18:08.898455 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:08.898335    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9306059-36e0-4eae-8dc8-3ce075dc9b21-original-pull-secret podName:d9306059-36e0-4eae-8dc8-3ce075dc9b21 nodeName:}" failed. No retries permitted until 2026-09-15 10:18:10.898323503 +0000 UTC m=+6.145290456 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d9306059-36e0-4eae-8dc8-3ce075dc9b21-original-pull-secret") pod "global-pull-secret-syncer-qjcwx" (UID: "d9306059-36e0-4eae-8dc8-3ce075dc9b21") : object "kube-system"/"original-pull-secret" not registered
Sep 15 10:18:09.209470 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:09.209277    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-14 10:13:08 +0000 UTC" deadline="2028-02-17 00:23:57.183825492 +0000 UTC"
Sep 15 10:18:09.209470 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:09.209345    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12470h5m47.974485304s"
Sep 15 10:18:09.296680 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:09.296647    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:09.296858 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:09.296800    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gsxvv" podUID="c5d3dc08-8291-4828-87d1-f0da01fd24c8"
Sep 15 10:18:09.297447 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:09.297215    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:18:09.297447 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:09.297313    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxb8p" podUID="88d49ac3-3a21-43ee-b74b-dfbd6239292e"
Sep 15 10:18:09.366575 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:09.365820    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-235.ec2.internal" event={"ID":"93a9cd305dd93bbe9f79180cbc0e6eaa","Type":"ContainerStarted","Data":"441d01a445d6d53789408ae61ca5249d467daf9bf3a84d24df07bd978bdad3b3"}
Sep 15 10:18:10.294469 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:10.294380    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:10.294958 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:10.294551    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjcwx" podUID="d9306059-36e0-4eae-8dc8-3ce075dc9b21"
Sep 15 10:18:10.913445 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:10.913406    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5d3dc08-8291-4828-87d1-f0da01fd24c8-metrics-certs\") pod \"network-metrics-daemon-gsxvv\" (UID: \"c5d3dc08-8291-4828-87d1-f0da01fd24c8\") " pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:10.913667 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:10.913482    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d9306059-36e0-4eae-8dc8-3ce075dc9b21-original-pull-secret\") pod \"global-pull-secret-syncer-qjcwx\" (UID: \"d9306059-36e0-4eae-8dc8-3ce075dc9b21\") " pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:10.913667 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:10.913526    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqtl9\" (UniqueName: \"kubernetes.io/projected/88d49ac3-3a21-43ee-b74b-dfbd6239292e-kube-api-access-dqtl9\") pod \"network-check-target-rxb8p\" (UID: \"88d49ac3-3a21-43ee-b74b-dfbd6239292e\") " pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:18:10.913667 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:10.913649    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 10:18:10.913667 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:10.913653    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 10:18:10.913896 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:10.913660    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 10:18:10.913896 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:10.913742    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 10:18:10.913896 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:10.913760    2583 projected.go:196] Error preparing data for projected volume kube-api-access-dqtl9 for pod openshift-network-diagnostics/network-check-target-rxb8p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 10:18:10.913896 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:10.913719    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9306059-36e0-4eae-8dc8-3ce075dc9b21-original-pull-secret podName:d9306059-36e0-4eae-8dc8-3ce075dc9b21 nodeName:}" failed. No retries permitted until 2026-09-15 10:18:14.913699408 +0000 UTC m=+10.160666360 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d9306059-36e0-4eae-8dc8-3ce075dc9b21-original-pull-secret") pod "global-pull-secret-syncer-qjcwx" (UID: "d9306059-36e0-4eae-8dc8-3ce075dc9b21") : object "kube-system"/"original-pull-secret" not registered
Sep 15 10:18:10.913896 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:10.913820    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5d3dc08-8291-4828-87d1-f0da01fd24c8-metrics-certs podName:c5d3dc08-8291-4828-87d1-f0da01fd24c8 nodeName:}" failed. No retries permitted until 2026-09-15 10:18:14.913801244 +0000 UTC m=+10.160768200 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5d3dc08-8291-4828-87d1-f0da01fd24c8-metrics-certs") pod "network-metrics-daemon-gsxvv" (UID: "c5d3dc08-8291-4828-87d1-f0da01fd24c8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 10:18:10.913896 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:10.913837    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88d49ac3-3a21-43ee-b74b-dfbd6239292e-kube-api-access-dqtl9 podName:88d49ac3-3a21-43ee-b74b-dfbd6239292e nodeName:}" failed. No retries permitted until 2026-09-15 10:18:14.913826196 +0000 UTC m=+10.160793149 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-dqtl9" (UniqueName: "kubernetes.io/projected/88d49ac3-3a21-43ee-b74b-dfbd6239292e-kube-api-access-dqtl9") pod "network-check-target-rxb8p" (UID: "88d49ac3-3a21-43ee-b74b-dfbd6239292e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 10:18:11.294382 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:11.294296    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:11.294567 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:11.294452    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gsxvv" podUID="c5d3dc08-8291-4828-87d1-f0da01fd24c8"
Sep 15 10:18:11.294945 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:11.294866    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:18:11.295023 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:11.294971    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxb8p" podUID="88d49ac3-3a21-43ee-b74b-dfbd6239292e"
Sep 15 10:18:12.294676 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:12.294124    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:12.294676 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:12.294264    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjcwx" podUID="d9306059-36e0-4eae-8dc8-3ce075dc9b21"
Sep 15 10:18:13.298242 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:13.298208    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:13.298747 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:13.298351    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gsxvv" podUID="c5d3dc08-8291-4828-87d1-f0da01fd24c8"
Sep 15 10:18:13.298843 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:13.298757    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:18:13.298900 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:13.298851    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxb8p" podUID="88d49ac3-3a21-43ee-b74b-dfbd6239292e"
Sep 15 10:18:14.294291 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:14.294254    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:14.294512 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:14.294382    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjcwx" podUID="d9306059-36e0-4eae-8dc8-3ce075dc9b21"
Sep 15 10:18:14.946194 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:14.946146    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5d3dc08-8291-4828-87d1-f0da01fd24c8-metrics-certs\") pod \"network-metrics-daemon-gsxvv\" (UID: \"c5d3dc08-8291-4828-87d1-f0da01fd24c8\") " pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:14.946701 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:14.946224    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d9306059-36e0-4eae-8dc8-3ce075dc9b21-original-pull-secret\") pod \"global-pull-secret-syncer-qjcwx\" (UID: \"d9306059-36e0-4eae-8dc8-3ce075dc9b21\") " pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:14.946701 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:14.946270    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqtl9\" (UniqueName: \"kubernetes.io/projected/88d49ac3-3a21-43ee-b74b-dfbd6239292e-kube-api-access-dqtl9\") pod \"network-check-target-rxb8p\" (UID: \"88d49ac3-3a21-43ee-b74b-dfbd6239292e\") " pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:18:14.946701 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:14.946358    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 10:18:14.946701 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:14.946433    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 10:18:14.946701 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:14.946458    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 10:18:14.946701 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:14.946472    2583 projected.go:196] Error preparing data for projected volume kube-api-access-dqtl9 for pod openshift-network-diagnostics/network-check-target-rxb8p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 10:18:14.946701 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:14.946607    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 10:18:14.946701 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:14.946441    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5d3dc08-8291-4828-87d1-f0da01fd24c8-metrics-certs podName:c5d3dc08-8291-4828-87d1-f0da01fd24c8 nodeName:}" failed. No retries permitted until 2026-09-15 10:18:22.946419294 +0000 UTC m=+18.193386259 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5d3dc08-8291-4828-87d1-f0da01fd24c8-metrics-certs") pod "network-metrics-daemon-gsxvv" (UID: "c5d3dc08-8291-4828-87d1-f0da01fd24c8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 10:18:14.946701 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:14.946664    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88d49ac3-3a21-43ee-b74b-dfbd6239292e-kube-api-access-dqtl9 podName:88d49ac3-3a21-43ee-b74b-dfbd6239292e nodeName:}" failed. No retries permitted until 2026-09-15 10:18:22.946647118 +0000 UTC m=+18.193614074 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-dqtl9" (UniqueName: "kubernetes.io/projected/88d49ac3-3a21-43ee-b74b-dfbd6239292e-kube-api-access-dqtl9") pod "network-check-target-rxb8p" (UID: "88d49ac3-3a21-43ee-b74b-dfbd6239292e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 10:18:14.946701 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:14.946680    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9306059-36e0-4eae-8dc8-3ce075dc9b21-original-pull-secret podName:d9306059-36e0-4eae-8dc8-3ce075dc9b21 nodeName:}" failed. No retries permitted until 2026-09-15 10:18:22.946670716 +0000 UTC m=+18.193637665 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d9306059-36e0-4eae-8dc8-3ce075dc9b21-original-pull-secret") pod "global-pull-secret-syncer-qjcwx" (UID: "d9306059-36e0-4eae-8dc8-3ce075dc9b21") : object "kube-system"/"original-pull-secret" not registered
Sep 15 10:18:15.295159 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:15.294547    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:15.295159 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:15.294676    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gsxvv" podUID="c5d3dc08-8291-4828-87d1-f0da01fd24c8"
Sep 15 10:18:15.295159 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:15.294976    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:18:15.295159 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:15.295068    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxb8p" podUID="88d49ac3-3a21-43ee-b74b-dfbd6239292e"
Sep 15 10:18:16.294351 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:16.294319    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:16.294870 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:16.294472    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjcwx" podUID="d9306059-36e0-4eae-8dc8-3ce075dc9b21"
Sep 15 10:18:17.293682 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:17.293647    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:18:17.293884 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:17.293793    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxb8p" podUID="88d49ac3-3a21-43ee-b74b-dfbd6239292e"
Sep 15 10:18:17.294222 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:17.294195    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:17.294333 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:17.294315    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gsxvv" podUID="c5d3dc08-8291-4828-87d1-f0da01fd24c8"
Sep 15 10:18:17.389515 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:17.389441    2583 generic.go:356] "Generic (PLEG): container finished" podID="93a9cd305dd93bbe9f79180cbc0e6eaa" containerID="441d01a445d6d53789408ae61ca5249d467daf9bf3a84d24df07bd978bdad3b3" exitCode=0
Sep 15 10:18:17.389515 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:17.389489    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-235.ec2.internal" event={"ID":"93a9cd305dd93bbe9f79180cbc0e6eaa","Type":"ContainerDied","Data":"441d01a445d6d53789408ae61ca5249d467daf9bf3a84d24df07bd978bdad3b3"}
Sep 15 10:18:18.294126 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:18.293864    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:18.294314 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:18.294236    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjcwx" podUID="d9306059-36e0-4eae-8dc8-3ce075dc9b21"
Sep 15 10:18:19.294506 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:19.294469    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:19.294864 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:19.294513    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:18:19.294864 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:19.294614    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gsxvv" podUID="c5d3dc08-8291-4828-87d1-f0da01fd24c8"
Sep 15 10:18:19.294864 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:19.294724    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxb8p" podUID="88d49ac3-3a21-43ee-b74b-dfbd6239292e"
Sep 15 10:18:20.294157 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:20.294126    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:20.294318 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:20.294251    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjcwx" podUID="d9306059-36e0-4eae-8dc8-3ce075dc9b21"
Sep 15 10:18:21.297349 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:21.297311    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:21.297810 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:21.297467    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gsxvv" podUID="c5d3dc08-8291-4828-87d1-f0da01fd24c8"
Sep 15 10:18:21.297810 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:21.297504    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:18:21.297810 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:21.297606    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxb8p" podUID="88d49ac3-3a21-43ee-b74b-dfbd6239292e"
Sep 15 10:18:22.293915 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:22.293870    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:22.294066 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:22.293982    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjcwx" podUID="d9306059-36e0-4eae-8dc8-3ce075dc9b21"
Sep 15 10:18:23.011900 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:23.011827    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5d3dc08-8291-4828-87d1-f0da01fd24c8-metrics-certs\") pod \"network-metrics-daemon-gsxvv\" (UID: \"c5d3dc08-8291-4828-87d1-f0da01fd24c8\") " pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:23.011900 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:23.011869    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d9306059-36e0-4eae-8dc8-3ce075dc9b21-original-pull-secret\") pod \"global-pull-secret-syncer-qjcwx\" (UID: \"d9306059-36e0-4eae-8dc8-3ce075dc9b21\") " pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:23.012308 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:23.011903    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqtl9\" (UniqueName: \"kubernetes.io/projected/88d49ac3-3a21-43ee-b74b-dfbd6239292e-kube-api-access-dqtl9\") pod \"network-check-target-rxb8p\" (UID: \"88d49ac3-3a21-43ee-b74b-dfbd6239292e\") " pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:18:23.012308 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:23.011994    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 10:18:23.012308 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:23.012024    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 10:18:23.012308 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:23.012041    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 10:18:23.012308 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:23.012053    2583 projected.go:196] Error preparing data for projected volume kube-api-access-dqtl9 for pod openshift-network-diagnostics/network-check-target-rxb8p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 10:18:23.012308 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:23.012071    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5d3dc08-8291-4828-87d1-f0da01fd24c8-metrics-certs podName:c5d3dc08-8291-4828-87d1-f0da01fd24c8 nodeName:}" failed. No retries permitted until 2026-09-15 10:18:39.012050752 +0000 UTC m=+34.259017714 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5d3dc08-8291-4828-87d1-f0da01fd24c8-metrics-certs") pod "network-metrics-daemon-gsxvv" (UID: "c5d3dc08-8291-4828-87d1-f0da01fd24c8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 10:18:23.012308 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:23.011995    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 10:18:23.012308 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:23.012087    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88d49ac3-3a21-43ee-b74b-dfbd6239292e-kube-api-access-dqtl9 podName:88d49ac3-3a21-43ee-b74b-dfbd6239292e nodeName:}" failed. No retries permitted until 2026-09-15 10:18:39.012077158 +0000 UTC m=+34.259044105 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-dqtl9" (UniqueName: "kubernetes.io/projected/88d49ac3-3a21-43ee-b74b-dfbd6239292e-kube-api-access-dqtl9") pod "network-check-target-rxb8p" (UID: "88d49ac3-3a21-43ee-b74b-dfbd6239292e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 10:18:23.012308 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:23.012167    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9306059-36e0-4eae-8dc8-3ce075dc9b21-original-pull-secret podName:d9306059-36e0-4eae-8dc8-3ce075dc9b21 nodeName:}" failed. No retries permitted until 2026-09-15 10:18:39.012143294 +0000 UTC m=+34.259110245 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d9306059-36e0-4eae-8dc8-3ce075dc9b21-original-pull-secret") pod "global-pull-secret-syncer-qjcwx" (UID: "d9306059-36e0-4eae-8dc8-3ce075dc9b21") : object "kube-system"/"original-pull-secret" not registered
Sep 15 10:18:23.297190 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:23.297113    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:18:23.297190 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:23.297153    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:23.297412 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:23.297245    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxb8p" podUID="88d49ac3-3a21-43ee-b74b-dfbd6239292e"
Sep 15 10:18:23.297485 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:23.297395    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gsxvv" podUID="c5d3dc08-8291-4828-87d1-f0da01fd24c8"
Sep 15 10:18:24.293796 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:24.293766    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:24.294206 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:24.293891    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjcwx" podUID="d9306059-36e0-4eae-8dc8-3ce075dc9b21"
Sep 15 10:18:25.294700 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:25.294675    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:18:25.295143 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:25.294783    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxb8p" podUID="88d49ac3-3a21-43ee-b74b-dfbd6239292e"
Sep 15 10:18:25.295143 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:25.294833    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:25.295143 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:25.294903    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gsxvv" podUID="c5d3dc08-8291-4828-87d1-f0da01fd24c8"
Sep 15 10:18:26.294076 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:26.294040    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:26.294279 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:26.294163    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjcwx" podUID="d9306059-36e0-4eae-8dc8-3ce075dc9b21"
Sep 15 10:18:27.294044 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:27.293871    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:18:27.294710 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:27.293936    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:27.294710 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:27.294119    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxb8p" podUID="88d49ac3-3a21-43ee-b74b-dfbd6239292e"
Sep 15 10:18:27.294710 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:27.294187    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gsxvv" podUID="c5d3dc08-8291-4828-87d1-f0da01fd24c8"
Sep 15 10:18:27.413690 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:27.413589    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vf2xj" event={"ID":"11f3ddfa-e976-47c6-abab-60db632f8e64","Type":"ContainerStarted","Data":"bd41f060044c3ff8941d10fd4f573276111cacb901a2443ab8b82c3a8d09706b"}
Sep 15 10:18:27.415197 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:27.415168    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4sn5" event={"ID":"36de6650-259d-4ed2-9f48-43caefc90a18","Type":"ContainerStarted","Data":"d48cf9ad12da822400d06c1fe00ca46e2ee0185cde18e5b91847596773ffdf2e"}
Sep 15 10:18:27.418709 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:27.418687    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf" event={"ID":"b05ac5f2-3cd6-48f4-857a-08403194adbe","Type":"ContainerStarted","Data":"1b6d9ef76ed772fd0c349978f0fe577d38def41f976af4f6182a6a679908217e"}
Sep 15 10:18:27.418840 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:27.418739    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf" event={"ID":"b05ac5f2-3cd6-48f4-857a-08403194adbe","Type":"ContainerStarted","Data":"e7d598c68fd2602caa83a7edbc11f22ec0fbacdee1b6fef0a3b366e1e55becf1"}
Sep 15 10:18:27.418840 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:27.418785    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf" event={"ID":"b05ac5f2-3cd6-48f4-857a-08403194adbe","Type":"ContainerStarted","Data":"0be4fada9bbc4828975c959a2129f4093e0dbd949809a8fb320456d3f19c0901"}
Sep 15 10:18:27.418957 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:27.418842    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf" event={"ID":"b05ac5f2-3cd6-48f4-857a-08403194adbe","Type":"ContainerStarted","Data":"c4e2b021d0e1b418234a9bdab6532f342327127ebbf8a3b9c67d3504fb30783c"}
Sep 15 10:18:27.418957 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:27.418887    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf" event={"ID":"b05ac5f2-3cd6-48f4-857a-08403194adbe","Type":"ContainerStarted","Data":"9c3f590baf7a269e0eda9eda0e1720c0c2bea030ae09d346f3415cb68738d669"}
Sep 15 10:18:27.418957 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:27.418932    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf" event={"ID":"b05ac5f2-3cd6-48f4-857a-08403194adbe","Type":"ContainerStarted","Data":"e1977859ca144deffe23d181bf0e6c38aaddd11ebca439f1e5801c088db78d65"}
Sep 15 10:18:27.420489 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:27.420463    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-n5sdr" event={"ID":"572e0352-78c0-4269-a50f-4d6e86d06f46","Type":"ContainerStarted","Data":"0b0e6c1fa33eca1f0272d07efa83f26e0c399215630212cede3e2f3198a9acda"}
Sep 15 10:18:27.421935 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:27.421908    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5gw48" event={"ID":"91c14855-f9ab-4882-8185-8289a2bd8269","Type":"ContainerStarted","Data":"e5038d01b8df5628f54565ed2757a80f712bb366280376de16a3f347d4a33a31"}
Sep 15 10:18:27.423540 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:27.423518    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-235.ec2.internal" event={"ID":"2b3d6833f74b07abf73dd136dadede80","Type":"ContainerStarted","Data":"a8b5e84cd1d3a21f48dc3d413566fd775f0b220923875252a2a6b81995dab96a"}
Sep 15 10:18:27.425694 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:27.425673    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-235.ec2.internal" event={"ID":"93a9cd305dd93bbe9f79180cbc0e6eaa","Type":"ContainerStarted","Data":"f4bbfbbb2c38f0d47799ac7907d80cacce7b3de1647008dac11fceb8be054678"}
Sep 15 10:18:27.427881 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:27.427858    2583 generic.go:356] "Generic (PLEG): container finished" podID="b64b5ead-1ece-4aff-8975-5c59302a42d6" containerID="9523b04edc2dba5b83b91be1eeae44d11bdea2ad9d17b8eed4f4c914f2d800cf" exitCode=0
Sep 15 10:18:27.427999 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:27.427925    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-58dk4" event={"ID":"b64b5ead-1ece-4aff-8975-5c59302a42d6","Type":"ContainerDied","Data":"9523b04edc2dba5b83b91be1eeae44d11bdea2ad9d17b8eed4f4c914f2d800cf"}
Sep 15 10:18:27.431309 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:27.431287    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mw292" event={"ID":"88f8318b-0faa-4708-afc6-de7f7ae3cee3","Type":"ContainerStarted","Data":"8fe0f465ea42dd0ef1f6d31e23117b2e4d4606bb322c815ff6d4175aabd87890"}
Sep 15 10:18:27.432793 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:27.432766    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8hkxj" event={"ID":"17509e5d-d9dc-4f53-a111-4929185b0a3c","Type":"ContainerStarted","Data":"450bcf56900ca8c201355d4a919e24525d08f66f6b615f24ba63962a74a87efb"}
Sep 15 10:18:27.442600 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:27.442554    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-vf2xj" podStartSLOduration=2.427826778 podStartE2EDuration="21.442528655s" totalImagesPullingTime="19.014701877s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 10:18:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:18:27.42928085 +0000 UTC m=+22.676247821" watchObservedRunningTime="2026-09-15 10:18:27.442528655 +0000 UTC m=+22.689495628"
Sep 15 10:18:27.442904 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:27.442880    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5gw48" podStartSLOduration=2.388943585 podStartE2EDuration="21.442874185s" totalImagesPullingTime="19.0539306s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 10:18:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:18:27.442208318 +0000 UTC m=+22.689175289" watchObservedRunningTime="2026-09-15 10:18:27.442874185 +0000 UTC m=+22.689841155"
Sep 15 10:18:27.471855 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:27.471822    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-235.ec2.internal" podStartSLOduration=20.471809208 podStartE2EDuration="20.471809208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 10:18:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:18:27.455512566 +0000 UTC m=+22.702479539" watchObservedRunningTime="2026-09-15 10:18:27.471809208 +0000 UTC m=+22.718776180"
Sep 15 10:18:27.491090 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:27.491055    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-235.ec2.internal" podStartSLOduration=20.491044705 podStartE2EDuration="20.491044705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 10:18:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:18:27.471755334 +0000 UTC m=+22.718722304" watchObservedRunningTime="2026-09-15 10:18:27.491044705 +0000 UTC m=+22.738011674"
Sep 15 10:18:27.508830 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:27.508790    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-n5sdr" podStartSLOduration=2.415763045 podStartE2EDuration="21.50877828s" totalImagesPullingTime="19.093015235s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 10:18:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:18:27.491000975 +0000 UTC m=+22.737967945" watchObservedRunningTime="2026-09-15 10:18:27.50877828 +0000 UTC m=+22.755745281"
Sep 15 10:18:27.522542 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:27.522498    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8hkxj" podStartSLOduration=6.348486611 podStartE2EDuration="21.522484524s" totalImagesPullingTime="15.173997913s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:18:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:18:27.521990055 +0000 UTC m=+22.768957026" watchObservedRunningTime="2026-09-15 10:18:27.522484524 +0000 UTC m=+22.769451495"
Sep 15 10:18:27.535204 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:27.535168    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mw292" podStartSLOduration=6.370700133 podStartE2EDuration="21.535157474s" totalImagesPullingTime="15.164457341s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 10:18:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:18:27.534891153 +0000 UTC m=+22.781858123" watchObservedRunningTime="2026-09-15 10:18:27.535157474 +0000 UTC m=+22.782124443"
Sep 15 10:18:27.631887 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:27.631866    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:27.732247 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:27.732151    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:28.001136 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:28.001116    2583 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 15 10:18:28.012837 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:28.012810    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:28.225055 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:28.224967    2583 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-15T10:18:28.001131432Z","UUID":"b6ab0aa7-49b8-4791-b060-1fc6889068bb","Handler":null,"Name":"","Endpoint":""}
Sep 15 10:18:28.226345 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:28.226323    2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 15 10:18:28.226489 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:28.226355    2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 15 10:18:28.294415 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:28.294331    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:28.294823 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:28.294463    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjcwx" podUID="d9306059-36e0-4eae-8dc8-3ce075dc9b21"
Sep 15 10:18:28.437089 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:28.437052    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4sn5" event={"ID":"36de6650-259d-4ed2-9f48-43caefc90a18","Type":"ContainerStarted","Data":"67b98e1b7b84215b9e7e92f6ea2cae500c8badd4b7db995dea5c20601e232b2c"}
Sep 15 10:18:28.438761 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:28.438727    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-w2qnt" event={"ID":"c512fef3-9326-49ca-a58e-4fa39b7d9814","Type":"ContainerStarted","Data":"bee35c905c63ca7f371cd029931e345e8ed76110352a10983c1e76668c2f861c"}
Sep 15 10:18:28.454457 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:28.454410    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-w2qnt" podStartSLOduration=3.408419503 podStartE2EDuration="22.454396802s" totalImagesPullingTime="19.045977299s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 10:18:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:18:28.453930338 +0000 UTC m=+23.700897308" watchObservedRunningTime="2026-09-15 10:18:28.454396802 +0000 UTC m=+23.701363772"
Sep 15 10:18:28.573690 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:28.573620    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:28.673720 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:28.673688    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:29.294093 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:29.294066    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:29.294245 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:29.294183    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gsxvv" podUID="c5d3dc08-8291-4828-87d1-f0da01fd24c8"
Sep 15 10:18:29.294346 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:29.294064    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:18:29.294516 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:29.294485    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxb8p" podUID="88d49ac3-3a21-43ee-b74b-dfbd6239292e"
Sep 15 10:18:29.443764 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:29.443732    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4sn5" event={"ID":"36de6650-259d-4ed2-9f48-43caefc90a18","Type":"ContainerStarted","Data":"b78b55e6032ed1a2f2dd729d9e7bfeac5a709c6f97b4fd4611dae02a08430f8a"}
Sep 15 10:18:29.447616 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:29.447581    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf" event={"ID":"b05ac5f2-3cd6-48f4-857a-08403194adbe","Type":"ContainerStarted","Data":"b54bd110fcfa41c8dbb13039a4459f7c7127e1147cf76f47b8955853d45fd297"}
Sep 15 10:18:29.464643 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:29.464603    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t4sn5" podStartSLOduration=2.296481841 podStartE2EDuration="23.464589191s" totalImagesPullingTime="21.16810735s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 10:18:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:18:29.463875539 +0000 UTC m=+24.710842510" watchObservedRunningTime="2026-09-15 10:18:29.464589191 +0000 UTC m=+24.711556161"
Sep 15 10:18:30.038184 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:30.038147    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:30.095508 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:30.095458    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:30.159731 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:30.159698    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8hkxj"
Sep 15 10:18:30.160703 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:30.160681    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8hkxj"
Sep 15 10:18:30.162380 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:30.162356    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:30.237647 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:30.237619    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:30.294540 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:30.294478    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:30.295021 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:30.294594    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjcwx" podUID="d9306059-36e0-4eae-8dc8-3ce075dc9b21"
Sep 15 10:18:30.296458 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:30.296428    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:30.744347 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:30.744320    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:30.818016 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:30.817978    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:31.294232 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:31.294201    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:18:31.294232 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:31.294223    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:31.294487 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:31.294324    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxb8p" podUID="88d49ac3-3a21-43ee-b74b-dfbd6239292e"
Sep 15 10:18:31.294554 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:31.294488    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gsxvv" podUID="c5d3dc08-8291-4828-87d1-f0da01fd24c8"
Sep 15 10:18:32.237329 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:32.237085    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:32.294309 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:32.294269    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:32.294430 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:32.294405    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjcwx" podUID="d9306059-36e0-4eae-8dc8-3ce075dc9b21"
Sep 15 10:18:32.338696 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:32.338660    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:32.444818 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:32.444783    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:32.457644 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:32.457609    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf" event={"ID":"b05ac5f2-3cd6-48f4-857a-08403194adbe","Type":"ContainerStarted","Data":"4ffb7f1f7b648077b695ede57f184ec99a4b82e4595ca81589c0db0a9918f3e6"}
Sep 15 10:18:32.457850 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:32.457829    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:32.459567 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:32.459545    2583 generic.go:356] "Generic (PLEG): container finished" podID="b64b5ead-1ece-4aff-8975-5c59302a42d6" containerID="81022bfc4f5c5aaeb464ed4e98f6f5346dc240aeeeec783f0567ee5ef685d894" exitCode=0
Sep 15 10:18:32.459645 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:32.459578    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-58dk4" event={"ID":"b64b5ead-1ece-4aff-8975-5c59302a42d6","Type":"ContainerDied","Data":"81022bfc4f5c5aaeb464ed4e98f6f5346dc240aeeeec783f0567ee5ef685d894"}
Sep 15 10:18:32.473261 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:32.473244    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:32.483965 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:32.483925    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf" podStartSLOduration=7.391128572 podStartE2EDuration="26.483913587s" totalImagesPullingTime="19.092785015s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 10:18:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:18:32.483816819 +0000 UTC m=+27.730783787" watchObservedRunningTime="2026-09-15 10:18:32.483913587 +0000 UTC m=+27.730880555"
Sep 15 10:18:33.294438 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:33.294403    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:18:33.294438 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:33.294434    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:33.294710 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:33.294548    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxb8p" podUID="88d49ac3-3a21-43ee-b74b-dfbd6239292e"
Sep 15 10:18:33.294710 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:33.294643    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gsxvv" podUID="c5d3dc08-8291-4828-87d1-f0da01fd24c8"
Sep 15 10:18:33.462456 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:33.462262    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:33.462863 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:33.462574    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:33.475832 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:33.475817    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:18:33.549130 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:33.549053    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:33.643506 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:33.643477    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qjcwx"]
Sep 15 10:18:33.643631 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:33.643584    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:33.643685 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:33.643667    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjcwx" podUID="d9306059-36e0-4eae-8dc8-3ce075dc9b21"
Sep 15 10:18:33.647074 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:33.647051    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gsxvv"]
Sep 15 10:18:33.647229 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:33.647165    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:33.647299 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:33.647255    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gsxvv" podUID="c5d3dc08-8291-4828-87d1-f0da01fd24c8"
Sep 15 10:18:33.647653 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:33.647634    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rxb8p"]
Sep 15 10:18:33.647742 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:33.647713    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:18:33.647801 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:33.647783    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxb8p" podUID="88d49ac3-3a21-43ee-b74b-dfbd6239292e"
Sep 15 10:18:33.849607 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:33.849532    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:34.002971 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:34.002937    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:34.182413 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:34.182385    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:34.278794 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:34.278762    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:34.445252 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:34.445179    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:34.465538 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:34.465511    2583 generic.go:356] "Generic (PLEG): container finished" podID="b64b5ead-1ece-4aff-8975-5c59302a42d6" containerID="2799b91c3ffd616985bf5ea70b111a603db90595b2a18c48fd9f7f1f65484a87" exitCode=0
Sep 15 10:18:34.465894 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:34.465593    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-58dk4" event={"ID":"b64b5ead-1ece-4aff-8975-5c59302a42d6","Type":"ContainerDied","Data":"2799b91c3ffd616985bf5ea70b111a603db90595b2a18c48fd9f7f1f65484a87"}
Sep 15 10:18:34.545033 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:34.545011    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:35.294601 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:35.294574    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:35.294712 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:35.294669    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:35.294712 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:35.294672    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjcwx" podUID="d9306059-36e0-4eae-8dc8-3ce075dc9b21"
Sep 15 10:18:35.294712 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:35.294697    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:18:35.294822 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:35.294768    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gsxvv" podUID="c5d3dc08-8291-4828-87d1-f0da01fd24c8"
Sep 15 10:18:35.294858 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:35.294835    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxb8p" podUID="88d49ac3-3a21-43ee-b74b-dfbd6239292e"
Sep 15 10:18:35.470492 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:35.470460    2583 generic.go:356] "Generic (PLEG): container finished" podID="b64b5ead-1ece-4aff-8975-5c59302a42d6" containerID="26beb72319bed918f42f6a1d71deadf5a6a6e4203e1e03dc9cbe9a0c0dc2ce73" exitCode=0
Sep 15 10:18:35.470841 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:35.470536    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-58dk4" event={"ID":"b64b5ead-1ece-4aff-8975-5c59302a42d6","Type":"ContainerDied","Data":"26beb72319bed918f42f6a1d71deadf5a6a6e4203e1e03dc9cbe9a0c0dc2ce73"}
Sep 15 10:18:35.748268 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:35.748197    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:35.945470 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:35.945424    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:36.144280 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:36.144248    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:36.409244 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:36.409155    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:36.450785 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:36.450746    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8hkxj"
Sep 15 10:18:36.450964 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:36.450892    2583 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 15 10:18:36.451485 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:36.451466    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8hkxj"
Sep 15 10:18:36.471711 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:36.471676    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:36.547199 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:36.547161    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:37.294336 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:37.294306    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:37.294549 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:37.294340    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:18:37.294549 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:37.294309    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:37.294549 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:37.294426    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gsxvv" podUID="c5d3dc08-8291-4828-87d1-f0da01fd24c8"
Sep 15 10:18:37.294715 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:37.294561    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjcwx" podUID="d9306059-36e0-4eae-8dc8-3ce075dc9b21"
Sep 15 10:18:37.294715 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:37.294634    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxb8p" podUID="88d49ac3-3a21-43ee-b74b-dfbd6239292e"
Sep 15 10:18:37.541555 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:37.541526    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:37.594621 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:37.592265    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:37.947454 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:37.947417    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:38.044797 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:38.044761    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:38.151471 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:38.151404    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:38.340775 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:38.340704    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:38.558672 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:38.558615    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:38.659944 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:38.659910    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:39.029773 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:39.029699    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqtl9\" (UniqueName: \"kubernetes.io/projected/88d49ac3-3a21-43ee-b74b-dfbd6239292e-kube-api-access-dqtl9\") pod \"network-check-target-rxb8p\" (UID: \"88d49ac3-3a21-43ee-b74b-dfbd6239292e\") " pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:18:39.029773 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:39.029753    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5d3dc08-8291-4828-87d1-f0da01fd24c8-metrics-certs\") pod \"network-metrics-daemon-gsxvv\" (UID: \"c5d3dc08-8291-4828-87d1-f0da01fd24c8\") " pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:39.029991 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:39.029788    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d9306059-36e0-4eae-8dc8-3ce075dc9b21-original-pull-secret\") pod \"global-pull-secret-syncer-qjcwx\" (UID: \"d9306059-36e0-4eae-8dc8-3ce075dc9b21\") " pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:39.029991 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:39.029870    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 15 10:18:39.029991 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:39.029923    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9306059-36e0-4eae-8dc8-3ce075dc9b21-original-pull-secret podName:d9306059-36e0-4eae-8dc8-3ce075dc9b21 nodeName:}" failed. No retries permitted until 2026-09-15 10:19:11.029909143 +0000 UTC m=+66.276876090 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d9306059-36e0-4eae-8dc8-3ce075dc9b21-original-pull-secret") pod "global-pull-secret-syncer-qjcwx" (UID: "d9306059-36e0-4eae-8dc8-3ce075dc9b21") : object "kube-system"/"original-pull-secret" not registered
Sep 15 10:18:39.029991 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:39.029921    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 10:18:39.029991 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:39.029874    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 15 10:18:39.029991 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:39.029952    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 15 10:18:39.029991 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:39.029963    2583 projected.go:196] Error preparing data for projected volume kube-api-access-dqtl9 for pod openshift-network-diagnostics/network-check-target-rxb8p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 10:18:39.029991 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:39.029979    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5d3dc08-8291-4828-87d1-f0da01fd24c8-metrics-certs podName:c5d3dc08-8291-4828-87d1-f0da01fd24c8 nodeName:}" failed. No retries permitted until 2026-09-15 10:19:11.02996207 +0000 UTC m=+66.276929041 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5d3dc08-8291-4828-87d1-f0da01fd24c8-metrics-certs") pod "network-metrics-daemon-gsxvv" (UID: "c5d3dc08-8291-4828-87d1-f0da01fd24c8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 15 10:18:39.030375 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:39.030000    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88d49ac3-3a21-43ee-b74b-dfbd6239292e-kube-api-access-dqtl9 podName:88d49ac3-3a21-43ee-b74b-dfbd6239292e nodeName:}" failed. No retries permitted until 2026-09-15 10:19:11.029989872 +0000 UTC m=+66.276956829 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-dqtl9" (UniqueName: "kubernetes.io/projected/88d49ac3-3a21-43ee-b74b-dfbd6239292e-kube-api-access-dqtl9") pod "network-check-target-rxb8p" (UID: "88d49ac3-3a21-43ee-b74b-dfbd6239292e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 15 10:18:39.294623 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:39.294547    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:39.294783 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:39.294547    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:39.294783 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:39.294690    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gsxvv" podUID="c5d3dc08-8291-4828-87d1-f0da01fd24c8"
Sep 15 10:18:39.294783 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:39.294715    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:18:39.294924 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:39.294799    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjcwx" podUID="d9306059-36e0-4eae-8dc8-3ce075dc9b21"
Sep 15 10:18:39.294924 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:39.294859    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxb8p" podUID="88d49ac3-3a21-43ee-b74b-dfbd6239292e"
Sep 15 10:18:39.542610 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:39.542577    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:39.745330 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:39.745289    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:39.867604 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:39.867575    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:39.968486 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:39.968458    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:40.241282 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:40.241243    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:40.345179 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:40.345120    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:40.446544 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:40.446518    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:40.485816 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:40.485791    2583 generic.go:356] "Generic (PLEG): container finished" podID="b64b5ead-1ece-4aff-8975-5c59302a42d6" containerID="509173203b4db1d5a5a7ac4d80db6d45610c1ba90790e7c242c7dfbd1d2e2b63" exitCode=0
Sep 15 10:18:40.485957 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:40.485845    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-58dk4" event={"ID":"b64b5ead-1ece-4aff-8975-5c59302a42d6","Type":"ContainerDied","Data":"509173203b4db1d5a5a7ac4d80db6d45610c1ba90790e7c242c7dfbd1d2e2b63"}
Sep 15 10:18:40.949985 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:40.949944    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:41.294156 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:41.294071    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:41.294156 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:41.294101    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:41.294411 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:41.294071    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:18:41.294411 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:41.294208    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gsxvv" podUID="c5d3dc08-8291-4828-87d1-f0da01fd24c8"
Sep 15 10:18:41.294411 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:41.294298    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxb8p" podUID="88d49ac3-3a21-43ee-b74b-dfbd6239292e"
Sep 15 10:18:41.294411 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:41.294356    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjcwx" podUID="d9306059-36e0-4eae-8dc8-3ce075dc9b21"
Sep 15 10:18:41.344350 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:41.344322    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:41.491133 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:41.491097    2583 generic.go:356] "Generic (PLEG): container finished" podID="b64b5ead-1ece-4aff-8975-5c59302a42d6" containerID="9e289933dc81aef05af34c1fbc1aea1e235d9cb4a20fd3f20bb04d466608bf7d" exitCode=0
Sep 15 10:18:41.491258 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:41.491146    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-58dk4" event={"ID":"b64b5ead-1ece-4aff-8975-5c59302a42d6","Type":"ContainerDied","Data":"9e289933dc81aef05af34c1fbc1aea1e235d9cb4a20fd3f20bb04d466608bf7d"}
Sep 15 10:18:41.551355 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:41.551252    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:42.146412 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:42.146382    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:42.496998 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:42.496919    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-58dk4" event={"ID":"b64b5ead-1ece-4aff-8975-5c59302a42d6","Type":"ContainerStarted","Data":"05a1e502c8a60d14ea97ab6cb7ff8414045af8fb75fc4dd0607237df98a1214f"}
Sep 15 10:18:42.520060 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:42.520017    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-58dk4" podStartSLOduration=10.833019109 podStartE2EDuration="36.520000798s" totalImagesPullingTime="25.686981689s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 10:18:06 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:18:42.51979044 +0000 UTC m=+37.766757407" watchObservedRunningTime="2026-09-15 10:18:42.520000798 +0000 UTC m=+37.766967768"
Sep 15 10:18:42.746335 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:42.746304    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:43.084072 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:43.084039    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:43.184672 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:43.184643    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:43.293918 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:43.293891    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:18:43.293918 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:43.293906    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:43.294082 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:43.293891    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:43.294082 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:43.293998    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxb8p" podUID="88d49ac3-3a21-43ee-b74b-dfbd6239292e"
Sep 15 10:18:43.294082 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:43.294062    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjcwx" podUID="d9306059-36e0-4eae-8dc8-3ce075dc9b21"
Sep 15 10:18:43.294176 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:43.294143    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gsxvv" podUID="c5d3dc08-8291-4828-87d1-f0da01fd24c8"
Sep 15 10:18:43.344131 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:43.344044    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:43.967294 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:43.967262    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:44.145615 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:44.145582    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:44.931722 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:44.931691    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:45.031088 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:45.031050    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:45.294671 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:45.294573    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:45.294820 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:45.294681    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gsxvv" podUID="c5d3dc08-8291-4828-87d1-f0da01fd24c8"
Sep 15 10:18:45.294820 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:45.294693    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:18:45.294820 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:45.294718    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:45.294820 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:45.294764    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxb8p" podUID="88d49ac3-3a21-43ee-b74b-dfbd6239292e"
Sep 15 10:18:45.295029 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:45.294836    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjcwx" podUID="d9306059-36e0-4eae-8dc8-3ce075dc9b21"
Sep 15 10:18:47.294351 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:47.294321    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:47.294844 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:47.294321    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:18:47.294844 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:47.294460    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gsxvv" podUID="c5d3dc08-8291-4828-87d1-f0da01fd24c8"
Sep 15 10:18:47.294844 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:47.294321    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:47.294844 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:47.294500    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxb8p" podUID="88d49ac3-3a21-43ee-b74b-dfbd6239292e"
Sep 15 10:18:47.294844 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:47.294577    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjcwx" podUID="d9306059-36e0-4eae-8dc8-3ce075dc9b21"
Sep 15 10:18:49.294640 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:49.294595    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:49.295152 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:49.294595    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:49.295152 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:49.294714    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjcwx" podUID="d9306059-36e0-4eae-8dc8-3ce075dc9b21"
Sep 15 10:18:49.295152 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:49.294726    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:18:49.295152 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:49.294803    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gsxvv" podUID="c5d3dc08-8291-4828-87d1-f0da01fd24c8"
Sep 15 10:18:49.295152 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:49.294862    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxb8p" podUID="88d49ac3-3a21-43ee-b74b-dfbd6239292e"
Sep 15 10:18:51.293681 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:51.293653    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:51.293681 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:51.293670    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:51.294032 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:51.293659    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:18:51.294032 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:51.293743    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjcwx" podUID="d9306059-36e0-4eae-8dc8-3ce075dc9b21"
Sep 15 10:18:51.294032 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:51.293835    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gsxvv" podUID="c5d3dc08-8291-4828-87d1-f0da01fd24c8"
Sep 15 10:18:51.294032 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:51.293880    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxb8p" podUID="88d49ac3-3a21-43ee-b74b-dfbd6239292e"
Sep 15 10:18:53.293912 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:53.293878    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:53.294355 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:53.293879    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:53.294355 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:53.293986    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:18:53.294355 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:53.294003    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gsxvv" podUID="c5d3dc08-8291-4828-87d1-f0da01fd24c8"
Sep 15 10:18:53.294355 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:53.294074    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjcwx" podUID="d9306059-36e0-4eae-8dc8-3ce075dc9b21"
Sep 15 10:18:53.294355 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:18:53.294127    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rxb8p" podUID="88d49ac3-3a21-43ee-b74b-dfbd6239292e"
Sep 15 10:18:53.543129 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:53.543093    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:53.661213 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:53.661179    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:53.765734 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:53.765691    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:53.856256 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:53.856230    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:53.958690 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:53.958615    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:54.215770 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.215699    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:54.281572 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.281543    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:54.520600 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.520533    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-235.ec2.internal" event="NodeReady"
Sep 15 10:18:54.520961 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.520645    2583 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 15 10:18:54.567397 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.567367    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-k2zv8"]
Sep 15 10:18:54.592636 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.592606    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-lr47g"]
Sep 15 10:18:54.592773 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.592723    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-k2zv8"
Sep 15 10:18:54.595013 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.594988    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 15 10:18:54.595013 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.594997    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-h7d4n\""
Sep 15 10:18:54.595204 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.595046    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 15 10:18:54.616668 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.616641    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-k2zv8"]
Sep 15 10:18:54.616793 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.616678    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lr47g"]
Sep 15 10:18:54.616793 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.616750    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lr47g"
Sep 15 10:18:54.619023 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.618990    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 15 10:18:54.619023 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.619007    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 15 10:18:54.619179 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.619109    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-4q5lf\""
Sep 15 10:18:54.619216 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.619207    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 15 10:18:54.642611 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.642587    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c724eb6e-3079-4433-993e-02a515bb2132-config-volume\") pod \"dns-default-k2zv8\" (UID: \"c724eb6e-3079-4433-993e-02a515bb2132\") " pod="openshift-dns/dns-default-k2zv8"
Sep 15 10:18:54.642761 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.642631    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jvpn\" (UniqueName: \"kubernetes.io/projected/c724eb6e-3079-4433-993e-02a515bb2132-kube-api-access-2jvpn\") pod \"dns-default-k2zv8\" (UID: \"c724eb6e-3079-4433-993e-02a515bb2132\") " pod="openshift-dns/dns-default-k2zv8"
Sep 15 10:18:54.642761 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.642653    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96zlf\" (UniqueName: \"kubernetes.io/projected/55758f39-d31a-4b0c-bb35-ef1f3be7d25b-kube-api-access-96zlf\") pod \"ingress-canary-lr47g\" (UID: \"55758f39-d31a-4b0c-bb35-ef1f3be7d25b\") " pod="openshift-ingress-canary/ingress-canary-lr47g"
Sep 15 10:18:54.642761 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.642710    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/55758f39-d31a-4b0c-bb35-ef1f3be7d25b-cert\") pod \"ingress-canary-lr47g\" (UID: \"55758f39-d31a-4b0c-bb35-ef1f3be7d25b\") " pod="openshift-ingress-canary/ingress-canary-lr47g"
Sep 15 10:18:54.642761 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.642738    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c724eb6e-3079-4433-993e-02a515bb2132-metrics-tls\") pod \"dns-default-k2zv8\" (UID: \"c724eb6e-3079-4433-993e-02a515bb2132\") " pod="openshift-dns/dns-default-k2zv8"
Sep 15 10:18:54.642761 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.642755    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c724eb6e-3079-4433-993e-02a515bb2132-tmp-dir\") pod \"dns-default-k2zv8\" (UID: \"c724eb6e-3079-4433-993e-02a515bb2132\") " pod="openshift-dns/dns-default-k2zv8"
Sep 15 10:18:54.735104 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.735073    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:54.743132 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.743107    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jvpn\" (UniqueName: \"kubernetes.io/projected/c724eb6e-3079-4433-993e-02a515bb2132-kube-api-access-2jvpn\") pod \"dns-default-k2zv8\" (UID: \"c724eb6e-3079-4433-993e-02a515bb2132\") " pod="openshift-dns/dns-default-k2zv8"
Sep 15 10:18:54.743232 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.743141    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96zlf\" (UniqueName: \"kubernetes.io/projected/55758f39-d31a-4b0c-bb35-ef1f3be7d25b-kube-api-access-96zlf\") pod \"ingress-canary-lr47g\" (UID: \"55758f39-d31a-4b0c-bb35-ef1f3be7d25b\") " pod="openshift-ingress-canary/ingress-canary-lr47g"
Sep 15 10:18:54.743232 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.743165    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/55758f39-d31a-4b0c-bb35-ef1f3be7d25b-cert\") pod \"ingress-canary-lr47g\" (UID: \"55758f39-d31a-4b0c-bb35-ef1f3be7d25b\") " pod="openshift-ingress-canary/ingress-canary-lr47g"
Sep 15 10:18:54.743232 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.743188    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c724eb6e-3079-4433-993e-02a515bb2132-metrics-tls\") pod \"dns-default-k2zv8\" (UID: \"c724eb6e-3079-4433-993e-02a515bb2132\") " pod="openshift-dns/dns-default-k2zv8"
Sep 15 10:18:54.743232 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.743210    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c724eb6e-3079-4433-993e-02a515bb2132-tmp-dir\") pod \"dns-default-k2zv8\" (UID: \"c724eb6e-3079-4433-993e-02a515bb2132\") " pod="openshift-dns/dns-default-k2zv8"
Sep 15 10:18:54.743436 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.743278    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c724eb6e-3079-4433-993e-02a515bb2132-config-volume\") pod \"dns-default-k2zv8\" (UID: \"c724eb6e-3079-4433-993e-02a515bb2132\") " pod="openshift-dns/dns-default-k2zv8"
Sep 15 10:18:54.743644 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.743626    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c724eb6e-3079-4433-993e-02a515bb2132-tmp-dir\") pod \"dns-default-k2zv8\" (UID: \"c724eb6e-3079-4433-993e-02a515bb2132\") " pod="openshift-dns/dns-default-k2zv8"
Sep 15 10:18:54.743893 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.743874    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c724eb6e-3079-4433-993e-02a515bb2132-config-volume\") pod \"dns-default-k2zv8\" (UID: \"c724eb6e-3079-4433-993e-02a515bb2132\") " pod="openshift-dns/dns-default-k2zv8"
Sep 15 10:18:54.747069 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.747046    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c724eb6e-3079-4433-993e-02a515bb2132-metrics-tls\") pod \"dns-default-k2zv8\" (UID: \"c724eb6e-3079-4433-993e-02a515bb2132\") " pod="openshift-dns/dns-default-k2zv8"
Sep 15 10:18:54.747160 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.747054    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/55758f39-d31a-4b0c-bb35-ef1f3be7d25b-cert\") pod \"ingress-canary-lr47g\" (UID: \"55758f39-d31a-4b0c-bb35-ef1f3be7d25b\") " pod="openshift-ingress-canary/ingress-canary-lr47g"
Sep 15 10:18:54.750258 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.750234    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96zlf\" (UniqueName: \"kubernetes.io/projected/55758f39-d31a-4b0c-bb35-ef1f3be7d25b-kube-api-access-96zlf\") pod \"ingress-canary-lr47g\" (UID: \"55758f39-d31a-4b0c-bb35-ef1f3be7d25b\") " pod="openshift-ingress-canary/ingress-canary-lr47g"
Sep 15 10:18:54.750498 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.750478    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jvpn\" (UniqueName: \"kubernetes.io/projected/c724eb6e-3079-4433-993e-02a515bb2132-kube-api-access-2jvpn\") pod \"dns-default-k2zv8\" (UID: \"c724eb6e-3079-4433-993e-02a515bb2132\") " pod="openshift-dns/dns-default-k2zv8"
Sep 15 10:18:54.838081 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.837980    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:54.901965 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.901932    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-k2zv8"
Sep 15 10:18:54.925432 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:54.925410    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lr47g"
Sep 15 10:18:55.222518 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:55.222359    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lr47g"]
Sep 15 10:18:55.225721 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:55.225700    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-k2zv8"]
Sep 15 10:18:55.227696 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:55.227674    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55758f39_d31a_4b0c_bb35_ef1f3be7d25b.slice/crio-7be55ad7a7f86145eb06a88d0966021cda05bd213598470995e0171e9311e06c WatchSource:0}: Error finding container 7be55ad7a7f86145eb06a88d0966021cda05bd213598470995e0171e9311e06c: Status 404 returned error can't find the container with id 7be55ad7a7f86145eb06a88d0966021cda05bd213598470995e0171e9311e06c
Sep 15 10:18:55.228238 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:18:55.228217    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc724eb6e_3079_4433_993e_02a515bb2132.slice/crio-b4c46f01c65ffae4e600e4faf3f37eea9558e675f10ff0bab696d697bd88f2a4 WatchSource:0}: Error finding container b4c46f01c65ffae4e600e4faf3f37eea9558e675f10ff0bab696d697bd88f2a4: Status 404 returned error can't find the container with id b4c46f01c65ffae4e600e4faf3f37eea9558e675f10ff0bab696d697bd88f2a4
Sep 15 10:18:55.295231 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:55.295196    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:18:55.295406 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:55.295299    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:18:55.295546 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:55.295523    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:18:55.298663 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:55.298632    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 15 10:18:55.298782 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:55.298702    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 15 10:18:55.298782 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:55.298714    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-88p68\""
Sep 15 10:18:55.298782 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:55.298710    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 15 10:18:55.298782 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:55.298752    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lrdhd\""
Sep 15 10:18:55.298990 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:55.298858    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 15 10:18:55.524211 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:55.524119    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-k2zv8" event={"ID":"c724eb6e-3079-4433-993e-02a515bb2132","Type":"ContainerStarted","Data":"b4c46f01c65ffae4e600e4faf3f37eea9558e675f10ff0bab696d697bd88f2a4"}
Sep 15 10:18:55.525206 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:55.525183    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lr47g" event={"ID":"55758f39-d31a-4b0c-bb35-ef1f3be7d25b","Type":"ContainerStarted","Data":"7be55ad7a7f86145eb06a88d0966021cda05bd213598470995e0171e9311e06c"}
Sep 15 10:18:56.702737 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:56.702700    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:56.803325 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:56.803287    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:56.903737 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:56.903700    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:56.958759 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:56.958693    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:18:58.537433 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:58.537393    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-k2zv8" event={"ID":"c724eb6e-3079-4433-993e-02a515bb2132","Type":"ContainerStarted","Data":"57cf4a3eca46f0a3ccd8325718fb08a8b16db25993c1afe64220929e7822f7df"}
Sep 15 10:18:58.538022 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:58.537514    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-k2zv8"
Sep 15 10:18:58.538022 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:58.537540    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-k2zv8" event={"ID":"c724eb6e-3079-4433-993e-02a515bb2132","Type":"ContainerStarted","Data":"53df60c6ca40f53ed3d17b209309571c08c25d42e194e93ec96b8557bd8b4579"}
Sep 15 10:18:58.538842 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:58.538820    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lr47g" event={"ID":"55758f39-d31a-4b0c-bb35-ef1f3be7d25b","Type":"ContainerStarted","Data":"2d27056dd84b2548c4e338425c93d5556b5462a40dc0eb2d4522d6306a005b93"}
Sep 15 10:18:58.556784 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:58.553861    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-k2zv8" podStartSLOduration=2.070164368 podStartE2EDuration="4.55350995s" totalImagesPullingTime="2.483345582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:18:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:18:58.552578832 +0000 UTC m=+53.799545799" watchObservedRunningTime="2026-09-15 10:18:58.55350995 +0000 UTC m=+53.800476915"
Sep 15 10:18:58.568101 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:18:58.568055    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-lr47g" podStartSLOduration=2.077219499 podStartE2EDuration="4.568039336s" totalImagesPullingTime="2.490819837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:18:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:18:58.567861097 +0000 UTC m=+53.814828066" watchObservedRunningTime="2026-09-15 10:18:58.568039336 +0000 UTC m=+53.815006307"
Sep 15 10:19:00.076309 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:00.076276    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:00.133182 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:00.133149    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:02.209975 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:02.209939    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:02.360183 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:02.360149    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:05.480890 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:05.480850    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pkmxf"
Sep 15 10:19:05.588733 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:05.588700    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:05.648559 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:05.648531    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:07.407578 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:07.407547    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:07.578057 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:07.578024    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-tqvwm"]
Sep 15 10:19:07.582144 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:07.582128    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-tqvwm"
Sep 15 10:19:07.584230 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:07.584208    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 15 10:19:07.584354 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:07.584289    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-dn69j\""
Sep 15 10:19:07.590075 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:07.590018    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-tqvwm"]
Sep 15 10:19:07.630622 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:07.630593    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a8ede461-d783-4166-a83f-22110b18dd42-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-tqvwm\" (UID: \"a8ede461-d783-4166-a83f-22110b18dd42\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-tqvwm"
Sep 15 10:19:07.678574 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:07.678505    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:07.731034 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:07.731004    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a8ede461-d783-4166-a83f-22110b18dd42-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-tqvwm\" (UID: \"a8ede461-d783-4166-a83f-22110b18dd42\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-tqvwm"
Sep 15 10:19:07.731148 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:19:07.731137    2583 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 15 10:19:07.731201 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:19:07.731191    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8ede461-d783-4166-a83f-22110b18dd42-tls-certificates podName:a8ede461-d783-4166-a83f-22110b18dd42 nodeName:}" failed. No retries permitted until 2026-09-15 10:19:08.231178118 +0000 UTC m=+63.478145071 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/a8ede461-d783-4166-a83f-22110b18dd42-tls-certificates") pod "prometheus-operator-admission-webhook-5864cd99d7-tqvwm" (UID: "a8ede461-d783-4166-a83f-22110b18dd42") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 15 10:19:07.748581 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:07.748551    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:07.953898 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:07.953825    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2cl9v"]
Sep 15 10:19:07.956729 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:07.956713    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2cl9v"
Sep 15 10:19:07.959140 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:07.959114    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 15 10:19:07.959268 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:07.959176    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 15 10:19:07.959268 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:07.959119    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 15 10:19:07.959268 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:07.959219    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 15 10:19:07.959436 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:07.959318    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fxj4z\""
Sep 15 10:19:07.968322 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:07.968303    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2cl9v"]
Sep 15 10:19:07.984765 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:07.984741    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:08.032733 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:08.032709    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6a655087-f02c-416c-87d2-7aba0811f5c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2cl9v\" (UID: \"6a655087-f02c-416c-87d2-7aba0811f5c0\") " pod="openshift-insights/insights-runtime-extractor-2cl9v"
Sep 15 10:19:08.032831 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:08.032758    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdplg\" (UniqueName: \"kubernetes.io/projected/6a655087-f02c-416c-87d2-7aba0811f5c0-kube-api-access-cdplg\") pod \"insights-runtime-extractor-2cl9v\" (UID: \"6a655087-f02c-416c-87d2-7aba0811f5c0\") " pod="openshift-insights/insights-runtime-extractor-2cl9v"
Sep 15 10:19:08.032831 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:08.032791    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a655087-f02c-416c-87d2-7aba0811f5c0-data-volume\") pod \"insights-runtime-extractor-2cl9v\" (UID: \"6a655087-f02c-416c-87d2-7aba0811f5c0\") " pod="openshift-insights/insights-runtime-extractor-2cl9v"
Sep 15 10:19:08.032831 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:08.032817    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6a655087-f02c-416c-87d2-7aba0811f5c0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2cl9v\" (UID: \"6a655087-f02c-416c-87d2-7aba0811f5c0\") " pod="openshift-insights/insights-runtime-extractor-2cl9v"
Sep 15 10:19:08.032955 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:08.032841    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6a655087-f02c-416c-87d2-7aba0811f5c0-crio-socket\") pod \"insights-runtime-extractor-2cl9v\" (UID: \"6a655087-f02c-416c-87d2-7aba0811f5c0\") " pod="openshift-insights/insights-runtime-extractor-2cl9v"
Sep 15 10:19:08.041713 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:08.041690    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:08.134028 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:08.134001    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a655087-f02c-416c-87d2-7aba0811f5c0-data-volume\") pod \"insights-runtime-extractor-2cl9v\" (UID: \"6a655087-f02c-416c-87d2-7aba0811f5c0\") " pod="openshift-insights/insights-runtime-extractor-2cl9v"
Sep 15 10:19:08.134153 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:08.134042    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6a655087-f02c-416c-87d2-7aba0811f5c0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2cl9v\" (UID: \"6a655087-f02c-416c-87d2-7aba0811f5c0\") " pod="openshift-insights/insights-runtime-extractor-2cl9v"
Sep 15 10:19:08.134153 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:08.134072    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6a655087-f02c-416c-87d2-7aba0811f5c0-crio-socket\") pod \"insights-runtime-extractor-2cl9v\" (UID: \"6a655087-f02c-416c-87d2-7aba0811f5c0\") " pod="openshift-insights/insights-runtime-extractor-2cl9v"
Sep 15 10:19:08.134153 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:08.134131    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6a655087-f02c-416c-87d2-7aba0811f5c0-crio-socket\") pod \"insights-runtime-extractor-2cl9v\" (UID: \"6a655087-f02c-416c-87d2-7aba0811f5c0\") " pod="openshift-insights/insights-runtime-extractor-2cl9v"
Sep 15 10:19:08.134153 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:19:08.134143    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 10:19:08.134343 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:08.134158    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6a655087-f02c-416c-87d2-7aba0811f5c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2cl9v\" (UID: \"6a655087-f02c-416c-87d2-7aba0811f5c0\") " pod="openshift-insights/insights-runtime-extractor-2cl9v"
Sep 15 10:19:08.134343 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:08.134197    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdplg\" (UniqueName: \"kubernetes.io/projected/6a655087-f02c-416c-87d2-7aba0811f5c0-kube-api-access-cdplg\") pod \"insights-runtime-extractor-2cl9v\" (UID: \"6a655087-f02c-416c-87d2-7aba0811f5c0\") " pod="openshift-insights/insights-runtime-extractor-2cl9v"
Sep 15 10:19:08.134343 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:19:08.134233    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6a655087-f02c-416c-87d2-7aba0811f5c0-kube-rbac-proxy-cm podName:6a655087-f02c-416c-87d2-7aba0811f5c0 nodeName:}" failed. No retries permitted until 2026-09-15 10:19:08.634210377 +0000 UTC m=+63.881177344 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6a655087-f02c-416c-87d2-7aba0811f5c0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2cl9v" (UID: "6a655087-f02c-416c-87d2-7aba0811f5c0") : configmap "kube-rbac-proxy" not found
Sep 15 10:19:08.134343 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:19:08.134304    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 10:19:08.134343 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:08.134332    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a655087-f02c-416c-87d2-7aba0811f5c0-data-volume\") pod \"insights-runtime-extractor-2cl9v\" (UID: \"6a655087-f02c-416c-87d2-7aba0811f5c0\") " pod="openshift-insights/insights-runtime-extractor-2cl9v"
Sep 15 10:19:08.134608 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:19:08.134360    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a655087-f02c-416c-87d2-7aba0811f5c0-insights-runtime-extractor-tls podName:6a655087-f02c-416c-87d2-7aba0811f5c0 nodeName:}" failed. No retries permitted until 2026-09-15 10:19:08.634345213 +0000 UTC m=+63.881312181 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6a655087-f02c-416c-87d2-7aba0811f5c0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2cl9v" (UID: "6a655087-f02c-416c-87d2-7aba0811f5c0") : secret "insights-runtime-extractor-tls" not found
Sep 15 10:19:08.142633 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:08.142614    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdplg\" (UniqueName: \"kubernetes.io/projected/6a655087-f02c-416c-87d2-7aba0811f5c0-kube-api-access-cdplg\") pod \"insights-runtime-extractor-2cl9v\" (UID: \"6a655087-f02c-416c-87d2-7aba0811f5c0\") " pod="openshift-insights/insights-runtime-extractor-2cl9v"
Sep 15 10:19:08.235608 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:08.235540    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a8ede461-d783-4166-a83f-22110b18dd42-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-tqvwm\" (UID: \"a8ede461-d783-4166-a83f-22110b18dd42\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-tqvwm"
Sep 15 10:19:08.237752 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:08.237732    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a8ede461-d783-4166-a83f-22110b18dd42-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-tqvwm\" (UID: \"a8ede461-d783-4166-a83f-22110b18dd42\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-tqvwm"
Sep 15 10:19:08.325075 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:08.325045    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:08.396769 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:08.396735    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:08.450562 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:08.450521    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:08.490981 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:08.490906    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-tqvwm"
Sep 15 10:19:08.543382 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:08.543360    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-k2zv8"
Sep 15 10:19:08.614128 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:08.614101    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:08.638378 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:08.638348    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6a655087-f02c-416c-87d2-7aba0811f5c0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2cl9v\" (UID: \"6a655087-f02c-416c-87d2-7aba0811f5c0\") " pod="openshift-insights/insights-runtime-extractor-2cl9v"
Sep 15 10:19:08.638519 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:19:08.638466    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 10:19:08.638566 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:19:08.638541    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6a655087-f02c-416c-87d2-7aba0811f5c0-kube-rbac-proxy-cm podName:6a655087-f02c-416c-87d2-7aba0811f5c0 nodeName:}" failed. No retries permitted until 2026-09-15 10:19:09.638518302 +0000 UTC m=+64.885485272 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6a655087-f02c-416c-87d2-7aba0811f5c0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2cl9v" (UID: "6a655087-f02c-416c-87d2-7aba0811f5c0") : configmap "kube-rbac-proxy" not found
Sep 15 10:19:08.638616 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:08.638570    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6a655087-f02c-416c-87d2-7aba0811f5c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2cl9v\" (UID: \"6a655087-f02c-416c-87d2-7aba0811f5c0\") " pod="openshift-insights/insights-runtime-extractor-2cl9v"
Sep 15 10:19:08.638693 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:19:08.638680    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 10:19:08.638740 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:19:08.638722    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a655087-f02c-416c-87d2-7aba0811f5c0-insights-runtime-extractor-tls podName:6a655087-f02c-416c-87d2-7aba0811f5c0 nodeName:}" failed. No retries permitted until 2026-09-15 10:19:09.638712262 +0000 UTC m=+64.885679214 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6a655087-f02c-416c-87d2-7aba0811f5c0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2cl9v" (UID: "6a655087-f02c-416c-87d2-7aba0811f5c0") : secret "insights-runtime-extractor-tls" not found
Sep 15 10:19:08.758093 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:08.758023    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-tqvwm"]
Sep 15 10:19:08.761040 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:19:08.761020    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8ede461_d783_4166_a83f_22110b18dd42.slice/crio-29f465d8ba36d9e46856cb262add12b347e64d61100b6eb375496f62d735534e WatchSource:0}: Error finding container 29f465d8ba36d9e46856cb262add12b347e64d61100b6eb375496f62d735534e: Status 404 returned error can't find the container with id 29f465d8ba36d9e46856cb262add12b347e64d61100b6eb375496f62d735534e
Sep 15 10:19:08.789124 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:08.789091    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:08.839772 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:08.839743    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:08.998910 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:08.998877    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:09.099983 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:09.099887    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:09.353216 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:09.353140    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-k2zv8_c724eb6e-3079-4433-993e-02a515bb2132/dns/0.log"
Sep 15 10:19:09.409875 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:09.409835    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:09.463057 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:09.463019    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:09.552823 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:09.552779    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-k2zv8_c724eb6e-3079-4433-993e-02a515bb2132/kube-rbac-proxy/0.log"
Sep 15 10:19:09.566633 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:09.566600    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-tqvwm" event={"ID":"a8ede461-d783-4166-a83f-22110b18dd42","Type":"ContainerStarted","Data":"29f465d8ba36d9e46856cb262add12b347e64d61100b6eb375496f62d735534e"}
Sep 15 10:19:09.645457 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:09.645413    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6a655087-f02c-416c-87d2-7aba0811f5c0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2cl9v\" (UID: \"6a655087-f02c-416c-87d2-7aba0811f5c0\") " pod="openshift-insights/insights-runtime-extractor-2cl9v"
Sep 15 10:19:09.645653 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:09.645490    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6a655087-f02c-416c-87d2-7aba0811f5c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2cl9v\" (UID: \"6a655087-f02c-416c-87d2-7aba0811f5c0\") " pod="openshift-insights/insights-runtime-extractor-2cl9v"
Sep 15 10:19:09.645653 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:19:09.645538    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 10:19:09.645653 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:19:09.645603    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6a655087-f02c-416c-87d2-7aba0811f5c0-kube-rbac-proxy-cm podName:6a655087-f02c-416c-87d2-7aba0811f5c0 nodeName:}" failed. No retries permitted until 2026-09-15 10:19:11.645583862 +0000 UTC m=+66.892550832 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6a655087-f02c-416c-87d2-7aba0811f5c0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2cl9v" (UID: "6a655087-f02c-416c-87d2-7aba0811f5c0") : configmap "kube-rbac-proxy" not found
Sep 15 10:19:09.645653 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:19:09.645646    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 10:19:09.645861 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:19:09.645715    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a655087-f02c-416c-87d2-7aba0811f5c0-insights-runtime-extractor-tls podName:6a655087-f02c-416c-87d2-7aba0811f5c0 nodeName:}" failed. No retries permitted until 2026-09-15 10:19:11.645697108 +0000 UTC m=+66.892664058 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6a655087-f02c-416c-87d2-7aba0811f5c0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2cl9v" (UID: "6a655087-f02c-416c-87d2-7aba0811f5c0") : secret "insights-runtime-extractor-tls" not found
Sep 15 10:19:09.753199 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:09.753172    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mw292_88f8318b-0faa-4708-afc6-de7f7ae3cee3/dns-node-resolver/0.log"
Sep 15 10:19:10.003364 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:10.003285    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:10.153418 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:10.153378    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:10.254132 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:10.254041    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:10.354137 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:10.354110    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:10.455716 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:10.455693    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:10.558340 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:10.558263    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:10.570227 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:10.570199    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-tqvwm" event={"ID":"a8ede461-d783-4166-a83f-22110b18dd42","Type":"ContainerStarted","Data":"232117a8c6a9ade5c3d82f3cba4fee22bfbf873956aa7bc43984ee7a99fa3e8c"}
Sep 15 10:19:10.570499 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:10.570480    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-tqvwm"
Sep 15 10:19:10.574904 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:10.574887    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-tqvwm"
Sep 15 10:19:10.584279 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:10.584238    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-tqvwm" podStartSLOduration=1.975299362 podStartE2EDuration="3.584225997s" totalImagesPullingTime="1.608926635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:19:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:19:10.583773943 +0000 UTC m=+65.830740915" watchObservedRunningTime="2026-09-15 10:19:10.584225997 +0000 UTC m=+65.831192966"
Sep 15 10:19:10.753229 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:10.753201    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5gw48_91c14855-f9ab-4882-8185-8289a2bd8269/node-ca/0.log"
Sep 15 10:19:11.057270 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.057237    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqtl9\" (UniqueName: \"kubernetes.io/projected/88d49ac3-3a21-43ee-b74b-dfbd6239292e-kube-api-access-dqtl9\") pod \"network-check-target-rxb8p\" (UID: \"88d49ac3-3a21-43ee-b74b-dfbd6239292e\") " pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:19:11.057461 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.057309    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5d3dc08-8291-4828-87d1-f0da01fd24c8-metrics-certs\") pod \"network-metrics-daemon-gsxvv\" (UID: \"c5d3dc08-8291-4828-87d1-f0da01fd24c8\") " pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:19:11.057461 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.057335    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d9306059-36e0-4eae-8dc8-3ce075dc9b21-original-pull-secret\") pod \"global-pull-secret-syncer-qjcwx\" (UID: \"d9306059-36e0-4eae-8dc8-3ce075dc9b21\") " pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:19:11.059694 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.059676    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 15 10:19:11.059734 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.059697    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 15 10:19:11.060418 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.060403    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 15 10:19:11.070059 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.070036    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 15 10:19:11.070303 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.070285    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d9306059-36e0-4eae-8dc8-3ce075dc9b21-original-pull-secret\") pod \"global-pull-secret-syncer-qjcwx\" (UID: \"d9306059-36e0-4eae-8dc8-3ce075dc9b21\") " pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:19:11.070398 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.070364    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5d3dc08-8291-4828-87d1-f0da01fd24c8-metrics-certs\") pod \"network-metrics-daemon-gsxvv\" (UID: \"c5d3dc08-8291-4828-87d1-f0da01fd24c8\") " pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:19:11.080777 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.080757    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqtl9\" (UniqueName: \"kubernetes.io/projected/88d49ac3-3a21-43ee-b74b-dfbd6239292e-kube-api-access-dqtl9\") pod \"network-check-target-rxb8p\" (UID: \"88d49ac3-3a21-43ee-b74b-dfbd6239292e\") " pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:19:11.206674 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.206643    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lrdhd\""
Sep 15 10:19:11.212355 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.212323    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-88p68\""
Sep 15 10:19:11.214994 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.214977    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gsxvv"
Sep 15 10:19:11.215103 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.214981    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjcwx"
Sep 15 10:19:11.220724 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.220705    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:19:11.490425 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.490103    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qjcwx"]
Sep 15 10:19:11.493336 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.493188    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rxb8p"]
Sep 15 10:19:11.493664 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:19:11.493597    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9306059_36e0_4eae_8dc8_3ce075dc9b21.slice/crio-8e55cc1135162c67b432993ad936075d4c1c318706758c2b06ac062a78cf6c87 WatchSource:0}: Error finding container 8e55cc1135162c67b432993ad936075d4c1c318706758c2b06ac062a78cf6c87: Status 404 returned error can't find the container with id 8e55cc1135162c67b432993ad936075d4c1c318706758c2b06ac062a78cf6c87
Sep 15 10:19:11.494166 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.494143    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gsxvv"]
Sep 15 10:19:11.498184 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:19:11.498159    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88d49ac3_3a21_43ee_b74b_dfbd6239292e.slice/crio-300f07c59cb588ebd04fe27d4b36672d030ea9b348e2b8885c002bedde3838b9 WatchSource:0}: Error finding container 300f07c59cb588ebd04fe27d4b36672d030ea9b348e2b8885c002bedde3838b9: Status 404 returned error can't find the container with id 300f07c59cb588ebd04fe27d4b36672d030ea9b348e2b8885c002bedde3838b9
Sep 15 10:19:11.499160 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:19:11.499138    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5d3dc08_8291_4828_87d1_f0da01fd24c8.slice/crio-1fe4cd669e1ecf0e5b6b11f81a606c57895291b6e6bd4e326fd7cced04445ae9 WatchSource:0}: Error finding container 1fe4cd669e1ecf0e5b6b11f81a606c57895291b6e6bd4e326fd7cced04445ae9: Status 404 returned error can't find the container with id 1fe4cd669e1ecf0e5b6b11f81a606c57895291b6e6bd4e326fd7cced04445ae9
Sep 15 10:19:11.573453 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.573406    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qjcwx" event={"ID":"d9306059-36e0-4eae-8dc8-3ce075dc9b21","Type":"ContainerStarted","Data":"8e55cc1135162c67b432993ad936075d4c1c318706758c2b06ac062a78cf6c87"}
Sep 15 10:19:11.574589 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.574563    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gsxvv" event={"ID":"c5d3dc08-8291-4828-87d1-f0da01fd24c8","Type":"ContainerStarted","Data":"1fe4cd669e1ecf0e5b6b11f81a606c57895291b6e6bd4e326fd7cced04445ae9"}
Sep 15 10:19:11.575705 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.575679    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rxb8p" event={"ID":"88d49ac3-3a21-43ee-b74b-dfbd6239292e","Type":"ContainerStarted","Data":"300f07c59cb588ebd04fe27d4b36672d030ea9b348e2b8885c002bedde3838b9"}
Sep 15 10:19:11.616003 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.615977    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:11.639352 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.639325    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-wkxvg"]
Sep 15 10:19:11.641617 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.641598    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-wkxvg"
Sep 15 10:19:11.644025 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.644007    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 15 10:19:11.644146 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.644088    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 15 10:19:11.644146 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.644119    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 15 10:19:11.644278 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.644145    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-qsm49\""
Sep 15 10:19:11.644865 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.644848    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 15 10:19:11.644956 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.644853    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 15 10:19:11.653003 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.652982    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-wkxvg"]
Sep 15 10:19:11.661506 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.661487    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6a655087-f02c-416c-87d2-7aba0811f5c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2cl9v\" (UID: \"6a655087-f02c-416c-87d2-7aba0811f5c0\") " pod="openshift-insights/insights-runtime-extractor-2cl9v"
Sep 15 10:19:11.661604 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.661561    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6a655087-f02c-416c-87d2-7aba0811f5c0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2cl9v\" (UID: \"6a655087-f02c-416c-87d2-7aba0811f5c0\") " pod="openshift-insights/insights-runtime-extractor-2cl9v"
Sep 15 10:19:11.661664 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:19:11.661638    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 10:19:11.661664 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:19:11.661655    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 10:19:11.661751 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:19:11.661712    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a655087-f02c-416c-87d2-7aba0811f5c0-insights-runtime-extractor-tls podName:6a655087-f02c-416c-87d2-7aba0811f5c0 nodeName:}" failed. No retries permitted until 2026-09-15 10:19:15.661690045 +0000 UTC m=+70.908657008 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6a655087-f02c-416c-87d2-7aba0811f5c0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2cl9v" (UID: "6a655087-f02c-416c-87d2-7aba0811f5c0") : secret "insights-runtime-extractor-tls" not found
Sep 15 10:19:11.661751 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:19:11.661734    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6a655087-f02c-416c-87d2-7aba0811f5c0-kube-rbac-proxy-cm podName:6a655087-f02c-416c-87d2-7aba0811f5c0 nodeName:}" failed. No retries permitted until 2026-09-15 10:19:15.661722002 +0000 UTC m=+70.908688950 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6a655087-f02c-416c-87d2-7aba0811f5c0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2cl9v" (UID: "6a655087-f02c-416c-87d2-7aba0811f5c0") : configmap "kube-rbac-proxy" not found
Sep 15 10:19:11.714395 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.714362    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:11.762712 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.762631    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/6124cb0d-6bc8-4be2-beaf-a215f84856cd-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-wkxvg\" (UID: \"6124cb0d-6bc8-4be2-beaf-a215f84856cd\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-wkxvg"
Sep 15 10:19:11.762712 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.762668    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6124cb0d-6bc8-4be2-beaf-a215f84856cd-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-wkxvg\" (UID: \"6124cb0d-6bc8-4be2-beaf-a215f84856cd\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-wkxvg"
Sep 15 10:19:11.762712 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.762687    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd2j9\" (UniqueName: \"kubernetes.io/projected/6124cb0d-6bc8-4be2-beaf-a215f84856cd-kube-api-access-wd2j9\") pod \"prometheus-operator-68bcd7f987-wkxvg\" (UID: \"6124cb0d-6bc8-4be2-beaf-a215f84856cd\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-wkxvg"
Sep 15 10:19:11.762916 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.762736    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6124cb0d-6bc8-4be2-beaf-a215f84856cd-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-wkxvg\" (UID: \"6124cb0d-6bc8-4be2-beaf-a215f84856cd\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-wkxvg"
Sep 15 10:19:11.823408 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.823376    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:11.863619 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.863594    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/6124cb0d-6bc8-4be2-beaf-a215f84856cd-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-wkxvg\" (UID: \"6124cb0d-6bc8-4be2-beaf-a215f84856cd\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-wkxvg"
Sep 15 10:19:11.863754 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.863630    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6124cb0d-6bc8-4be2-beaf-a215f84856cd-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-wkxvg\" (UID: \"6124cb0d-6bc8-4be2-beaf-a215f84856cd\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-wkxvg"
Sep 15 10:19:11.863754 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.863649    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd2j9\" (UniqueName: \"kubernetes.io/projected/6124cb0d-6bc8-4be2-beaf-a215f84856cd-kube-api-access-wd2j9\") pod \"prometheus-operator-68bcd7f987-wkxvg\" (UID: \"6124cb0d-6bc8-4be2-beaf-a215f84856cd\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-wkxvg"
Sep 15 10:19:11.863754 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.863678    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6124cb0d-6bc8-4be2-beaf-a215f84856cd-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-wkxvg\" (UID: \"6124cb0d-6bc8-4be2-beaf-a215f84856cd\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-wkxvg"
Sep 15 10:19:11.863878 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:19:11.863750    2583 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 15 10:19:11.863878 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:19:11.863820    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6124cb0d-6bc8-4be2-beaf-a215f84856cd-prometheus-operator-tls podName:6124cb0d-6bc8-4be2-beaf-a215f84856cd nodeName:}" failed. No retries permitted until 2026-09-15 10:19:12.363801571 +0000 UTC m=+67.610768533 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/6124cb0d-6bc8-4be2-beaf-a215f84856cd-prometheus-operator-tls") pod "prometheus-operator-68bcd7f987-wkxvg" (UID: "6124cb0d-6bc8-4be2-beaf-a215f84856cd") : secret "prometheus-operator-tls" not found
Sep 15 10:19:11.864368 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.864350    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6124cb0d-6bc8-4be2-beaf-a215f84856cd-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-wkxvg\" (UID: \"6124cb0d-6bc8-4be2-beaf-a215f84856cd\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-wkxvg"
Sep 15 10:19:11.865955 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.865939    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6124cb0d-6bc8-4be2-beaf-a215f84856cd-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-wkxvg\" (UID: \"6124cb0d-6bc8-4be2-beaf-a215f84856cd\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-wkxvg"
Sep 15 10:19:11.871416 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.871397    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd2j9\" (UniqueName: \"kubernetes.io/projected/6124cb0d-6bc8-4be2-beaf-a215f84856cd-kube-api-access-wd2j9\") pod \"prometheus-operator-68bcd7f987-wkxvg\" (UID: \"6124cb0d-6bc8-4be2-beaf-a215f84856cd\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-wkxvg"
Sep 15 10:19:11.953132 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:11.953096    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lr47g_55758f39-d31a-4b0c-bb35-ef1f3be7d25b/serve-healthcheck-canary/0.log"
Sep 15 10:19:12.367710 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:12.367674    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/6124cb0d-6bc8-4be2-beaf-a215f84856cd-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-wkxvg\" (UID: \"6124cb0d-6bc8-4be2-beaf-a215f84856cd\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-wkxvg"
Sep 15 10:19:12.370426 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:12.370399    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/6124cb0d-6bc8-4be2-beaf-a215f84856cd-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-wkxvg\" (UID: \"6124cb0d-6bc8-4be2-beaf-a215f84856cd\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-wkxvg"
Sep 15 10:19:12.403115 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:12.403083    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:12.550802 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:12.550706    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-wkxvg"
Sep 15 10:19:12.856404 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:12.856328    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-wkxvg"]
Sep 15 10:19:12.860664 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:19:12.860634    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6124cb0d_6bc8_4be2_beaf_a215f84856cd.slice/crio-43961213604d50493069d53757327fd78569a5fcb636051ba6e2da67786df3dd WatchSource:0}: Error finding container 43961213604d50493069d53757327fd78569a5fcb636051ba6e2da67786df3dd: Status 404 returned error can't find the container with id 43961213604d50493069d53757327fd78569a5fcb636051ba6e2da67786df3dd
Sep 15 10:19:13.583542 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:13.583510    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-wkxvg" event={"ID":"6124cb0d-6bc8-4be2-beaf-a215f84856cd","Type":"ContainerStarted","Data":"43961213604d50493069d53757327fd78569a5fcb636051ba6e2da67786df3dd"}
Sep 15 10:19:13.585370 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:13.585340    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gsxvv" event={"ID":"c5d3dc08-8291-4828-87d1-f0da01fd24c8","Type":"ContainerStarted","Data":"fe48ded6cb039c144729df46c78e99a06d88b26c5b71450d8e503ba478b96529"}
Sep 15 10:19:13.585506 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:13.585395    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gsxvv" event={"ID":"c5d3dc08-8291-4828-87d1-f0da01fd24c8","Type":"ContainerStarted","Data":"ffc15d94d924b311372f7800539ec7b6f3b094bfb820f2d793a67385fa1ea5bd"}
Sep 15 10:19:13.603884 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:13.603836    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-gsxvv" podStartSLOduration=66.474174159 podStartE2EDuration="1m7.603822718s" totalImagesPullingTime="1.129648559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:18:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:19:13.603184852 +0000 UTC m=+68.850151823" watchObservedRunningTime="2026-09-15 10:19:13.603822718 +0000 UTC m=+68.850789688"
Sep 15 10:19:13.700559 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:13.700522    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:13.802641 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:13.802605    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:14.723324 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:14.723287    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:14.880302 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:14.880216    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:15.596606 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:15.596567    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rxb8p" event={"ID":"88d49ac3-3a21-43ee-b74b-dfbd6239292e","Type":"ContainerStarted","Data":"da86f55cd8e22986f54378dcb3ab1d77248a49720a0ad19e193a7bf9a778d6fc"}
Sep 15 10:19:15.596772 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:15.596672    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:19:15.612117 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:15.612060    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rxb8p" podStartSLOduration=66.544285039 podStartE2EDuration="1m9.612043399s" totalImagesPullingTime="3.06775836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:18:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:19:15.611347547 +0000 UTC m=+70.858314519" watchObservedRunningTime="2026-09-15 10:19:15.612043399 +0000 UTC m=+70.859010391"
Sep 15 10:19:15.696076 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:15.696035    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6a655087-f02c-416c-87d2-7aba0811f5c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2cl9v\" (UID: \"6a655087-f02c-416c-87d2-7aba0811f5c0\") " pod="openshift-insights/insights-runtime-extractor-2cl9v"
Sep 15 10:19:15.696258 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:15.696194    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6a655087-f02c-416c-87d2-7aba0811f5c0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2cl9v\" (UID: \"6a655087-f02c-416c-87d2-7aba0811f5c0\") " pod="openshift-insights/insights-runtime-extractor-2cl9v"
Sep 15 10:19:15.696258 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:19:15.696200    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 10:19:15.696362 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:19:15.696295    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 15 10:19:15.696362 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:19:15.696313    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a655087-f02c-416c-87d2-7aba0811f5c0-insights-runtime-extractor-tls podName:6a655087-f02c-416c-87d2-7aba0811f5c0 nodeName:}" failed. No retries permitted until 2026-09-15 10:19:23.696292764 +0000 UTC m=+78.943259715 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6a655087-f02c-416c-87d2-7aba0811f5c0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2cl9v" (UID: "6a655087-f02c-416c-87d2-7aba0811f5c0") : secret "insights-runtime-extractor-tls" not found
Sep 15 10:19:15.696362 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:19:15.696343    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6a655087-f02c-416c-87d2-7aba0811f5c0-kube-rbac-proxy-cm podName:6a655087-f02c-416c-87d2-7aba0811f5c0 nodeName:}" failed. No retries permitted until 2026-09-15 10:19:23.696330911 +0000 UTC m=+78.943297859 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6a655087-f02c-416c-87d2-7aba0811f5c0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2cl9v" (UID: "6a655087-f02c-416c-87d2-7aba0811f5c0") : configmap "kube-rbac-proxy" not found
Sep 15 10:19:15.777458 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:15.777414    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:16.121646 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:16.121606    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:16.222113 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:16.222071    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:16.719016 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:16.718991    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:16.819022 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:16.818979    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:16.919232 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:16.919127    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:17.205175 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:17.205054    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:17.308553 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:17.308521    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:17.605154 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:17.605060    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-wkxvg" event={"ID":"6124cb0d-6bc8-4be2-beaf-a215f84856cd","Type":"ContainerStarted","Data":"3ba32f5872cb914309b8757a420374fad21d95629944b4856dc7aeb65ca80ddf"}
Sep 15 10:19:17.605154 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:17.605121    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-wkxvg" event={"ID":"6124cb0d-6bc8-4be2-beaf-a215f84856cd","Type":"ContainerStarted","Data":"75bd4d0e8038f561fc98d3ce046aa5abdcceb6afe5ccdc0371884e0db96088df"}
Sep 15 10:19:17.606409 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:17.606384    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qjcwx" event={"ID":"d9306059-36e0-4eae-8dc8-3ce075dc9b21","Type":"ContainerStarted","Data":"0f3c5237d50049e94c9f9427e87b2f8c9c8c730459e667714fd6e828a5a8f805"}
Sep 15 10:19:17.621701 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:17.621656    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-wkxvg" podStartSLOduration=2.831687529 podStartE2EDuration="6.621642015s" totalImagesPullingTime="3.789954486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:19:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:19:17.619890732 +0000 UTC m=+72.866857703" watchObservedRunningTime="2026-09-15 10:19:17.621642015 +0000 UTC m=+72.868609030"
Sep 15 10:19:17.634526 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:17.634489    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-qjcwx" podStartSLOduration=66.470985437 podStartE2EDuration="1m11.634476917s" totalImagesPullingTime="5.16349148s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 10:18:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:19:17.633814925 +0000 UTC m=+72.880781896" watchObservedRunningTime="2026-09-15 10:19:17.634476917 +0000 UTC m=+72.881443886"
Sep 15 10:19:18.122797 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:18.122766    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d647d49f-jn5v7"]
Sep 15 10:19:18.125724 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:18.125706    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d647d49f-jn5v7"
Sep 15 10:19:18.128278 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:18.128258    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 15 10:19:18.128278 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:18.128261    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 15 10:19:18.128471 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:18.128293    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 15 10:19:18.128990 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:18.128974    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-stw5v\""
Sep 15 10:19:18.129043 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:18.129015    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 15 10:19:18.129218 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:18.129196    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 15 10:19:18.129306 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:18.129196    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 15 10:19:18.129306 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:18.129274    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 15 10:19:18.135845 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:18.135826    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d647d49f-jn5v7"]
Sep 15 10:19:18.216721 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:18.216685    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bed954a8-d423-41c8-b5a3-16f83573dae7-console-oauth-config\") pod \"console-5d647d49f-jn5v7\" (UID: \"bed954a8-d423-41c8-b5a3-16f83573dae7\") " pod="openshift-console/console-5d647d49f-jn5v7"
Sep 15 10:19:18.216721 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:18.216728    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bed954a8-d423-41c8-b5a3-16f83573dae7-console-serving-cert\") pod \"console-5d647d49f-jn5v7\" (UID: \"bed954a8-d423-41c8-b5a3-16f83573dae7\") " pod="openshift-console/console-5d647d49f-jn5v7"
Sep 15 10:19:18.216947 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:18.216752    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bed954a8-d423-41c8-b5a3-16f83573dae7-oauth-serving-cert\") pod \"console-5d647d49f-jn5v7\" (UID: \"bed954a8-d423-41c8-b5a3-16f83573dae7\") " pod="openshift-console/console-5d647d49f-jn5v7"
Sep 15 10:19:18.216947 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:18.216776    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bed954a8-d423-41c8-b5a3-16f83573dae7-console-config\") pod \"console-5d647d49f-jn5v7\" (UID: \"bed954a8-d423-41c8-b5a3-16f83573dae7\") " pod="openshift-console/console-5d647d49f-jn5v7"
Sep 15 10:19:18.216947 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:18.216842    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bed954a8-d423-41c8-b5a3-16f83573dae7-service-ca\") pod \"console-5d647d49f-jn5v7\" (UID: \"bed954a8-d423-41c8-b5a3-16f83573dae7\") " pod="openshift-console/console-5d647d49f-jn5v7"
Sep 15 10:19:18.216947 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:18.216904    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhftb\" (UniqueName: \"kubernetes.io/projected/bed954a8-d423-41c8-b5a3-16f83573dae7-kube-api-access-bhftb\") pod \"console-5d647d49f-jn5v7\" (UID: \"bed954a8-d423-41c8-b5a3-16f83573dae7\") " pod="openshift-console/console-5d647d49f-jn5v7"
Sep 15 10:19:18.317538 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:18.317504    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bed954a8-d423-41c8-b5a3-16f83573dae7-console-oauth-config\") pod \"console-5d647d49f-jn5v7\" (UID: \"bed954a8-d423-41c8-b5a3-16f83573dae7\") " pod="openshift-console/console-5d647d49f-jn5v7"
Sep 15 10:19:18.317538 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:18.317545    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bed954a8-d423-41c8-b5a3-16f83573dae7-console-serving-cert\") pod \"console-5d647d49f-jn5v7\" (UID: \"bed954a8-d423-41c8-b5a3-16f83573dae7\") " pod="openshift-console/console-5d647d49f-jn5v7"
Sep 15 10:19:18.317762 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:18.317567    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bed954a8-d423-41c8-b5a3-16f83573dae7-oauth-serving-cert\") pod \"console-5d647d49f-jn5v7\" (UID: \"bed954a8-d423-41c8-b5a3-16f83573dae7\") " pod="openshift-console/console-5d647d49f-jn5v7"
Sep 15 10:19:18.317762 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:18.317584    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bed954a8-d423-41c8-b5a3-16f83573dae7-console-config\") pod \"console-5d647d49f-jn5v7\" (UID: \"bed954a8-d423-41c8-b5a3-16f83573dae7\") " pod="openshift-console/console-5d647d49f-jn5v7"
Sep 15 10:19:18.317762 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:18.317649    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bed954a8-d423-41c8-b5a3-16f83573dae7-service-ca\") pod \"console-5d647d49f-jn5v7\" (UID: \"bed954a8-d423-41c8-b5a3-16f83573dae7\") " pod="openshift-console/console-5d647d49f-jn5v7"
Sep 15 10:19:18.317762 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:18.317675    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhftb\" (UniqueName: \"kubernetes.io/projected/bed954a8-d423-41c8-b5a3-16f83573dae7-kube-api-access-bhftb\") pod \"console-5d647d49f-jn5v7\" (UID: \"bed954a8-d423-41c8-b5a3-16f83573dae7\") " pod="openshift-console/console-5d647d49f-jn5v7"
Sep 15 10:19:18.318334 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:18.318308    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bed954a8-d423-41c8-b5a3-16f83573dae7-oauth-serving-cert\") pod \"console-5d647d49f-jn5v7\" (UID: \"bed954a8-d423-41c8-b5a3-16f83573dae7\") " pod="openshift-console/console-5d647d49f-jn5v7"
Sep 15 10:19:18.318421 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:18.318332    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bed954a8-d423-41c8-b5a3-16f83573dae7-console-config\") pod \"console-5d647d49f-jn5v7\" (UID: \"bed954a8-d423-41c8-b5a3-16f83573dae7\") " pod="openshift-console/console-5d647d49f-jn5v7"
Sep 15 10:19:18.318421 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:18.318369    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bed954a8-d423-41c8-b5a3-16f83573dae7-service-ca\") pod \"console-5d647d49f-jn5v7\" (UID: \"bed954a8-d423-41c8-b5a3-16f83573dae7\") " pod="openshift-console/console-5d647d49f-jn5v7"
Sep 15 10:19:18.320075 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:18.320059    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bed954a8-d423-41c8-b5a3-16f83573dae7-console-serving-cert\") pod \"console-5d647d49f-jn5v7\" (UID: \"bed954a8-d423-41c8-b5a3-16f83573dae7\") " pod="openshift-console/console-5d647d49f-jn5v7"
Sep 15 10:19:18.320154 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:18.320138    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bed954a8-d423-41c8-b5a3-16f83573dae7-console-oauth-config\") pod \"console-5d647d49f-jn5v7\" (UID: \"bed954a8-d423-41c8-b5a3-16f83573dae7\") " pod="openshift-console/console-5d647d49f-jn5v7"
Sep 15 10:19:18.325617 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:18.325592    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhftb\" (UniqueName: \"kubernetes.io/projected/bed954a8-d423-41c8-b5a3-16f83573dae7-kube-api-access-bhftb\") pod \"console-5d647d49f-jn5v7\" (UID: \"bed954a8-d423-41c8-b5a3-16f83573dae7\") " pod="openshift-console/console-5d647d49f-jn5v7"
Sep 15 10:19:18.434768 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:18.434730    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d647d49f-jn5v7"
Sep 15 10:19:18.702993 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:18.702924    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d647d49f-jn5v7"]
Sep 15 10:19:18.705956 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:19:18.705932    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbed954a8_d423_41c8_b5a3_16f83573dae7.slice/crio-be0fc0c57f065df23b4595d7007c2d67668e4552bf1b43ce5b7498e0a676f7bc WatchSource:0}: Error finding container be0fc0c57f065df23b4595d7007c2d67668e4552bf1b43ce5b7498e0a676f7bc: Status 404 returned error can't find the container with id be0fc0c57f065df23b4595d7007c2d67668e4552bf1b43ce5b7498e0a676f7bc
Sep 15 10:19:19.529312 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:19.529271    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:19.615514 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:19.615473    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d647d49f-jn5v7" event={"ID":"bed954a8-d423-41c8-b5a3-16f83573dae7","Type":"ContainerStarted","Data":"be0fc0c57f065df23b4595d7007c2d67668e4552bf1b43ce5b7498e0a676f7bc"}
Sep 15 10:19:19.627290 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:19.627261    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:19.747036 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:19.747006    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:19.961528 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:19.961490    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:20.016334 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.016300    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ftgzh"]
Sep 15 10:19:20.019765 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.019743    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ftgzh"
Sep 15 10:19:20.024136 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.022634    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 15 10:19:20.024136 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.022901    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vpk4z\""
Sep 15 10:19:20.024136 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.022939    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 15 10:19:20.024136 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.023135    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 15 10:19:20.047281 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.047227    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:20.131890 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.131858    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4fxn\" (UniqueName: \"kubernetes.io/projected/22a055ff-e4bf-49e1-aaaf-81b643287419-kube-api-access-m4fxn\") pod \"node-exporter-ftgzh\" (UID: \"22a055ff-e4bf-49e1-aaaf-81b643287419\") " pod="openshift-monitoring/node-exporter-ftgzh"
Sep 15 10:19:20.132058 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.131902    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/22a055ff-e4bf-49e1-aaaf-81b643287419-node-exporter-textfile\") pod \"node-exporter-ftgzh\" (UID: \"22a055ff-e4bf-49e1-aaaf-81b643287419\") " pod="openshift-monitoring/node-exporter-ftgzh"
Sep 15 10:19:20.132058 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.131995    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22a055ff-e4bf-49e1-aaaf-81b643287419-metrics-client-ca\") pod \"node-exporter-ftgzh\" (UID: \"22a055ff-e4bf-49e1-aaaf-81b643287419\") " pod="openshift-monitoring/node-exporter-ftgzh"
Sep 15 10:19:20.132058 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.132043    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/22a055ff-e4bf-49e1-aaaf-81b643287419-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ftgzh\" (UID: \"22a055ff-e4bf-49e1-aaaf-81b643287419\") " pod="openshift-monitoring/node-exporter-ftgzh"
Sep 15 10:19:20.132204 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.132072    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/22a055ff-e4bf-49e1-aaaf-81b643287419-sys\") pod \"node-exporter-ftgzh\" (UID: \"22a055ff-e4bf-49e1-aaaf-81b643287419\") " pod="openshift-monitoring/node-exporter-ftgzh"
Sep 15 10:19:20.132204 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.132097    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/22a055ff-e4bf-49e1-aaaf-81b643287419-root\") pod \"node-exporter-ftgzh\" (UID: \"22a055ff-e4bf-49e1-aaaf-81b643287419\") " pod="openshift-monitoring/node-exporter-ftgzh"
Sep 15 10:19:20.132204 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.132166    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/22a055ff-e4bf-49e1-aaaf-81b643287419-node-exporter-tls\") pod \"node-exporter-ftgzh\" (UID: \"22a055ff-e4bf-49e1-aaaf-81b643287419\") " pod="openshift-monitoring/node-exporter-ftgzh"
Sep 15 10:19:20.132313 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.132260    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/22a055ff-e4bf-49e1-aaaf-81b643287419-node-exporter-accelerators-collector-config\") pod \"node-exporter-ftgzh\" (UID: \"22a055ff-e4bf-49e1-aaaf-81b643287419\") " pod="openshift-monitoring/node-exporter-ftgzh"
Sep 15 10:19:20.132354 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.132308    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/22a055ff-e4bf-49e1-aaaf-81b643287419-node-exporter-wtmp\") pod \"node-exporter-ftgzh\" (UID: \"22a055ff-e4bf-49e1-aaaf-81b643287419\") " pod="openshift-monitoring/node-exporter-ftgzh"
Sep 15 10:19:20.140610 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.140576    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:20.233164 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.233078    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4fxn\" (UniqueName: \"kubernetes.io/projected/22a055ff-e4bf-49e1-aaaf-81b643287419-kube-api-access-m4fxn\") pod \"node-exporter-ftgzh\" (UID: \"22a055ff-e4bf-49e1-aaaf-81b643287419\") " pod="openshift-monitoring/node-exporter-ftgzh"
Sep 15 10:19:20.233164 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.233115    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/22a055ff-e4bf-49e1-aaaf-81b643287419-node-exporter-textfile\") pod \"node-exporter-ftgzh\" (UID: \"22a055ff-e4bf-49e1-aaaf-81b643287419\") " pod="openshift-monitoring/node-exporter-ftgzh"
Sep 15 10:19:20.233164 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.233151    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22a055ff-e4bf-49e1-aaaf-81b643287419-metrics-client-ca\") pod \"node-exporter-ftgzh\" (UID: \"22a055ff-e4bf-49e1-aaaf-81b643287419\") " pod="openshift-monitoring/node-exporter-ftgzh"
Sep 15 10:19:20.233433 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.233172    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/22a055ff-e4bf-49e1-aaaf-81b643287419-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ftgzh\" (UID: \"22a055ff-e4bf-49e1-aaaf-81b643287419\") " pod="openshift-monitoring/node-exporter-ftgzh"
Sep 15 10:19:20.233433 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.233200    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/22a055ff-e4bf-49e1-aaaf-81b643287419-sys\") pod \"node-exporter-ftgzh\" (UID: \"22a055ff-e4bf-49e1-aaaf-81b643287419\") " pod="openshift-monitoring/node-exporter-ftgzh"
Sep 15 10:19:20.233433 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.233228    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/22a055ff-e4bf-49e1-aaaf-81b643287419-root\") pod \"node-exporter-ftgzh\" (UID: \"22a055ff-e4bf-49e1-aaaf-81b643287419\") " pod="openshift-monitoring/node-exporter-ftgzh"
Sep 15 10:19:20.233433 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.233266    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/22a055ff-e4bf-49e1-aaaf-81b643287419-node-exporter-tls\") pod \"node-exporter-ftgzh\" (UID: \"22a055ff-e4bf-49e1-aaaf-81b643287419\") " pod="openshift-monitoring/node-exporter-ftgzh"
Sep 15 10:19:20.233433 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.233315    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/22a055ff-e4bf-49e1-aaaf-81b643287419-node-exporter-accelerators-collector-config\") pod \"node-exporter-ftgzh\" (UID: \"22a055ff-e4bf-49e1-aaaf-81b643287419\") " pod="openshift-monitoring/node-exporter-ftgzh"
Sep 15 10:19:20.233433 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.233341    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/22a055ff-e4bf-49e1-aaaf-81b643287419-node-exporter-wtmp\") pod \"node-exporter-ftgzh\" (UID: \"22a055ff-e4bf-49e1-aaaf-81b643287419\") " pod="openshift-monitoring/node-exporter-ftgzh"
Sep 15 10:19:20.233433 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.233388    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/22a055ff-e4bf-49e1-aaaf-81b643287419-root\") pod \"node-exporter-ftgzh\" (UID: \"22a055ff-e4bf-49e1-aaaf-81b643287419\") " pod="openshift-monitoring/node-exporter-ftgzh"
Sep 15 10:19:20.233791 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.233480    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/22a055ff-e4bf-49e1-aaaf-81b643287419-sys\") pod \"node-exporter-ftgzh\" (UID: \"22a055ff-e4bf-49e1-aaaf-81b643287419\") " pod="openshift-monitoring/node-exporter-ftgzh"
Sep 15 10:19:20.233791 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.233520    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/22a055ff-e4bf-49e1-aaaf-81b643287419-node-exporter-wtmp\") pod \"node-exporter-ftgzh\" (UID: \"22a055ff-e4bf-49e1-aaaf-81b643287419\") " pod="openshift-monitoring/node-exporter-ftgzh"
Sep 15 10:19:20.233791 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.233529    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/22a055ff-e4bf-49e1-aaaf-81b643287419-node-exporter-textfile\") pod \"node-exporter-ftgzh\" (UID: \"22a055ff-e4bf-49e1-aaaf-81b643287419\") " pod="openshift-monitoring/node-exporter-ftgzh"
Sep 15 10:19:20.233947 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.233849    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22a055ff-e4bf-49e1-aaaf-81b643287419-metrics-client-ca\") pod \"node-exporter-ftgzh\" (UID: \"22a055ff-e4bf-49e1-aaaf-81b643287419\") " pod="openshift-monitoring/node-exporter-ftgzh"
Sep 15 10:19:20.234023 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.234004    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/22a055ff-e4bf-49e1-aaaf-81b643287419-node-exporter-accelerators-collector-config\") pod \"node-exporter-ftgzh\" (UID: \"22a055ff-e4bf-49e1-aaaf-81b643287419\") " pod="openshift-monitoring/node-exporter-ftgzh"
Sep 15 10:19:20.235751 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.235722    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/22a055ff-e4bf-49e1-aaaf-81b643287419-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ftgzh\" (UID: \"22a055ff-e4bf-49e1-aaaf-81b643287419\") " pod="openshift-monitoring/node-exporter-ftgzh"
Sep 15 10:19:20.235878 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.235810    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:20.236107 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.236087    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/22a055ff-e4bf-49e1-aaaf-81b643287419-node-exporter-tls\") pod \"node-exporter-ftgzh\" (UID: \"22a055ff-e4bf-49e1-aaaf-81b643287419\") " pod="openshift-monitoring/node-exporter-ftgzh"
Sep 15 10:19:20.247031 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.247001    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4fxn\" (UniqueName: \"kubernetes.io/projected/22a055ff-e4bf-49e1-aaaf-81b643287419-kube-api-access-m4fxn\") pod \"node-exporter-ftgzh\" (UID: \"22a055ff-e4bf-49e1-aaaf-81b643287419\") " pod="openshift-monitoring/node-exporter-ftgzh"
Sep 15 10:19:20.332538 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.332507    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ftgzh"
Sep 15 10:19:20.336954 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.336655    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:20.344855 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:19:20.344820    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22a055ff_e4bf_49e1_aaaf_81b643287419.slice/crio-61ae9cfdec41c95c9cad7ee2885d27a4fc21a86894e49bdb95b15afd864cbe00 WatchSource:0}: Error finding container 61ae9cfdec41c95c9cad7ee2885d27a4fc21a86894e49bdb95b15afd864cbe00: Status 404 returned error can't find the container with id 61ae9cfdec41c95c9cad7ee2885d27a4fc21a86894e49bdb95b15afd864cbe00
Sep 15 10:19:20.512664 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.512587    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:20.620722 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.620684    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ftgzh" event={"ID":"22a055ff-e4bf-49e1-aaaf-81b643287419","Type":"ContainerStarted","Data":"61ae9cfdec41c95c9cad7ee2885d27a4fc21a86894e49bdb95b15afd864cbe00"}
Sep 15 10:19:20.672213 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.672175    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:20.949414 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:20.949364    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:21.042487 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:21.042436    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:21.156050 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:21.156013    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:22.008081 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.008054    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:22.030396 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.030363    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-78bc77696b-9bk68"]
Sep 15 10:19:22.035070 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.035045    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68"
Sep 15 10:19:22.037513 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.037480    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 15 10:19:22.037890 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.037718    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-82nvdrah719cd\""
Sep 15 10:19:22.037890 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.037742    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 15 10:19:22.037890 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.037774    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 15 10:19:22.037890 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.037796    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-gzgvt\""
Sep 15 10:19:22.037890 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.037742    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 15 10:19:22.038928 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.038905    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 15 10:19:22.045011 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.044988    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-78bc77696b-9bk68"]
Sep 15 10:19:22.156197 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.156164    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/24e0b95f-40c1-44b4-9d78-63961811f3cf-secret-thanos-querier-tls\") pod \"thanos-querier-78bc77696b-9bk68\" (UID: \"24e0b95f-40c1-44b4-9d78-63961811f3cf\") " pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68"
Sep 15 10:19:22.156197 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.156200    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/24e0b95f-40c1-44b4-9d78-63961811f3cf-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78bc77696b-9bk68\" (UID: \"24e0b95f-40c1-44b4-9d78-63961811f3cf\") " pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68"
Sep 15 10:19:22.156428 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.156228    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4kqr\" (UniqueName: \"kubernetes.io/projected/24e0b95f-40c1-44b4-9d78-63961811f3cf-kube-api-access-x4kqr\") pod \"thanos-querier-78bc77696b-9bk68\" (UID: \"24e0b95f-40c1-44b4-9d78-63961811f3cf\") " pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68"
Sep 15 10:19:22.156428 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.156266    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/24e0b95f-40c1-44b4-9d78-63961811f3cf-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78bc77696b-9bk68\" (UID: \"24e0b95f-40c1-44b4-9d78-63961811f3cf\") " pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68"
Sep 15 10:19:22.156428 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.156293    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/24e0b95f-40c1-44b4-9d78-63961811f3cf-metrics-client-ca\") pod \"thanos-querier-78bc77696b-9bk68\" (UID: \"24e0b95f-40c1-44b4-9d78-63961811f3cf\") " pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68"
Sep 15 10:19:22.156428 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.156389    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/24e0b95f-40c1-44b4-9d78-63961811f3cf-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78bc77696b-9bk68\" (UID: \"24e0b95f-40c1-44b4-9d78-63961811f3cf\") " pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68"
Sep 15 10:19:22.156621 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.156431    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/24e0b95f-40c1-44b4-9d78-63961811f3cf-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78bc77696b-9bk68\" (UID: \"24e0b95f-40c1-44b4-9d78-63961811f3cf\") " pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68"
Sep 15 10:19:22.156621 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.156499    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/24e0b95f-40c1-44b4-9d78-63961811f3cf-secret-grpc-tls\") pod \"thanos-querier-78bc77696b-9bk68\" (UID: \"24e0b95f-40c1-44b4-9d78-63961811f3cf\") " pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68"
Sep 15 10:19:22.257424 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.257334    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/24e0b95f-40c1-44b4-9d78-63961811f3cf-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78bc77696b-9bk68\" (UID: \"24e0b95f-40c1-44b4-9d78-63961811f3cf\") " pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68"
Sep 15 10:19:22.257424 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.257377    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/24e0b95f-40c1-44b4-9d78-63961811f3cf-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78bc77696b-9bk68\" (UID: \"24e0b95f-40c1-44b4-9d78-63961811f3cf\") " pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68"
Sep 15 10:19:22.257424 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.257421    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/24e0b95f-40c1-44b4-9d78-63961811f3cf-secret-grpc-tls\") pod \"thanos-querier-78bc77696b-9bk68\" (UID: \"24e0b95f-40c1-44b4-9d78-63961811f3cf\") " pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68"
Sep 15 10:19:22.257621 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.257602    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/24e0b95f-40c1-44b4-9d78-63961811f3cf-secret-thanos-querier-tls\") pod \"thanos-querier-78bc77696b-9bk68\" (UID: \"24e0b95f-40c1-44b4-9d78-63961811f3cf\") " pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68"
Sep 15 10:19:22.257679 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.257636    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/24e0b95f-40c1-44b4-9d78-63961811f3cf-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78bc77696b-9bk68\" (UID: \"24e0b95f-40c1-44b4-9d78-63961811f3cf\") " pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68"
Sep 15 10:19:22.257679 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.257664    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4kqr\" (UniqueName: \"kubernetes.io/projected/24e0b95f-40c1-44b4-9d78-63961811f3cf-kube-api-access-x4kqr\") pod \"thanos-querier-78bc77696b-9bk68\" (UID: \"24e0b95f-40c1-44b4-9d78-63961811f3cf\") " pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68"
Sep 15 10:19:22.257782 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.257711    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/24e0b95f-40c1-44b4-9d78-63961811f3cf-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78bc77696b-9bk68\" (UID: \"24e0b95f-40c1-44b4-9d78-63961811f3cf\") " pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68"
Sep 15 10:19:22.257782 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.257739    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/24e0b95f-40c1-44b4-9d78-63961811f3cf-metrics-client-ca\") pod \"thanos-querier-78bc77696b-9bk68\" (UID: \"24e0b95f-40c1-44b4-9d78-63961811f3cf\") " pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68"
Sep 15 10:19:22.258741 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.258477    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/24e0b95f-40c1-44b4-9d78-63961811f3cf-metrics-client-ca\") pod \"thanos-querier-78bc77696b-9bk68\" (UID: \"24e0b95f-40c1-44b4-9d78-63961811f3cf\") " pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68"
Sep 15 10:19:22.260573 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.260527    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/24e0b95f-40c1-44b4-9d78-63961811f3cf-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78bc77696b-9bk68\" (UID: \"24e0b95f-40c1-44b4-9d78-63961811f3cf\") " pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68"
Sep 15 10:19:22.260674 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.260577    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/24e0b95f-40c1-44b4-9d78-63961811f3cf-secret-grpc-tls\") pod \"thanos-querier-78bc77696b-9bk68\" (UID: \"24e0b95f-40c1-44b4-9d78-63961811f3cf\") " pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68"
Sep 15 10:19:22.260734 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.260708    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/24e0b95f-40c1-44b4-9d78-63961811f3cf-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78bc77696b-9bk68\" (UID: \"24e0b95f-40c1-44b4-9d78-63961811f3cf\") " pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68"
Sep 15 10:19:22.260975 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.260956    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/24e0b95f-40c1-44b4-9d78-63961811f3cf-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78bc77696b-9bk68\" (UID: \"24e0b95f-40c1-44b4-9d78-63961811f3cf\") " pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68"
Sep 15 10:19:22.261016 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.260978    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/24e0b95f-40c1-44b4-9d78-63961811f3cf-secret-thanos-querier-tls\") pod \"thanos-querier-78bc77696b-9bk68\" (UID: \"24e0b95f-40c1-44b4-9d78-63961811f3cf\") " pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68"
Sep 15 10:19:22.261079 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.261064    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/24e0b95f-40c1-44b4-9d78-63961811f3cf-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78bc77696b-9bk68\" (UID: \"24e0b95f-40c1-44b4-9d78-63961811f3cf\") " pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68"
Sep 15 10:19:22.265343 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.265327    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4kqr\" (UniqueName: \"kubernetes.io/projected/24e0b95f-40c1-44b4-9d78-63961811f3cf-kube-api-access-x4kqr\") pod \"thanos-querier-78bc77696b-9bk68\" (UID: \"24e0b95f-40c1-44b4-9d78-63961811f3cf\") " pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68"
Sep 15 10:19:22.346628 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.346603    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68"
Sep 15 10:19:22.620285 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.620258    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-78bc77696b-9bk68"]
Sep 15 10:19:22.623057 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:19:22.623030    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24e0b95f_40c1_44b4_9d78_63961811f3cf.slice/crio-c2ad611f135b6f512c6866a1c8416d0426440e22e02f62e9319bd9f526655155 WatchSource:0}: Error finding container c2ad611f135b6f512c6866a1c8416d0426440e22e02f62e9319bd9f526655155: Status 404 returned error can't find the container with id c2ad611f135b6f512c6866a1c8416d0426440e22e02f62e9319bd9f526655155
Sep 15 10:19:22.629527 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.629497    2583 generic.go:356] "Generic (PLEG): container finished" podID="22a055ff-e4bf-49e1-aaaf-81b643287419" containerID="b6a017d540c30933bd20f12550c0f276bf79c8d03839dba6c729f7a0338106d9" exitCode=0
Sep 15 10:19:22.629615 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.629578    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ftgzh" event={"ID":"22a055ff-e4bf-49e1-aaaf-81b643287419","Type":"ContainerDied","Data":"b6a017d540c30933bd20f12550c0f276bf79c8d03839dba6c729f7a0338106d9"}
Sep 15 10:19:22.631181 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.631157    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d647d49f-jn5v7" event={"ID":"bed954a8-d423-41c8-b5a3-16f83573dae7","Type":"ContainerStarted","Data":"da00998d0966e8f3f28517b2676b85e4faffde91bd31db43424df3f4456b3821"}
Sep 15 10:19:22.632358 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.632338    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68" event={"ID":"24e0b95f-40c1-44b4-9d78-63961811f3cf","Type":"ContainerStarted","Data":"c2ad611f135b6f512c6866a1c8416d0426440e22e02f62e9319bd9f526655155"}
Sep 15 10:19:22.662067 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.662018    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5d647d49f-jn5v7" podStartSLOduration=1.7768166490000001 podStartE2EDuration="4.66200145s" totalImagesPullingTime="2.885184801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:19:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:19:22.661201919 +0000 UTC m=+77.908168893" watchObservedRunningTime="2026-09-15 10:19:22.66200145 +0000 UTC m=+77.908968421"
Sep 15 10:19:22.909645 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:22.909615    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:23.007425 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:23.007398    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:23.153988 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:23.153955    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:23.385636 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:23.385610    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:23.586732 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:23.586683    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:23.638332 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:23.638247    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ftgzh" event={"ID":"22a055ff-e4bf-49e1-aaaf-81b643287419","Type":"ContainerStarted","Data":"cc9506afe3401008cc532e0cfe11ee99ef626c1a2611b5befbbff0aaf33524bb"}
Sep 15 10:19:23.638332 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:23.638315    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ftgzh" event={"ID":"22a055ff-e4bf-49e1-aaaf-81b643287419","Type":"ContainerStarted","Data":"752af9da90257ec765263767accfccc630e6cd51a9592fe09d0277cdccb0d068"}
Sep 15 10:19:23.657013 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:23.656964    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ftgzh" podStartSLOduration=2.6981513010000002 podStartE2EDuration="4.656944063s" totalImagesPullingTime="1.895873739s" totalInitContainerRuntime="62.919023ms" isStatefulPod=true podCreationTimestamp="2026-09-15 10:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:19:23.656273041 +0000 UTC m=+78.903240016" watchObservedRunningTime="2026-09-15 10:19:23.656944063 +0000 UTC m=+78.903911035"
Sep 15 10:19:23.769396 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:23.769351    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6a655087-f02c-416c-87d2-7aba0811f5c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2cl9v\" (UID: \"6a655087-f02c-416c-87d2-7aba0811f5c0\") " pod="openshift-insights/insights-runtime-extractor-2cl9v"
Sep 15 10:19:23.769584 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:19:23.769503    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 15 10:19:23.769584 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:19:23.769571    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a655087-f02c-416c-87d2-7aba0811f5c0-insights-runtime-extractor-tls podName:6a655087-f02c-416c-87d2-7aba0811f5c0 nodeName:}" failed. No retries permitted until 2026-09-15 10:19:39.769549962 +0000 UTC m=+95.016516924 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6a655087-f02c-416c-87d2-7aba0811f5c0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2cl9v" (UID: "6a655087-f02c-416c-87d2-7aba0811f5c0") : secret "insights-runtime-extractor-tls" not found
Sep 15 10:19:23.769719 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:23.769638    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6a655087-f02c-416c-87d2-7aba0811f5c0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2cl9v\" (UID: \"6a655087-f02c-416c-87d2-7aba0811f5c0\") " pod="openshift-insights/insights-runtime-extractor-2cl9v"
Sep 15 10:19:23.770359 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:23.770336    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6a655087-f02c-416c-87d2-7aba0811f5c0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2cl9v\" (UID: \"6a655087-f02c-416c-87d2-7aba0811f5c0\") " pod="openshift-insights/insights-runtime-extractor-2cl9v"
Sep 15 10:19:23.956679 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:23.956602    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:24.055009 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:24.054968    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:24.113595 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:24.113558    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:24.723290 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:24.723262    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:24.760880 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:24.760852    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-c2gkm"]
Sep 15 10:19:24.764174 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:24.764153    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-c2gkm"
Sep 15 10:19:24.766321 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:24.766305    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 15 10:19:24.766494 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:24.766480    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-2v42f\""
Sep 15 10:19:24.772772 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:24.772751    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-c2gkm"]
Sep 15 10:19:24.787780 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:24.787759    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:24.878738 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:24.878707    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4nlt\" (UniqueName: \"kubernetes.io/projected/344ccc05-67d1-4e88-8750-288583c42a5b-kube-api-access-c4nlt\") pod \"monitoring-plugin-5d85778b9d-c2gkm\" (UID: \"344ccc05-67d1-4e88-8750-288583c42a5b\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-c2gkm"
Sep 15 10:19:24.878824 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:24.878740    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/344ccc05-67d1-4e88-8750-288583c42a5b-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-c2gkm\" (UID: \"344ccc05-67d1-4e88-8750-288583c42a5b\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-c2gkm"
Sep 15 10:19:24.928433 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:24.928406    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:24.979728 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:24.979660    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4nlt\" (UniqueName: \"kubernetes.io/projected/344ccc05-67d1-4e88-8750-288583c42a5b-kube-api-access-c4nlt\") pod \"monitoring-plugin-5d85778b9d-c2gkm\" (UID: \"344ccc05-67d1-4e88-8750-288583c42a5b\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-c2gkm"
Sep 15 10:19:24.979728 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:24.979696    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/344ccc05-67d1-4e88-8750-288583c42a5b-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-c2gkm\" (UID: \"344ccc05-67d1-4e88-8750-288583c42a5b\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-c2gkm"
Sep 15 10:19:24.981959 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:24.981937    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/344ccc05-67d1-4e88-8750-288583c42a5b-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-c2gkm\" (UID: \"344ccc05-67d1-4e88-8750-288583c42a5b\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-c2gkm"
Sep 15 10:19:24.986812 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:24.986793    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4nlt\" (UniqueName: \"kubernetes.io/projected/344ccc05-67d1-4e88-8750-288583c42a5b-kube-api-access-c4nlt\") pod \"monitoring-plugin-5d85778b9d-c2gkm\" (UID: \"344ccc05-67d1-4e88-8750-288583c42a5b\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-c2gkm"
Sep 15 10:19:25.029068 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.029045    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:25.076798 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.076782    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-c2gkm"
Sep 15 10:19:25.194840 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.194806    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7d8f48c558-95vcf"]
Sep 15 10:19:25.198700 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.198683    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7d8f48c558-95vcf"
Sep 15 10:19:25.203112 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.203086    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 15 10:19:25.203112 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.203111    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 15 10:19:25.203288 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.203094    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 15 10:19:25.203288 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.203225    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 15 10:19:25.203288 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.203248    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-dgn42\""
Sep 15 10:19:25.204082 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.204062    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 15 10:19:25.209779 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.209756    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 15 10:19:25.210347 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.210331    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7d8f48c558-95vcf"]
Sep 15 10:19:25.282890 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.282819    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/dfa404fd-cf1f-498c-8767-14639fb2bb4d-federate-client-tls\") pod \"telemeter-client-7d8f48c558-95vcf\" (UID: \"dfa404fd-cf1f-498c-8767-14639fb2bb4d\") " pod="openshift-monitoring/telemeter-client-7d8f48c558-95vcf"
Sep 15 10:19:25.282890 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.282867    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dfa404fd-cf1f-498c-8767-14639fb2bb4d-serving-certs-ca-bundle\") pod \"telemeter-client-7d8f48c558-95vcf\" (UID: \"dfa404fd-cf1f-498c-8767-14639fb2bb4d\") " pod="openshift-monitoring/telemeter-client-7d8f48c558-95vcf"
Sep 15 10:19:25.283078 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.282923    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/dfa404fd-cf1f-498c-8767-14639fb2bb4d-secret-telemeter-client\") pod \"telemeter-client-7d8f48c558-95vcf\" (UID: \"dfa404fd-cf1f-498c-8767-14639fb2bb4d\") " pod="openshift-monitoring/telemeter-client-7d8f48c558-95vcf"
Sep 15 10:19:25.283078 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.282954    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dfa404fd-cf1f-498c-8767-14639fb2bb4d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7d8f48c558-95vcf\" (UID: \"dfa404fd-cf1f-498c-8767-14639fb2bb4d\") " pod="openshift-monitoring/telemeter-client-7d8f48c558-95vcf"
Sep 15 10:19:25.283078 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.282978    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54t5g\" (UniqueName: \"kubernetes.io/projected/dfa404fd-cf1f-498c-8767-14639fb2bb4d-kube-api-access-54t5g\") pod \"telemeter-client-7d8f48c558-95vcf\" (UID: \"dfa404fd-cf1f-498c-8767-14639fb2bb4d\") " pod="openshift-monitoring/telemeter-client-7d8f48c558-95vcf"
Sep 15 10:19:25.283078 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.283002    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dfa404fd-cf1f-498c-8767-14639fb2bb4d-metrics-client-ca\") pod \"telemeter-client-7d8f48c558-95vcf\" (UID: \"dfa404fd-cf1f-498c-8767-14639fb2bb4d\") " pod="openshift-monitoring/telemeter-client-7d8f48c558-95vcf"
Sep 15 10:19:25.283200 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.283078    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dfa404fd-cf1f-498c-8767-14639fb2bb4d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7d8f48c558-95vcf\" (UID: \"dfa404fd-cf1f-498c-8767-14639fb2bb4d\") " pod="openshift-monitoring/telemeter-client-7d8f48c558-95vcf"
Sep 15 10:19:25.283200 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.283104    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/dfa404fd-cf1f-498c-8767-14639fb2bb4d-telemeter-client-tls\") pod \"telemeter-client-7d8f48c558-95vcf\" (UID: \"dfa404fd-cf1f-498c-8767-14639fb2bb4d\") " pod="openshift-monitoring/telemeter-client-7d8f48c558-95vcf"
Sep 15 10:19:25.340985 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.340960    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-c2gkm"]
Sep 15 10:19:25.343667 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:19:25.343647    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod344ccc05_67d1_4e88_8750_288583c42a5b.slice/crio-b09cbbcfa5cad827cc1ca70b36238af11a9d72d5b65df73db291cf649584460a WatchSource:0}: Error finding container b09cbbcfa5cad827cc1ca70b36238af11a9d72d5b65df73db291cf649584460a: Status 404 returned error can't find the container with id b09cbbcfa5cad827cc1ca70b36238af11a9d72d5b65df73db291cf649584460a
Sep 15 10:19:25.384076 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.384057    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/dfa404fd-cf1f-498c-8767-14639fb2bb4d-federate-client-tls\") pod \"telemeter-client-7d8f48c558-95vcf\" (UID: \"dfa404fd-cf1f-498c-8767-14639fb2bb4d\") " pod="openshift-monitoring/telemeter-client-7d8f48c558-95vcf"
Sep 15 10:19:25.384179 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.384092    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dfa404fd-cf1f-498c-8767-14639fb2bb4d-serving-certs-ca-bundle\") pod \"telemeter-client-7d8f48c558-95vcf\" (UID: \"dfa404fd-cf1f-498c-8767-14639fb2bb4d\") " pod="openshift-monitoring/telemeter-client-7d8f48c558-95vcf"
Sep 15 10:19:25.384179 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.384116    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/dfa404fd-cf1f-498c-8767-14639fb2bb4d-secret-telemeter-client\") pod \"telemeter-client-7d8f48c558-95vcf\" (UID: \"dfa404fd-cf1f-498c-8767-14639fb2bb4d\") " pod="openshift-monitoring/telemeter-client-7d8f48c558-95vcf"
Sep 15 10:19:25.384179 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.384141    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dfa404fd-cf1f-498c-8767-14639fb2bb4d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7d8f48c558-95vcf\" (UID: \"dfa404fd-cf1f-498c-8767-14639fb2bb4d\") " pod="openshift-monitoring/telemeter-client-7d8f48c558-95vcf"
Sep 15 10:19:25.384179 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.384167    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54t5g\" (UniqueName: \"kubernetes.io/projected/dfa404fd-cf1f-498c-8767-14639fb2bb4d-kube-api-access-54t5g\") pod \"telemeter-client-7d8f48c558-95vcf\" (UID: \"dfa404fd-cf1f-498c-8767-14639fb2bb4d\") " pod="openshift-monitoring/telemeter-client-7d8f48c558-95vcf"
Sep 15 10:19:25.384356 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.384194    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dfa404fd-cf1f-498c-8767-14639fb2bb4d-metrics-client-ca\") pod \"telemeter-client-7d8f48c558-95vcf\" (UID: \"dfa404fd-cf1f-498c-8767-14639fb2bb4d\") " pod="openshift-monitoring/telemeter-client-7d8f48c558-95vcf"
Sep 15 10:19:25.384356 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.384252    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dfa404fd-cf1f-498c-8767-14639fb2bb4d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7d8f48c558-95vcf\" (UID: \"dfa404fd-cf1f-498c-8767-14639fb2bb4d\") " pod="openshift-monitoring/telemeter-client-7d8f48c558-95vcf"
Sep 15 10:19:25.384356 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.384278    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/dfa404fd-cf1f-498c-8767-14639fb2bb4d-telemeter-client-tls\") pod \"telemeter-client-7d8f48c558-95vcf\" (UID: \"dfa404fd-cf1f-498c-8767-14639fb2bb4d\") " pod="openshift-monitoring/telemeter-client-7d8f48c558-95vcf"
Sep 15 10:19:25.384912 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.384890    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dfa404fd-cf1f-498c-8767-14639fb2bb4d-serving-certs-ca-bundle\") pod \"telemeter-client-7d8f48c558-95vcf\" (UID: \"dfa404fd-cf1f-498c-8767-14639fb2bb4d\") " pod="openshift-monitoring/telemeter-client-7d8f48c558-95vcf"
Sep 15 10:19:25.385057 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.384927    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dfa404fd-cf1f-498c-8767-14639fb2bb4d-metrics-client-ca\") pod \"telemeter-client-7d8f48c558-95vcf\" (UID: \"dfa404fd-cf1f-498c-8767-14639fb2bb4d\") " pod="openshift-monitoring/telemeter-client-7d8f48c558-95vcf"
Sep 15 10:19:25.385287 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.385264    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dfa404fd-cf1f-498c-8767-14639fb2bb4d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7d8f48c558-95vcf\" (UID: \"dfa404fd-cf1f-498c-8767-14639fb2bb4d\") " pod="openshift-monitoring/telemeter-client-7d8f48c558-95vcf"
Sep 15 10:19:25.386710 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.386691    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/dfa404fd-cf1f-498c-8767-14639fb2bb4d-telemeter-client-tls\") pod \"telemeter-client-7d8f48c558-95vcf\" (UID: \"dfa404fd-cf1f-498c-8767-14639fb2bb4d\") " pod="openshift-monitoring/telemeter-client-7d8f48c558-95vcf"
Sep 15 10:19:25.386851 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.386833    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/dfa404fd-cf1f-498c-8767-14639fb2bb4d-secret-telemeter-client\") pod \"telemeter-client-7d8f48c558-95vcf\" (UID: \"dfa404fd-cf1f-498c-8767-14639fb2bb4d\") " pod="openshift-monitoring/telemeter-client-7d8f48c558-95vcf"
Sep 15 10:19:25.386909 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.386870    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dfa404fd-cf1f-498c-8767-14639fb2bb4d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7d8f48c558-95vcf\" (UID: \"dfa404fd-cf1f-498c-8767-14639fb2bb4d\") " pod="openshift-monitoring/telemeter-client-7d8f48c558-95vcf"
Sep 15 10:19:25.386967 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.386951    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/dfa404fd-cf1f-498c-8767-14639fb2bb4d-federate-client-tls\") pod \"telemeter-client-7d8f48c558-95vcf\" (UID: \"dfa404fd-cf1f-498c-8767-14639fb2bb4d\") " pod="openshift-monitoring/telemeter-client-7d8f48c558-95vcf"
Sep 15 10:19:25.391253 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.391234    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54t5g\" (UniqueName: \"kubernetes.io/projected/dfa404fd-cf1f-498c-8767-14639fb2bb4d-kube-api-access-54t5g\") pod \"telemeter-client-7d8f48c558-95vcf\" (UID: \"dfa404fd-cf1f-498c-8767-14639fb2bb4d\") " pod="openshift-monitoring/telemeter-client-7d8f48c558-95vcf"
Sep 15 10:19:25.512513 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.512478    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7d8f48c558-95vcf"
Sep 15 10:19:25.651115 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.651079    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68" event={"ID":"24e0b95f-40c1-44b4-9d78-63961811f3cf","Type":"ContainerStarted","Data":"385cf4348416156c8b7dab41782bfde04430a2ac3d732fd75243d83a7f34940c"}
Sep 15 10:19:25.651277 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.651164    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68" event={"ID":"24e0b95f-40c1-44b4-9d78-63961811f3cf","Type":"ContainerStarted","Data":"c4a77504a02b5d595f82c568ee0945c1f090c72eddfbdc0ba7e1833473b69986"}
Sep 15 10:19:25.651277 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.651214    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68" event={"ID":"24e0b95f-40c1-44b4-9d78-63961811f3cf","Type":"ContainerStarted","Data":"6ff379c24dcb9a1856c82e8a49f73a1ea4e54d9a902351979c8e773fd7d3a13c"}
Sep 15 10:19:25.652346 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.652321    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-c2gkm" event={"ID":"344ccc05-67d1-4e88-8750-288583c42a5b","Type":"ContainerStarted","Data":"b09cbbcfa5cad827cc1ca70b36238af11a9d72d5b65df73db291cf649584460a"}
Sep 15 10:19:25.789079 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:25.789051    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7d8f48c558-95vcf"]
Sep 15 10:19:25.790199 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:19:25.790177    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfa404fd_cf1f_498c_8767_14639fb2bb4d.slice/crio-4ee57bf9ebf2b80416e5e60a79dcb812b341e833f67f3e864995b71384935c26 WatchSource:0}: Error finding container 4ee57bf9ebf2b80416e5e60a79dcb812b341e833f67f3e864995b71384935c26: Status 404 returned error can't find the container with id 4ee57bf9ebf2b80416e5e60a79dcb812b341e833f67f3e864995b71384935c26
Sep 15 10:19:26.168332 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.168250    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:26.418504 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.418402    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8677b7c699-fbcgt"]
Sep 15 10:19:26.421713 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.421691    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8677b7c699-fbcgt"
Sep 15 10:19:26.435282 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.433459    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8677b7c699-fbcgt"]
Sep 15 10:19:26.435282 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.434672    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 15 10:19:26.595483 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.595427    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kq66v\" (UniqueName: \"kubernetes.io/projected/7e60a1f1-da0a-4c68-957b-7e042203bddb-kube-api-access-kq66v\") pod \"console-8677b7c699-fbcgt\" (UID: \"7e60a1f1-da0a-4c68-957b-7e042203bddb\") " pod="openshift-console/console-8677b7c699-fbcgt"
Sep 15 10:19:26.595661 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.595529    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7e60a1f1-da0a-4c68-957b-7e042203bddb-oauth-serving-cert\") pod \"console-8677b7c699-fbcgt\" (UID: \"7e60a1f1-da0a-4c68-957b-7e042203bddb\") " pod="openshift-console/console-8677b7c699-fbcgt"
Sep 15 10:19:26.595661 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.595614    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7e60a1f1-da0a-4c68-957b-7e042203bddb-console-serving-cert\") pod \"console-8677b7c699-fbcgt\" (UID: \"7e60a1f1-da0a-4c68-957b-7e042203bddb\") " pod="openshift-console/console-8677b7c699-fbcgt"
Sep 15 10:19:26.595770 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.595668    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7e60a1f1-da0a-4c68-957b-7e042203bddb-service-ca\") pod \"console-8677b7c699-fbcgt\" (UID: \"7e60a1f1-da0a-4c68-957b-7e042203bddb\") " pod="openshift-console/console-8677b7c699-fbcgt"
Sep 15 10:19:26.595770 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.595696    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e60a1f1-da0a-4c68-957b-7e042203bddb-trusted-ca-bundle\") pod \"console-8677b7c699-fbcgt\" (UID: \"7e60a1f1-da0a-4c68-957b-7e042203bddb\") " pod="openshift-console/console-8677b7c699-fbcgt"
Sep 15 10:19:26.595770 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.595725    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7e60a1f1-da0a-4c68-957b-7e042203bddb-console-oauth-config\") pod \"console-8677b7c699-fbcgt\" (UID: \"7e60a1f1-da0a-4c68-957b-7e042203bddb\") " pod="openshift-console/console-8677b7c699-fbcgt"
Sep 15 10:19:26.595917 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.595783    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7e60a1f1-da0a-4c68-957b-7e042203bddb-console-config\") pod \"console-8677b7c699-fbcgt\" (UID: \"7e60a1f1-da0a-4c68-957b-7e042203bddb\") " pod="openshift-console/console-8677b7c699-fbcgt"
Sep 15 10:19:26.657952 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.657913    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d8f48c558-95vcf" event={"ID":"dfa404fd-cf1f-498c-8767-14639fb2bb4d","Type":"ContainerStarted","Data":"4ee57bf9ebf2b80416e5e60a79dcb812b341e833f67f3e864995b71384935c26"}
Sep 15 10:19:26.661672 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.661639    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68" event={"ID":"24e0b95f-40c1-44b4-9d78-63961811f3cf","Type":"ContainerStarted","Data":"c1f0a93bf1f9b6d847d41cb16537c5a815c7a29201355c00193bfc9f266eefa0"}
Sep 15 10:19:26.661796 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.661705    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68" event={"ID":"24e0b95f-40c1-44b4-9d78-63961811f3cf","Type":"ContainerStarted","Data":"12742134c99ff7321430cfcb73796616f906430426c007ba36caef114456e55b"}
Sep 15 10:19:26.661796 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.661744    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68" event={"ID":"24e0b95f-40c1-44b4-9d78-63961811f3cf","Type":"ContainerStarted","Data":"d9568ba0e0f530c67ecfbfb23be4f1ef779911514b16d24976fe478262f74361"}
Sep 15 10:19:26.661899 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.661851    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68"
Sep 15 10:19:26.684153 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.684064    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68" podStartSLOduration=1.72526216 podStartE2EDuration="4.684043252s" totalImagesPullingTime="2.958781092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:19:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:19:26.682919896 +0000 UTC m=+81.929886866" watchObservedRunningTime="2026-09-15 10:19:26.684043252 +0000 UTC m=+81.931010235"
Sep 15 10:19:26.696679 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.696652    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7e60a1f1-da0a-4c68-957b-7e042203bddb-service-ca\") pod \"console-8677b7c699-fbcgt\" (UID: \"7e60a1f1-da0a-4c68-957b-7e042203bddb\") " pod="openshift-console/console-8677b7c699-fbcgt"
Sep 15 10:19:26.696809 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.696693    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e60a1f1-da0a-4c68-957b-7e042203bddb-trusted-ca-bundle\") pod \"console-8677b7c699-fbcgt\" (UID: \"7e60a1f1-da0a-4c68-957b-7e042203bddb\") " pod="openshift-console/console-8677b7c699-fbcgt"
Sep 15 10:19:26.696809 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.696726    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7e60a1f1-da0a-4c68-957b-7e042203bddb-console-oauth-config\") pod \"console-8677b7c699-fbcgt\" (UID: \"7e60a1f1-da0a-4c68-957b-7e042203bddb\") " pod="openshift-console/console-8677b7c699-fbcgt"
Sep 15 10:19:26.696809 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.696752    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7e60a1f1-da0a-4c68-957b-7e042203bddb-console-config\") pod \"console-8677b7c699-fbcgt\" (UID: \"7e60a1f1-da0a-4c68-957b-7e042203bddb\") " pod="openshift-console/console-8677b7c699-fbcgt"
Sep 15 10:19:26.696809 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.696797    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kq66v\" (UniqueName: \"kubernetes.io/projected/7e60a1f1-da0a-4c68-957b-7e042203bddb-kube-api-access-kq66v\") pod \"console-8677b7c699-fbcgt\" (UID: \"7e60a1f1-da0a-4c68-957b-7e042203bddb\") " pod="openshift-console/console-8677b7c699-fbcgt"
Sep 15 10:19:26.697027 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.696840    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7e60a1f1-da0a-4c68-957b-7e042203bddb-oauth-serving-cert\") pod \"console-8677b7c699-fbcgt\" (UID: \"7e60a1f1-da0a-4c68-957b-7e042203bddb\") " pod="openshift-console/console-8677b7c699-fbcgt"
Sep 15 10:19:26.697027 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.696895    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7e60a1f1-da0a-4c68-957b-7e042203bddb-console-serving-cert\") pod \"console-8677b7c699-fbcgt\" (UID: \"7e60a1f1-da0a-4c68-957b-7e042203bddb\") " pod="openshift-console/console-8677b7c699-fbcgt"
Sep 15 10:19:26.697530 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.697501    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7e60a1f1-da0a-4c68-957b-7e042203bddb-service-ca\") pod \"console-8677b7c699-fbcgt\" (UID: \"7e60a1f1-da0a-4c68-957b-7e042203bddb\") " pod="openshift-console/console-8677b7c699-fbcgt"
Sep 15 10:19:26.697662 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.697612    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7e60a1f1-da0a-4c68-957b-7e042203bddb-console-config\") pod \"console-8677b7c699-fbcgt\" (UID: \"7e60a1f1-da0a-4c68-957b-7e042203bddb\") " pod="openshift-console/console-8677b7c699-fbcgt"
Sep 15 10:19:26.697662 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.697637    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e60a1f1-da0a-4c68-957b-7e042203bddb-trusted-ca-bundle\") pod \"console-8677b7c699-fbcgt\" (UID: \"7e60a1f1-da0a-4c68-957b-7e042203bddb\") " pod="openshift-console/console-8677b7c699-fbcgt"
Sep 15 10:19:26.697768 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.697725    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7e60a1f1-da0a-4c68-957b-7e042203bddb-oauth-serving-cert\") pod \"console-8677b7c699-fbcgt\" (UID: \"7e60a1f1-da0a-4c68-957b-7e042203bddb\") " pod="openshift-console/console-8677b7c699-fbcgt"
Sep 15 10:19:26.700052 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.700030    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7e60a1f1-da0a-4c68-957b-7e042203bddb-console-oauth-config\") pod \"console-8677b7c699-fbcgt\" (UID: \"7e60a1f1-da0a-4c68-957b-7e042203bddb\") " pod="openshift-console/console-8677b7c699-fbcgt"
Sep 15 10:19:26.700156 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.700032    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7e60a1f1-da0a-4c68-957b-7e042203bddb-console-serving-cert\") pod \"console-8677b7c699-fbcgt\" (UID: \"7e60a1f1-da0a-4c68-957b-7e042203bddb\") " pod="openshift-console/console-8677b7c699-fbcgt"
Sep 15 10:19:26.706265 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.706240    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kq66v\" (UniqueName: \"kubernetes.io/projected/7e60a1f1-da0a-4c68-957b-7e042203bddb-kube-api-access-kq66v\") pod \"console-8677b7c699-fbcgt\" (UID: \"7e60a1f1-da0a-4c68-957b-7e042203bddb\") " pod="openshift-console/console-8677b7c699-fbcgt"
Sep 15 10:19:26.736339 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.736310    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8677b7c699-fbcgt"
Sep 15 10:19:26.826861 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.826823    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:26.876168 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:26.876134    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:27.142227 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:27.142191    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:27.150221 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:27.150196    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8677b7c699-fbcgt"]
Sep 15 10:19:27.154456 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:19:27.154412    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e60a1f1_da0a_4c68_957b_7e042203bddb.slice/crio-b758ec3dbe728410f99412f090ef0b933ced69fc67b89f4aefb4bdb59c76f16c WatchSource:0}: Error finding container b758ec3dbe728410f99412f090ef0b933ced69fc67b89f4aefb4bdb59c76f16c: Status 404 returned error can't find the container with id b758ec3dbe728410f99412f090ef0b933ced69fc67b89f4aefb4bdb59c76f16c
Sep 15 10:19:27.197612 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:27.197591    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:27.668220 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:27.668087    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8677b7c699-fbcgt" event={"ID":"7e60a1f1-da0a-4c68-957b-7e042203bddb","Type":"ContainerStarted","Data":"ab5d593d69486ccdeec9d97365a5e5066c182483b81b00d8710cd77234fb9b49"}
Sep 15 10:19:27.668220 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:27.668153    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8677b7c699-fbcgt" event={"ID":"7e60a1f1-da0a-4c68-957b-7e042203bddb","Type":"ContainerStarted","Data":"b758ec3dbe728410f99412f090ef0b933ced69fc67b89f4aefb4bdb59c76f16c"}
Sep 15 10:19:27.669951 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:27.669920    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-c2gkm" event={"ID":"344ccc05-67d1-4e88-8750-288583c42a5b","Type":"ContainerStarted","Data":"049331223ed0f6bd0f5900f6c3752d18114c97d318e7634ad8645ca574e27389"}
Sep 15 10:19:27.670316 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:27.670297    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-c2gkm"
Sep 15 10:19:27.676137 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:27.676119    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-c2gkm"
Sep 15 10:19:27.685765 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:27.685728    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8677b7c699-fbcgt" podStartSLOduration=1.6857121290000001 podStartE2EDuration="1.685712129s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:19:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:19:27.685079631 +0000 UTC m=+82.932046601" watchObservedRunningTime="2026-09-15 10:19:27.685712129 +0000 UTC m=+82.932679096"
Sep 15 10:19:27.700199 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:27.700163    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-c2gkm" podStartSLOduration=2.109279741 podStartE2EDuration="3.70015025s" totalImagesPullingTime="1.590870509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:19:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:19:27.699736254 +0000 UTC m=+82.946703226" watchObservedRunningTime="2026-09-15 10:19:27.70015025 +0000 UTC m=+82.947117219"
Sep 15 10:19:28.435673 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:28.435643    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d647d49f-jn5v7"
Sep 15 10:19:28.436083 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:28.435721    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d647d49f-jn5v7"
Sep 15 10:19:28.440346 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:28.440328    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d647d49f-jn5v7"
Sep 15 10:19:28.674828 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:28.674791    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d8f48c558-95vcf" event={"ID":"dfa404fd-cf1f-498c-8767-14639fb2bb4d","Type":"ContainerStarted","Data":"0c506c74feb79f13574f041b7885622018f79895b5e6af6bb64af8f4aeaf2892"}
Sep 15 10:19:28.678812 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:28.678795    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d647d49f-jn5v7"
Sep 15 10:19:29.680542 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:29.680463    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d8f48c558-95vcf" event={"ID":"dfa404fd-cf1f-498c-8767-14639fb2bb4d","Type":"ContainerStarted","Data":"16979fcb45b7529af578502d9bedf4463d265051f91357bbeee29f6a87852b3c"}
Sep 15 10:19:29.680542 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:29.680521    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d8f48c558-95vcf" event={"ID":"dfa404fd-cf1f-498c-8767-14639fb2bb4d","Type":"ContainerStarted","Data":"f61e935f44002b7223ae93e3ea05d269769024d9f43661ae591211d2d4a04087"}
Sep 15 10:19:29.703583 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:29.703538    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7d8f48c558-95vcf" podStartSLOduration=1.167382299 podStartE2EDuration="4.703522349s" totalImagesPullingTime="3.53614005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:19:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:19:29.70206271 +0000 UTC m=+84.949029680" watchObservedRunningTime="2026-09-15 10:19:29.703522349 +0000 UTC m=+84.950489318"
Sep 15 10:19:32.208386 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:32.208351    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:32.308804 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:32.308771    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:32.676045 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:32.676019    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-78bc77696b-9bk68"
Sep 15 10:19:33.433049 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:33.433013    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8677b7c699-fbcgt"]
Sep 15 10:19:33.467193 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:33.467162    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f4f5f9486-4qj6p"]
Sep 15 10:19:33.471069 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:33.471045    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f4f5f9486-4qj6p"
Sep 15 10:19:33.481421 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:33.481390    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f4f5f9486-4qj6p"]
Sep 15 10:19:33.653280 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:33.653247    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66f0230d-d695-4187-9861-054fa55ac0a0-service-ca\") pod \"console-7f4f5f9486-4qj6p\" (UID: \"66f0230d-d695-4187-9861-054fa55ac0a0\") " pod="openshift-console/console-7f4f5f9486-4qj6p"
Sep 15 10:19:33.653280 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:33.653280    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66f0230d-d695-4187-9861-054fa55ac0a0-console-config\") pod \"console-7f4f5f9486-4qj6p\" (UID: \"66f0230d-d695-4187-9861-054fa55ac0a0\") " pod="openshift-console/console-7f4f5f9486-4qj6p"
Sep 15 10:19:33.653588 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:33.653314    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66f0230d-d695-4187-9861-054fa55ac0a0-console-oauth-config\") pod \"console-7f4f5f9486-4qj6p\" (UID: \"66f0230d-d695-4187-9861-054fa55ac0a0\") " pod="openshift-console/console-7f4f5f9486-4qj6p"
Sep 15 10:19:33.653588 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:33.653333    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66f0230d-d695-4187-9861-054fa55ac0a0-trusted-ca-bundle\") pod \"console-7f4f5f9486-4qj6p\" (UID: \"66f0230d-d695-4187-9861-054fa55ac0a0\") " pod="openshift-console/console-7f4f5f9486-4qj6p"
Sep 15 10:19:33.653588 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:33.653378    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66f0230d-d695-4187-9861-054fa55ac0a0-console-serving-cert\") pod \"console-7f4f5f9486-4qj6p\" (UID: \"66f0230d-d695-4187-9861-054fa55ac0a0\") " pod="openshift-console/console-7f4f5f9486-4qj6p"
Sep 15 10:19:33.653588 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:33.653409    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66f0230d-d695-4187-9861-054fa55ac0a0-oauth-serving-cert\") pod \"console-7f4f5f9486-4qj6p\" (UID: \"66f0230d-d695-4187-9861-054fa55ac0a0\") " pod="openshift-console/console-7f4f5f9486-4qj6p"
Sep 15 10:19:33.653588 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:33.653465    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rv69w\" (UniqueName: \"kubernetes.io/projected/66f0230d-d695-4187-9861-054fa55ac0a0-kube-api-access-rv69w\") pod \"console-7f4f5f9486-4qj6p\" (UID: \"66f0230d-d695-4187-9861-054fa55ac0a0\") " pod="openshift-console/console-7f4f5f9486-4qj6p"
Sep 15 10:19:33.754875 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:33.754787    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66f0230d-d695-4187-9861-054fa55ac0a0-console-config\") pod \"console-7f4f5f9486-4qj6p\" (UID: \"66f0230d-d695-4187-9861-054fa55ac0a0\") " pod="openshift-console/console-7f4f5f9486-4qj6p"
Sep 15 10:19:33.754875 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:33.754837    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66f0230d-d695-4187-9861-054fa55ac0a0-console-oauth-config\") pod \"console-7f4f5f9486-4qj6p\" (UID: \"66f0230d-d695-4187-9861-054fa55ac0a0\") " pod="openshift-console/console-7f4f5f9486-4qj6p"
Sep 15 10:19:33.754875 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:33.754855    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66f0230d-d695-4187-9861-054fa55ac0a0-trusted-ca-bundle\") pod \"console-7f4f5f9486-4qj6p\" (UID: \"66f0230d-d695-4187-9861-054fa55ac0a0\") " pod="openshift-console/console-7f4f5f9486-4qj6p"
Sep 15 10:19:33.755082 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:33.754887    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66f0230d-d695-4187-9861-054fa55ac0a0-console-serving-cert\") pod \"console-7f4f5f9486-4qj6p\" (UID: \"66f0230d-d695-4187-9861-054fa55ac0a0\") " pod="openshift-console/console-7f4f5f9486-4qj6p"
Sep 15 10:19:33.755082 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:33.754934    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66f0230d-d695-4187-9861-054fa55ac0a0-oauth-serving-cert\") pod \"console-7f4f5f9486-4qj6p\" (UID: \"66f0230d-d695-4187-9861-054fa55ac0a0\") " pod="openshift-console/console-7f4f5f9486-4qj6p"
Sep 15 10:19:33.755082 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:33.754968    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rv69w\" (UniqueName: \"kubernetes.io/projected/66f0230d-d695-4187-9861-054fa55ac0a0-kube-api-access-rv69w\") pod \"console-7f4f5f9486-4qj6p\" (UID: \"66f0230d-d695-4187-9861-054fa55ac0a0\") " pod="openshift-console/console-7f4f5f9486-4qj6p"
Sep 15 10:19:33.755309 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:33.754995    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66f0230d-d695-4187-9861-054fa55ac0a0-service-ca\") pod \"console-7f4f5f9486-4qj6p\" (UID: \"66f0230d-d695-4187-9861-054fa55ac0a0\") " pod="openshift-console/console-7f4f5f9486-4qj6p"
Sep 15 10:19:33.756133 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:33.756101    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66f0230d-d695-4187-9861-054fa55ac0a0-oauth-serving-cert\") pod \"console-7f4f5f9486-4qj6p\" (UID: \"66f0230d-d695-4187-9861-054fa55ac0a0\") " pod="openshift-console/console-7f4f5f9486-4qj6p"
Sep 15 10:19:33.756254 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:33.756161    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66f0230d-d695-4187-9861-054fa55ac0a0-trusted-ca-bundle\") pod \"console-7f4f5f9486-4qj6p\" (UID: \"66f0230d-d695-4187-9861-054fa55ac0a0\") " pod="openshift-console/console-7f4f5f9486-4qj6p"
Sep 15 10:19:33.756724 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:33.756701    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66f0230d-d695-4187-9861-054fa55ac0a0-service-ca\") pod \"console-7f4f5f9486-4qj6p\" (UID: \"66f0230d-d695-4187-9861-054fa55ac0a0\") " pod="openshift-console/console-7f4f5f9486-4qj6p"
Sep 15 10:19:33.760355 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:33.758394    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66f0230d-d695-4187-9861-054fa55ac0a0-console-oauth-config\") pod \"console-7f4f5f9486-4qj6p\" (UID: \"66f0230d-d695-4187-9861-054fa55ac0a0\") " pod="openshift-console/console-7f4f5f9486-4qj6p"
Sep 15 10:19:33.760355 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:33.758400    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66f0230d-d695-4187-9861-054fa55ac0a0-console-serving-cert\") pod \"console-7f4f5f9486-4qj6p\" (UID: \"66f0230d-d695-4187-9861-054fa55ac0a0\") " pod="openshift-console/console-7f4f5f9486-4qj6p"
Sep 15 10:19:33.761165 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:33.761100    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66f0230d-d695-4187-9861-054fa55ac0a0-console-config\") pod \"console-7f4f5f9486-4qj6p\" (UID: \"66f0230d-d695-4187-9861-054fa55ac0a0\") " pod="openshift-console/console-7f4f5f9486-4qj6p"
Sep 15 10:19:33.763297 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:33.763277    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rv69w\" (UniqueName: \"kubernetes.io/projected/66f0230d-d695-4187-9861-054fa55ac0a0-kube-api-access-rv69w\") pod \"console-7f4f5f9486-4qj6p\" (UID: \"66f0230d-d695-4187-9861-054fa55ac0a0\") " pod="openshift-console/console-7f4f5f9486-4qj6p"
Sep 15 10:19:33.780417 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:33.780395    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f4f5f9486-4qj6p"
Sep 15 10:19:34.047313 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:34.047234    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f4f5f9486-4qj6p"]
Sep 15 10:19:34.051968 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:19:34.051940    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66f0230d_d695_4187_9861_054fa55ac0a0.slice/crio-c1840a110233159c1e1d357bfcf22f286597b6a4bfe711bb5b38ff7da1ae85c1 WatchSource:0}: Error finding container c1840a110233159c1e1d357bfcf22f286597b6a4bfe711bb5b38ff7da1ae85c1: Status 404 returned error can't find the container with id c1840a110233159c1e1d357bfcf22f286597b6a4bfe711bb5b38ff7da1ae85c1
Sep 15 10:19:34.699290 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:34.699255    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f4f5f9486-4qj6p" event={"ID":"66f0230d-d695-4187-9861-054fa55ac0a0","Type":"ContainerStarted","Data":"1a914cec44666393d59483873722966b5705e463510959b6f42bc8724efb254f"}
Sep 15 10:19:34.699687 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:34.699301    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f4f5f9486-4qj6p" event={"ID":"66f0230d-d695-4187-9861-054fa55ac0a0","Type":"ContainerStarted","Data":"c1840a110233159c1e1d357bfcf22f286597b6a4bfe711bb5b38ff7da1ae85c1"}
Sep 15 10:19:34.715540 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:34.715480    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f4f5f9486-4qj6p" podStartSLOduration=1.7154598650000001 podStartE2EDuration="1.715459865s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:19:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:19:34.714917542 +0000 UTC m=+89.961884517" watchObservedRunningTime="2026-09-15 10:19:34.715459865 +0000 UTC m=+89.962426874"
Sep 15 10:19:36.230877 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:36.230844    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:36.286189 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:36.286157    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:36.736714 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:36.736681    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8677b7c699-fbcgt"
Sep 15 10:19:38.021998 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:38.021967    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:38.122307 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:38.122276    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:39.430545 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:39.430517    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:39.479904 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:39.479878    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:39.811511 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:39.811413    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6a655087-f02c-416c-87d2-7aba0811f5c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2cl9v\" (UID: \"6a655087-f02c-416c-87d2-7aba0811f5c0\") " pod="openshift-insights/insights-runtime-extractor-2cl9v"
Sep 15 10:19:39.813727 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:39.813701    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6a655087-f02c-416c-87d2-7aba0811f5c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2cl9v\" (UID: \"6a655087-f02c-416c-87d2-7aba0811f5c0\") " pod="openshift-insights/insights-runtime-extractor-2cl9v"
Sep 15 10:19:39.832715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:39.832689    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:39.891235 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:39.891209    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:40.065305 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:40.065222    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2cl9v"
Sep 15 10:19:40.335437 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:40.335246    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2cl9v"]
Sep 15 10:19:40.338008 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:19:40.337983    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a655087_f02c_416c_87d2_7aba0811f5c0.slice/crio-8a7707dec710cd1250e9571034ba62a77877e70d6ca25e6495ffa3ca57c901a9 WatchSource:0}: Error finding container 8a7707dec710cd1250e9571034ba62a77877e70d6ca25e6495ffa3ca57c901a9: Status 404 returned error can't find the container with id 8a7707dec710cd1250e9571034ba62a77877e70d6ca25e6495ffa3ca57c901a9
Sep 15 10:19:40.722221 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:40.722184    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2cl9v" event={"ID":"6a655087-f02c-416c-87d2-7aba0811f5c0","Type":"ContainerStarted","Data":"8620f43dfbd6f1e0bde601138f779b497581f30a4950f079c2fd2d8c192bb64c"}
Sep 15 10:19:40.722587 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:40.722243    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2cl9v" event={"ID":"6a655087-f02c-416c-87d2-7aba0811f5c0","Type":"ContainerStarted","Data":"8a7707dec710cd1250e9571034ba62a77877e70d6ca25e6495ffa3ca57c901a9"}
Sep 15 10:19:41.731023 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:41.730987    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2cl9v" event={"ID":"6a655087-f02c-416c-87d2-7aba0811f5c0","Type":"ContainerStarted","Data":"063e063ec29e94cd2f9324443f508500dcd9e80ba6dd674cb8310bd04a54c0c6"}
Sep 15 10:19:43.740172 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:43.740079    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2cl9v" event={"ID":"6a655087-f02c-416c-87d2-7aba0811f5c0","Type":"ContainerStarted","Data":"cffc99cb4e390a19ccee0319f4c6a9a074396f37ea75dd0f282399716cd47423"}
Sep 15 10:19:43.760404 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:43.760345    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2cl9v" podStartSLOduration=33.828396137 podStartE2EDuration="36.760329368s" totalImagesPullingTime="2.931933231s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 10:19:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:19:43.758277452 +0000 UTC m=+99.005244435" watchObservedRunningTime="2026-09-15 10:19:43.760329368 +0000 UTC m=+99.007296339"
Sep 15 10:19:43.781078 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:43.781040    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7f4f5f9486-4qj6p"
Sep 15 10:19:43.781266 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:43.781109    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f4f5f9486-4qj6p"
Sep 15 10:19:43.786012 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:43.785980    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7f4f5f9486-4qj6p"
Sep 15 10:19:44.747654 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:44.747622    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7f4f5f9486-4qj6p"
Sep 15 10:19:44.792674 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:44.792637    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d647d49f-jn5v7"]
Sep 15 10:19:44.891232 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:44.891202    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:44.992434 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:44.992390    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:45.047320 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:45.047245    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:45.972655 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:45.972607    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:46.602723 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:46.602691    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rxb8p"
Sep 15 10:19:47.203701 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:47.203654    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:47.256675 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:47.256652    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:50.981430 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:50.981402    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:51.030663 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:51.030640    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:53.949928 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:53.949892    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:54.048307 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:54.048272    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:54.101043 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:54.101009    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:19:58.455005 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:58.454943    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8677b7c699-fbcgt" podUID="7e60a1f1-da0a-4c68-957b-7e042203bddb" containerName="console" containerID="cri-o://ab5d593d69486ccdeec9d97365a5e5066c182483b81b00d8710cd77234fb9b49" gracePeriod=15
Sep 15 10:19:58.689704 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:58.689681    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8677b7c699-fbcgt"
Sep 15 10:19:58.759266 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:58.759198    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7e60a1f1-da0a-4c68-957b-7e042203bddb-console-serving-cert\") pod \"7e60a1f1-da0a-4c68-957b-7e042203bddb\" (UID: \"7e60a1f1-da0a-4c68-957b-7e042203bddb\") "
Sep 15 10:19:58.759266 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:58.759225    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7e60a1f1-da0a-4c68-957b-7e042203bddb-service-ca\") pod \"7e60a1f1-da0a-4c68-957b-7e042203bddb\" (UID: \"7e60a1f1-da0a-4c68-957b-7e042203bddb\") "
Sep 15 10:19:58.759266 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:58.759258    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e60a1f1-da0a-4c68-957b-7e042203bddb-trusted-ca-bundle\") pod \"7e60a1f1-da0a-4c68-957b-7e042203bddb\" (UID: \"7e60a1f1-da0a-4c68-957b-7e042203bddb\") "
Sep 15 10:19:58.759603 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:58.759279    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kq66v\" (UniqueName: \"kubernetes.io/projected/7e60a1f1-da0a-4c68-957b-7e042203bddb-kube-api-access-kq66v\") pod \"7e60a1f1-da0a-4c68-957b-7e042203bddb\" (UID: \"7e60a1f1-da0a-4c68-957b-7e042203bddb\") "
Sep 15 10:19:58.759603 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:58.759300    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7e60a1f1-da0a-4c68-957b-7e042203bddb-oauth-serving-cert\") pod \"7e60a1f1-da0a-4c68-957b-7e042203bddb\" (UID: \"7e60a1f1-da0a-4c68-957b-7e042203bddb\") "
Sep 15 10:19:58.759603 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:58.759363    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7e60a1f1-da0a-4c68-957b-7e042203bddb-console-oauth-config\") pod \"7e60a1f1-da0a-4c68-957b-7e042203bddb\" (UID: \"7e60a1f1-da0a-4c68-957b-7e042203bddb\") "
Sep 15 10:19:58.759603 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:58.759383    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7e60a1f1-da0a-4c68-957b-7e042203bddb-console-config\") pod \"7e60a1f1-da0a-4c68-957b-7e042203bddb\" (UID: \"7e60a1f1-da0a-4c68-957b-7e042203bddb\") "
Sep 15 10:19:58.759804 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:58.759754    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e60a1f1-da0a-4c68-957b-7e042203bddb-service-ca" (OuterVolumeSpecName: "service-ca") pod "7e60a1f1-da0a-4c68-957b-7e042203bddb" (UID: "7e60a1f1-da0a-4c68-957b-7e042203bddb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:19:58.759804 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:58.759774    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e60a1f1-da0a-4c68-957b-7e042203bddb-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7e60a1f1-da0a-4c68-957b-7e042203bddb" (UID: "7e60a1f1-da0a-4c68-957b-7e042203bddb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:19:58.759878 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:58.759809    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e60a1f1-da0a-4c68-957b-7e042203bddb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7e60a1f1-da0a-4c68-957b-7e042203bddb" (UID: "7e60a1f1-da0a-4c68-957b-7e042203bddb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:19:58.759917 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:58.759900    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e60a1f1-da0a-4c68-957b-7e042203bddb-console-config" (OuterVolumeSpecName: "console-config") pod "7e60a1f1-da0a-4c68-957b-7e042203bddb" (UID: "7e60a1f1-da0a-4c68-957b-7e042203bddb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:19:58.761561 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:58.761532    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e60a1f1-da0a-4c68-957b-7e042203bddb-kube-api-access-kq66v" (OuterVolumeSpecName: "kube-api-access-kq66v") pod "7e60a1f1-da0a-4c68-957b-7e042203bddb" (UID: "7e60a1f1-da0a-4c68-957b-7e042203bddb"). InnerVolumeSpecName "kube-api-access-kq66v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:19:58.761670 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:58.761564    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e60a1f1-da0a-4c68-957b-7e042203bddb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7e60a1f1-da0a-4c68-957b-7e042203bddb" (UID: "7e60a1f1-da0a-4c68-957b-7e042203bddb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:19:58.761670 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:58.761591    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e60a1f1-da0a-4c68-957b-7e042203bddb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7e60a1f1-da0a-4c68-957b-7e042203bddb" (UID: "7e60a1f1-da0a-4c68-957b-7e042203bddb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:19:58.792460 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:58.792416    2583 generic.go:356] "Generic (PLEG): container finished" podID="7e60a1f1-da0a-4c68-957b-7e042203bddb" containerID="ab5d593d69486ccdeec9d97365a5e5066c182483b81b00d8710cd77234fb9b49" exitCode=0
Sep 15 10:19:58.792562 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:58.792480    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8677b7c699-fbcgt"
Sep 15 10:19:58.792562 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:58.792476    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8677b7c699-fbcgt" event={"ID":"7e60a1f1-da0a-4c68-957b-7e042203bddb","Type":"ContainerDied","Data":"ab5d593d69486ccdeec9d97365a5e5066c182483b81b00d8710cd77234fb9b49"}
Sep 15 10:19:58.792562 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:58.792529    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8677b7c699-fbcgt" event={"ID":"7e60a1f1-da0a-4c68-957b-7e042203bddb","Type":"ContainerDied","Data":"b758ec3dbe728410f99412f090ef0b933ced69fc67b89f4aefb4bdb59c76f16c"}
Sep 15 10:19:58.792562 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:58.792548    2583 scope.go:117] "RemoveContainer" containerID="ab5d593d69486ccdeec9d97365a5e5066c182483b81b00d8710cd77234fb9b49"
Sep 15 10:19:58.801672 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:58.801656    2583 scope.go:117] "RemoveContainer" containerID="ab5d593d69486ccdeec9d97365a5e5066c182483b81b00d8710cd77234fb9b49"
Sep 15 10:19:58.801923 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:19:58.801900    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab5d593d69486ccdeec9d97365a5e5066c182483b81b00d8710cd77234fb9b49\": container with ID starting with ab5d593d69486ccdeec9d97365a5e5066c182483b81b00d8710cd77234fb9b49 not found: ID does not exist" containerID="ab5d593d69486ccdeec9d97365a5e5066c182483b81b00d8710cd77234fb9b49"
Sep 15 10:19:58.802009 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:58.801930    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab5d593d69486ccdeec9d97365a5e5066c182483b81b00d8710cd77234fb9b49"} err="failed to get container status \"ab5d593d69486ccdeec9d97365a5e5066c182483b81b00d8710cd77234fb9b49\": rpc error: code = NotFound desc = could not find container \"ab5d593d69486ccdeec9d97365a5e5066c182483b81b00d8710cd77234fb9b49\": container with ID starting with ab5d593d69486ccdeec9d97365a5e5066c182483b81b00d8710cd77234fb9b49 not found: ID does not exist"
Sep 15 10:19:58.813574 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:58.813553    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8677b7c699-fbcgt"]
Sep 15 10:19:58.816866 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:58.816848    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8677b7c699-fbcgt"]
Sep 15 10:19:58.860585 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:58.860561    2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7e60a1f1-da0a-4c68-957b-7e042203bddb-console-oauth-config\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:19:58.860585 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:58.860582    2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7e60a1f1-da0a-4c68-957b-7e042203bddb-console-config\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:19:58.860715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:58.860591    2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7e60a1f1-da0a-4c68-957b-7e042203bddb-console-serving-cert\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:19:58.860715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:58.860599    2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7e60a1f1-da0a-4c68-957b-7e042203bddb-service-ca\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:19:58.860715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:58.860607    2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e60a1f1-da0a-4c68-957b-7e042203bddb-trusted-ca-bundle\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:19:58.860715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:58.860616    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kq66v\" (UniqueName: \"kubernetes.io/projected/7e60a1f1-da0a-4c68-957b-7e042203bddb-kube-api-access-kq66v\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:19:58.860715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:58.860625    2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7e60a1f1-da0a-4c68-957b-7e042203bddb-oauth-serving-cert\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:19:59.298987 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:19:59.298956    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e60a1f1-da0a-4c68-957b-7e042203bddb" path="/var/lib/kubelet/pods/7e60a1f1-da0a-4c68-957b-7e042203bddb/volumes"
Sep 15 10:20:02.299660 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:02.299627    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:20:02.400219 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:02.400188    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:20:09.815035 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:09.814980    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5d647d49f-jn5v7" podUID="bed954a8-d423-41c8-b5a3-16f83573dae7" containerName="console" containerID="cri-o://da00998d0966e8f3f28517b2676b85e4faffde91bd31db43424df3f4456b3821" gracePeriod=15
Sep 15 10:20:10.048826 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:10.048800    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d647d49f-jn5v7"
Sep 15 10:20:10.139149 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:10.139124    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bed954a8-d423-41c8-b5a3-16f83573dae7-console-config\") pod \"bed954a8-d423-41c8-b5a3-16f83573dae7\" (UID: \"bed954a8-d423-41c8-b5a3-16f83573dae7\") "
Sep 15 10:20:10.139323 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:10.139177    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bed954a8-d423-41c8-b5a3-16f83573dae7-service-ca\") pod \"bed954a8-d423-41c8-b5a3-16f83573dae7\" (UID: \"bed954a8-d423-41c8-b5a3-16f83573dae7\") "
Sep 15 10:20:10.139323 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:10.139200    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bed954a8-d423-41c8-b5a3-16f83573dae7-console-oauth-config\") pod \"bed954a8-d423-41c8-b5a3-16f83573dae7\" (UID: \"bed954a8-d423-41c8-b5a3-16f83573dae7\") "
Sep 15 10:20:10.139323 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:10.139227    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhftb\" (UniqueName: \"kubernetes.io/projected/bed954a8-d423-41c8-b5a3-16f83573dae7-kube-api-access-bhftb\") pod \"bed954a8-d423-41c8-b5a3-16f83573dae7\" (UID: \"bed954a8-d423-41c8-b5a3-16f83573dae7\") "
Sep 15 10:20:10.139323 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:10.139253    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bed954a8-d423-41c8-b5a3-16f83573dae7-console-serving-cert\") pod \"bed954a8-d423-41c8-b5a3-16f83573dae7\" (UID: \"bed954a8-d423-41c8-b5a3-16f83573dae7\") "
Sep 15 10:20:10.139323 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:10.139317    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bed954a8-d423-41c8-b5a3-16f83573dae7-oauth-serving-cert\") pod \"bed954a8-d423-41c8-b5a3-16f83573dae7\" (UID: \"bed954a8-d423-41c8-b5a3-16f83573dae7\") "
Sep 15 10:20:10.139619 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:10.139596    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bed954a8-d423-41c8-b5a3-16f83573dae7-console-config" (OuterVolumeSpecName: "console-config") pod "bed954a8-d423-41c8-b5a3-16f83573dae7" (UID: "bed954a8-d423-41c8-b5a3-16f83573dae7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:20:10.139717 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:10.139604    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bed954a8-d423-41c8-b5a3-16f83573dae7-service-ca" (OuterVolumeSpecName: "service-ca") pod "bed954a8-d423-41c8-b5a3-16f83573dae7" (UID: "bed954a8-d423-41c8-b5a3-16f83573dae7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:20:10.139818 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:10.139786    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bed954a8-d423-41c8-b5a3-16f83573dae7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bed954a8-d423-41c8-b5a3-16f83573dae7" (UID: "bed954a8-d423-41c8-b5a3-16f83573dae7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:20:10.141350 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:10.141321    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bed954a8-d423-41c8-b5a3-16f83573dae7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bed954a8-d423-41c8-b5a3-16f83573dae7" (UID: "bed954a8-d423-41c8-b5a3-16f83573dae7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:20:10.141350 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:10.141323    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bed954a8-d423-41c8-b5a3-16f83573dae7-kube-api-access-bhftb" (OuterVolumeSpecName: "kube-api-access-bhftb") pod "bed954a8-d423-41c8-b5a3-16f83573dae7" (UID: "bed954a8-d423-41c8-b5a3-16f83573dae7"). InnerVolumeSpecName "kube-api-access-bhftb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:20:10.141503 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:10.141394    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bed954a8-d423-41c8-b5a3-16f83573dae7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bed954a8-d423-41c8-b5a3-16f83573dae7" (UID: "bed954a8-d423-41c8-b5a3-16f83573dae7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:20:10.239821 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:10.239800    2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bed954a8-d423-41c8-b5a3-16f83573dae7-service-ca\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:20:10.239821 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:10.239818    2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bed954a8-d423-41c8-b5a3-16f83573dae7-console-oauth-config\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:20:10.239952 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:10.239827    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhftb\" (UniqueName: \"kubernetes.io/projected/bed954a8-d423-41c8-b5a3-16f83573dae7-kube-api-access-bhftb\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:20:10.239952 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:10.239836    2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bed954a8-d423-41c8-b5a3-16f83573dae7-console-serving-cert\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:20:10.239952 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:10.239844    2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bed954a8-d423-41c8-b5a3-16f83573dae7-oauth-serving-cert\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:20:10.239952 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:10.239854    2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bed954a8-d423-41c8-b5a3-16f83573dae7-console-config\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:20:10.836306 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:10.836275    2583 generic.go:356] "Generic (PLEG): container finished" podID="bed954a8-d423-41c8-b5a3-16f83573dae7" containerID="da00998d0966e8f3f28517b2676b85e4faffde91bd31db43424df3f4456b3821" exitCode=0
Sep 15 10:20:10.836733 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:10.836316    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d647d49f-jn5v7"
Sep 15 10:20:10.836733 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:10.836362    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d647d49f-jn5v7" event={"ID":"bed954a8-d423-41c8-b5a3-16f83573dae7","Type":"ContainerDied","Data":"da00998d0966e8f3f28517b2676b85e4faffde91bd31db43424df3f4456b3821"}
Sep 15 10:20:10.836733 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:10.836416    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d647d49f-jn5v7" event={"ID":"bed954a8-d423-41c8-b5a3-16f83573dae7","Type":"ContainerDied","Data":"be0fc0c57f065df23b4595d7007c2d67668e4552bf1b43ce5b7498e0a676f7bc"}
Sep 15 10:20:10.836733 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:10.836462    2583 scope.go:117] "RemoveContainer" containerID="da00998d0966e8f3f28517b2676b85e4faffde91bd31db43424df3f4456b3821"
Sep 15 10:20:10.844991 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:10.844976    2583 scope.go:117] "RemoveContainer" containerID="da00998d0966e8f3f28517b2676b85e4faffde91bd31db43424df3f4456b3821"
Sep 15 10:20:10.845239 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:20:10.845220    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da00998d0966e8f3f28517b2676b85e4faffde91bd31db43424df3f4456b3821\": container with ID starting with da00998d0966e8f3f28517b2676b85e4faffde91bd31db43424df3f4456b3821 not found: ID does not exist" containerID="da00998d0966e8f3f28517b2676b85e4faffde91bd31db43424df3f4456b3821"
Sep 15 10:20:10.845309 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:10.845246    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da00998d0966e8f3f28517b2676b85e4faffde91bd31db43424df3f4456b3821"} err="failed to get container status \"da00998d0966e8f3f28517b2676b85e4faffde91bd31db43424df3f4456b3821\": rpc error: code = NotFound desc = could not find container \"da00998d0966e8f3f28517b2676b85e4faffde91bd31db43424df3f4456b3821\": container with ID starting with da00998d0966e8f3f28517b2676b85e4faffde91bd31db43424df3f4456b3821 not found: ID does not exist"
Sep 15 10:20:10.856054 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:10.856034    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d647d49f-jn5v7"]
Sep 15 10:20:10.859963 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:10.859944    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5d647d49f-jn5v7"]
Sep 15 10:20:11.297798 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:11.297771    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bed954a8-d423-41c8-b5a3-16f83573dae7" path="/var/lib/kubelet/pods/bed954a8-d423-41c8-b5a3-16f83573dae7/volumes"
Sep 15 10:20:22.146764 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:22.146712    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:20:36.272335 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:36.272293    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:20:36.322999 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:36.322975    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:20:38.436857 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:38.436827    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:20:38.488183 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:38.488157    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:20:42.558801 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.558773    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 15 10:20:42.559154 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.559019    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e60a1f1-da0a-4c68-957b-7e042203bddb" containerName="console"
Sep 15 10:20:42.559154 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.559029    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e60a1f1-da0a-4c68-957b-7e042203bddb" containerName="console"
Sep 15 10:20:42.559154 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.559048    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bed954a8-d423-41c8-b5a3-16f83573dae7" containerName="console"
Sep 15 10:20:42.559154 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.559053    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="bed954a8-d423-41c8-b5a3-16f83573dae7" containerName="console"
Sep 15 10:20:42.559154 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.559108    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e60a1f1-da0a-4c68-957b-7e042203bddb" containerName="console"
Sep 15 10:20:42.559154 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.559117    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="bed954a8-d423-41c8-b5a3-16f83573dae7" containerName="console"
Sep 15 10:20:42.564245 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.564225    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.566612 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.566587    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-s2b8b\""
Sep 15 10:20:42.566733 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.566631    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 15 10:20:42.566831 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.566818    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 15 10:20:42.566882 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.566838    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 15 10:20:42.566882 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.566854    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 15 10:20:42.566985 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.566838    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 15 10:20:42.567248 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.567230    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 15 10:20:42.567248 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.567241    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 15 10:20:42.567405 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.567236    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 15 10:20:42.571745 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.571725    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 15 10:20:42.580766 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.580744    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 15 10:20:42.663529 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.663506    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-config-volume\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.663622 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.663538    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvwzv\" (UniqueName: \"kubernetes.io/projected/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-kube-api-access-nvwzv\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.663622 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.663563    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.663622 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.663592    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.663782 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.663620    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.663782 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.663646    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.663782 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.663681    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.663782 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.663707    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-web-config\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.663782 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.663731    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.664004 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.663779    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.664004 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.663852    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.664004 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.663879    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.664004 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.663920    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-config-out\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.764493 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.764430    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.764590 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.764499    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.764590 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.764525    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-config-out\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.764590 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.764541    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-config-volume\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.764590 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.764557    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvwzv\" (UniqueName: \"kubernetes.io/projected/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-kube-api-access-nvwzv\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.764590 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.764573    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.764794 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.764698    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.764794 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.764733    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.765131 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.765104    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.765250 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.765155    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.765250 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.765194    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-web-config\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.765250 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.765220    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.765418 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.765260    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.765889 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.765465    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.765889 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.765052    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.766635 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.766609    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.767972 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.767933    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-config-out\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.768143 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.768119    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-config-volume\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.768301 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.768283    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.768301 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.768294    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.768586 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.768567    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.768636 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.768610    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.768920 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.768897    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-web-config\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.769140 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.769115    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.769290 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.769275    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.772098 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.772073    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvwzv\" (UniqueName: \"kubernetes.io/projected/e09a6ac0-aeb1-4dae-97c1-08fcaee53de4-kube-api-access-nvwzv\") pod \"alertmanager-main-0\" (UID: \"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:42.873789 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:42.873739    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 15 10:20:43.147804 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:43.147780    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 15 10:20:43.148872 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:20:43.148843    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode09a6ac0_aeb1_4dae_97c1_08fcaee53de4.slice/crio-c9dbd3961e4407d717bb052b178e560f26235ed11b1bd50b7a67c45f298d23a9 WatchSource:0}: Error finding container c9dbd3961e4407d717bb052b178e560f26235ed11b1bd50b7a67c45f298d23a9: Status 404 returned error can't find the container with id c9dbd3961e4407d717bb052b178e560f26235ed11b1bd50b7a67c45f298d23a9
Sep 15 10:20:43.950870 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:43.950838    2583 generic.go:356] "Generic (PLEG): container finished" podID="e09a6ac0-aeb1-4dae-97c1-08fcaee53de4" containerID="9190f9e1587ae5cd178394ced993ac49e0348351e45ff2b9880fadadf972008b" exitCode=0
Sep 15 10:20:43.951244 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:43.950926    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4","Type":"ContainerDied","Data":"9190f9e1587ae5cd178394ced993ac49e0348351e45ff2b9880fadadf972008b"}
Sep 15 10:20:43.951244 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:43.950994    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4","Type":"ContainerStarted","Data":"c9dbd3961e4407d717bb052b178e560f26235ed11b1bd50b7a67c45f298d23a9"}
Sep 15 10:20:46.964987 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:46.964951    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4","Type":"ContainerStarted","Data":"8f5c095e24912191419fcefc1f160a362df5ce063f75b4e5b08b63ee8b6fa45f"}
Sep 15 10:20:46.965328 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:46.965011    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4","Type":"ContainerStarted","Data":"2a6e33481a071a0ab57d99d290704e0bfff9fcc3934b27222f1bdbfd486964f7"}
Sep 15 10:20:46.965328 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:46.965041    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4","Type":"ContainerStarted","Data":"bfd0fcc07dedcb705bc15bce195a4673c76132bbf0bdc4d9340b3e14f6c029fe"}
Sep 15 10:20:46.965328 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:46.965064    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4","Type":"ContainerStarted","Data":"b3da4e7cfae3a56f5864a3713c0d12763884257d6ba145141477f8dfd37bd098"}
Sep 15 10:20:46.965328 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:46.965086    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4","Type":"ContainerStarted","Data":"cb34cc217e396021812f687a604e96458f323272fd90d17960240ed964f28928"}
Sep 15 10:20:46.965328 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:46.965107    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e09a6ac0-aeb1-4dae-97c1-08fcaee53de4","Type":"ContainerStarted","Data":"753250a4639099e93a6a3f3ea9e2bd0aab18f3c39d1bafa9fa9a469d4ddb976b"}
Sep 15 10:20:46.992661 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:46.992617    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.964720707 podStartE2EDuration="4.992601362s" totalImagesPullingTime="2.012304304s" totalInitContainerRuntime="15.576351ms" isStatefulPod=false podCreationTimestamp="2026-09-15 10:20:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:20:46.991299625 +0000 UTC m=+162.238266605" watchObservedRunningTime="2026-09-15 10:20:46.992601362 +0000 UTC m=+162.239568332"
Sep 15 10:20:47.998270 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:47.998236    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:20:48.099389 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:20:48.099362    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:21:02.218161 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:02.218075    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:21:02.315568 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:02.315530    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:21:08.327613 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:08.327584    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f4bdf6bf5-6clb8"]
Sep 15 10:21:08.332308 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:08.332289    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f4bdf6bf5-6clb8"
Sep 15 10:21:08.340075 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:08.340055    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f4bdf6bf5-6clb8"]
Sep 15 10:21:08.410288 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:08.410260    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:21:08.441325 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:08.441299    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-console-config\") pod \"console-5f4bdf6bf5-6clb8\" (UID: \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\") " pod="openshift-console/console-5f4bdf6bf5-6clb8"
Sep 15 10:21:08.441420 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:08.441345    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-console-oauth-config\") pod \"console-5f4bdf6bf5-6clb8\" (UID: \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\") " pod="openshift-console/console-5f4bdf6bf5-6clb8"
Sep 15 10:21:08.441420 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:08.441372    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-trusted-ca-bundle\") pod \"console-5f4bdf6bf5-6clb8\" (UID: \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\") " pod="openshift-console/console-5f4bdf6bf5-6clb8"
Sep 15 10:21:08.441519 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:08.441458    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-service-ca\") pod \"console-5f4bdf6bf5-6clb8\" (UID: \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\") " pod="openshift-console/console-5f4bdf6bf5-6clb8"
Sep 15 10:21:08.441519 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:08.441514    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-oauth-serving-cert\") pod \"console-5f4bdf6bf5-6clb8\" (UID: \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\") " pod="openshift-console/console-5f4bdf6bf5-6clb8"
Sep 15 10:21:08.441586 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:08.441530    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skdmk\" (UniqueName: \"kubernetes.io/projected/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-kube-api-access-skdmk\") pod \"console-5f4bdf6bf5-6clb8\" (UID: \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\") " pod="openshift-console/console-5f4bdf6bf5-6clb8"
Sep 15 10:21:08.441586 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:08.441549    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-console-serving-cert\") pod \"console-5f4bdf6bf5-6clb8\" (UID: \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\") " pod="openshift-console/console-5f4bdf6bf5-6clb8"
Sep 15 10:21:08.514205 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:08.514182    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:21:08.541826 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:08.541800    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-oauth-serving-cert\") pod \"console-5f4bdf6bf5-6clb8\" (UID: \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\") " pod="openshift-console/console-5f4bdf6bf5-6clb8"
Sep 15 10:21:08.541926 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:08.541831    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skdmk\" (UniqueName: \"kubernetes.io/projected/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-kube-api-access-skdmk\") pod \"console-5f4bdf6bf5-6clb8\" (UID: \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\") " pod="openshift-console/console-5f4bdf6bf5-6clb8"
Sep 15 10:21:08.541926 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:08.541858    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-console-serving-cert\") pod \"console-5f4bdf6bf5-6clb8\" (UID: \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\") " pod="openshift-console/console-5f4bdf6bf5-6clb8"
Sep 15 10:21:08.541926 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:08.541883    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-console-config\") pod \"console-5f4bdf6bf5-6clb8\" (UID: \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\") " pod="openshift-console/console-5f4bdf6bf5-6clb8"
Sep 15 10:21:08.542077 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:08.541930    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-console-oauth-config\") pod \"console-5f4bdf6bf5-6clb8\" (UID: \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\") " pod="openshift-console/console-5f4bdf6bf5-6clb8"
Sep 15 10:21:08.542077 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:08.541955    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-trusted-ca-bundle\") pod \"console-5f4bdf6bf5-6clb8\" (UID: \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\") " pod="openshift-console/console-5f4bdf6bf5-6clb8"
Sep 15 10:21:08.542077 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:08.542004    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-service-ca\") pod \"console-5f4bdf6bf5-6clb8\" (UID: \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\") " pod="openshift-console/console-5f4bdf6bf5-6clb8"
Sep 15 10:21:08.542552 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:08.542532    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-oauth-serving-cert\") pod \"console-5f4bdf6bf5-6clb8\" (UID: \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\") " pod="openshift-console/console-5f4bdf6bf5-6clb8"
Sep 15 10:21:08.542685 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:08.542662    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-console-config\") pod \"console-5f4bdf6bf5-6clb8\" (UID: \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\") " pod="openshift-console/console-5f4bdf6bf5-6clb8"
Sep 15 10:21:08.542750 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:08.542724    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-service-ca\") pod \"console-5f4bdf6bf5-6clb8\" (UID: \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\") " pod="openshift-console/console-5f4bdf6bf5-6clb8"
Sep 15 10:21:08.542871 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:08.542852    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-trusted-ca-bundle\") pod \"console-5f4bdf6bf5-6clb8\" (UID: \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\") " pod="openshift-console/console-5f4bdf6bf5-6clb8"
Sep 15 10:21:08.544460 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:08.544411    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-console-oauth-config\") pod \"console-5f4bdf6bf5-6clb8\" (UID: \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\") " pod="openshift-console/console-5f4bdf6bf5-6clb8"
Sep 15 10:21:08.544534 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:08.544460    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-console-serving-cert\") pod \"console-5f4bdf6bf5-6clb8\" (UID: \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\") " pod="openshift-console/console-5f4bdf6bf5-6clb8"
Sep 15 10:21:08.549638 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:08.549618    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skdmk\" (UniqueName: \"kubernetes.io/projected/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-kube-api-access-skdmk\") pod \"console-5f4bdf6bf5-6clb8\" (UID: \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\") " pod="openshift-console/console-5f4bdf6bf5-6clb8"
Sep 15 10:21:08.641862 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:08.641840    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f4bdf6bf5-6clb8"
Sep 15 10:21:08.910724 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:08.910703    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f4bdf6bf5-6clb8"]
Sep 15 10:21:08.912863 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:21:08.912834    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb5990c4_0a70_4f63_a145_0d81cf3e35c7.slice/crio-6fdb950a6b8a649644b53eb009e33bb892ddec2651a9bf98258b05ca94ab39c9 WatchSource:0}: Error finding container 6fdb950a6b8a649644b53eb009e33bb892ddec2651a9bf98258b05ca94ab39c9: Status 404 returned error can't find the container with id 6fdb950a6b8a649644b53eb009e33bb892ddec2651a9bf98258b05ca94ab39c9
Sep 15 10:21:09.044861 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:09.044824    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f4bdf6bf5-6clb8" event={"ID":"fb5990c4-0a70-4f63-a145-0d81cf3e35c7","Type":"ContainerStarted","Data":"e0fa95c50352dfc1643e71185164fe5c6105a6b7031edb3cd41a2e1ffef5a3d8"}
Sep 15 10:21:09.045067 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:09.044880    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f4bdf6bf5-6clb8" event={"ID":"fb5990c4-0a70-4f63-a145-0d81cf3e35c7","Type":"ContainerStarted","Data":"6fdb950a6b8a649644b53eb009e33bb892ddec2651a9bf98258b05ca94ab39c9"}
Sep 15 10:21:18.642504 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:18.642471    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f4bdf6bf5-6clb8"
Sep 15 10:21:18.642969 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:18.642555    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f4bdf6bf5-6clb8"
Sep 15 10:21:18.646849 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:18.646827    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f4bdf6bf5-6clb8"
Sep 15 10:21:18.664824 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:18.664781    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f4bdf6bf5-6clb8" podStartSLOduration=10.664769005 podStartE2EDuration="10.664769005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:21:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:21:09.061206388 +0000 UTC m=+184.308173358" watchObservedRunningTime="2026-09-15 10:21:18.664769005 +0000 UTC m=+193.911736021"
Sep 15 10:21:19.086074 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:19.085995    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f4bdf6bf5-6clb8"
Sep 15 10:21:19.131747 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:19.131721    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f4f5f9486-4qj6p"]
Sep 15 10:21:19.238012 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:19.237986    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:21:19.287736 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:19.287711    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:21:19.346658 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:19.346602    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:21:38.405423 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:38.405387    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:21:38.456027 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:38.455997    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:21:44.150228 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:44.150189    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7f4f5f9486-4qj6p" podUID="66f0230d-d695-4187-9861-054fa55ac0a0" containerName="console" containerID="cri-o://1a914cec44666393d59483873722966b5705e463510959b6f42bc8724efb254f" gracePeriod=15
Sep 15 10:21:44.385345 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:44.385321    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f4f5f9486-4qj6p"
Sep 15 10:21:44.497103 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:44.496996    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66f0230d-d695-4187-9861-054fa55ac0a0-console-config\") pod \"66f0230d-d695-4187-9861-054fa55ac0a0\" (UID: \"66f0230d-d695-4187-9861-054fa55ac0a0\") "
Sep 15 10:21:44.497103 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:44.497058    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66f0230d-d695-4187-9861-054fa55ac0a0-service-ca\") pod \"66f0230d-d695-4187-9861-054fa55ac0a0\" (UID: \"66f0230d-d695-4187-9861-054fa55ac0a0\") "
Sep 15 10:21:44.497103 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:44.497087    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66f0230d-d695-4187-9861-054fa55ac0a0-oauth-serving-cert\") pod \"66f0230d-d695-4187-9861-054fa55ac0a0\" (UID: \"66f0230d-d695-4187-9861-054fa55ac0a0\") "
Sep 15 10:21:44.497348 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:44.497188    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66f0230d-d695-4187-9861-054fa55ac0a0-console-oauth-config\") pod \"66f0230d-d695-4187-9861-054fa55ac0a0\" (UID: \"66f0230d-d695-4187-9861-054fa55ac0a0\") "
Sep 15 10:21:44.497348 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:44.497223    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66f0230d-d695-4187-9861-054fa55ac0a0-console-serving-cert\") pod \"66f0230d-d695-4187-9861-054fa55ac0a0\" (UID: \"66f0230d-d695-4187-9861-054fa55ac0a0\") "
Sep 15 10:21:44.497348 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:44.497262    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66f0230d-d695-4187-9861-054fa55ac0a0-trusted-ca-bundle\") pod \"66f0230d-d695-4187-9861-054fa55ac0a0\" (UID: \"66f0230d-d695-4187-9861-054fa55ac0a0\") "
Sep 15 10:21:44.497348 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:44.497299    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rv69w\" (UniqueName: \"kubernetes.io/projected/66f0230d-d695-4187-9861-054fa55ac0a0-kube-api-access-rv69w\") pod \"66f0230d-d695-4187-9861-054fa55ac0a0\" (UID: \"66f0230d-d695-4187-9861-054fa55ac0a0\") "
Sep 15 10:21:44.497581 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:44.497512    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66f0230d-d695-4187-9861-054fa55ac0a0-console-config" (OuterVolumeSpecName: "console-config") pod "66f0230d-d695-4187-9861-054fa55ac0a0" (UID: "66f0230d-d695-4187-9861-054fa55ac0a0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:21:44.497581 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:44.497545    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66f0230d-d695-4187-9861-054fa55ac0a0-service-ca" (OuterVolumeSpecName: "service-ca") pod "66f0230d-d695-4187-9861-054fa55ac0a0" (UID: "66f0230d-d695-4187-9861-054fa55ac0a0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:21:44.497729 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:44.497704    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66f0230d-d695-4187-9861-054fa55ac0a0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "66f0230d-d695-4187-9861-054fa55ac0a0" (UID: "66f0230d-d695-4187-9861-054fa55ac0a0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:21:44.497807 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:44.497779    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66f0230d-d695-4187-9861-054fa55ac0a0-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "66f0230d-d695-4187-9861-054fa55ac0a0" (UID: "66f0230d-d695-4187-9861-054fa55ac0a0"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:21:44.499397 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:44.499366    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66f0230d-d695-4187-9861-054fa55ac0a0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "66f0230d-d695-4187-9861-054fa55ac0a0" (UID: "66f0230d-d695-4187-9861-054fa55ac0a0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:21:44.499510 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:44.499396    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66f0230d-d695-4187-9861-054fa55ac0a0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "66f0230d-d695-4187-9861-054fa55ac0a0" (UID: "66f0230d-d695-4187-9861-054fa55ac0a0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:21:44.499510 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:44.499415    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66f0230d-d695-4187-9861-054fa55ac0a0-kube-api-access-rv69w" (OuterVolumeSpecName: "kube-api-access-rv69w") pod "66f0230d-d695-4187-9861-054fa55ac0a0" (UID: "66f0230d-d695-4187-9861-054fa55ac0a0"). InnerVolumeSpecName "kube-api-access-rv69w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:21:44.598505 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:44.598457    2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66f0230d-d695-4187-9861-054fa55ac0a0-console-oauth-config\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:21:44.598505 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:44.598498    2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66f0230d-d695-4187-9861-054fa55ac0a0-console-serving-cert\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:21:44.598505 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:44.598509    2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66f0230d-d695-4187-9861-054fa55ac0a0-trusted-ca-bundle\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:21:44.598505 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:44.598518    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rv69w\" (UniqueName: \"kubernetes.io/projected/66f0230d-d695-4187-9861-054fa55ac0a0-kube-api-access-rv69w\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:21:44.598772 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:44.598528    2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66f0230d-d695-4187-9861-054fa55ac0a0-console-config\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:21:44.598772 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:44.598537    2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66f0230d-d695-4187-9861-054fa55ac0a0-service-ca\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:21:44.598772 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:44.598544    2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66f0230d-d695-4187-9861-054fa55ac0a0-oauth-serving-cert\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:21:45.176206 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:45.176173    2583 generic.go:356] "Generic (PLEG): container finished" podID="66f0230d-d695-4187-9861-054fa55ac0a0" containerID="1a914cec44666393d59483873722966b5705e463510959b6f42bc8724efb254f" exitCode=0
Sep 15 10:21:45.176701 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:45.176223    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f4f5f9486-4qj6p"
Sep 15 10:21:45.176701 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:45.176260    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f4f5f9486-4qj6p" event={"ID":"66f0230d-d695-4187-9861-054fa55ac0a0","Type":"ContainerDied","Data":"1a914cec44666393d59483873722966b5705e463510959b6f42bc8724efb254f"}
Sep 15 10:21:45.176701 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:45.176310    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f4f5f9486-4qj6p" event={"ID":"66f0230d-d695-4187-9861-054fa55ac0a0","Type":"ContainerDied","Data":"c1840a110233159c1e1d357bfcf22f286597b6a4bfe711bb5b38ff7da1ae85c1"}
Sep 15 10:21:45.176701 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:45.176332    2583 scope.go:117] "RemoveContainer" containerID="1a914cec44666393d59483873722966b5705e463510959b6f42bc8724efb254f"
Sep 15 10:21:45.184655 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:45.184637    2583 scope.go:117] "RemoveContainer" containerID="1a914cec44666393d59483873722966b5705e463510959b6f42bc8724efb254f"
Sep 15 10:21:45.184931 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:21:45.184914    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a914cec44666393d59483873722966b5705e463510959b6f42bc8724efb254f\": container with ID starting with 1a914cec44666393d59483873722966b5705e463510959b6f42bc8724efb254f not found: ID does not exist" containerID="1a914cec44666393d59483873722966b5705e463510959b6f42bc8724efb254f"
Sep 15 10:21:45.184977 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:45.184939    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a914cec44666393d59483873722966b5705e463510959b6f42bc8724efb254f"} err="failed to get container status \"1a914cec44666393d59483873722966b5705e463510959b6f42bc8724efb254f\": rpc error: code = NotFound desc = could not find container \"1a914cec44666393d59483873722966b5705e463510959b6f42bc8724efb254f\": container with ID starting with 1a914cec44666393d59483873722966b5705e463510959b6f42bc8724efb254f not found: ID does not exist"
Sep 15 10:21:45.196026 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:45.195997    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f4f5f9486-4qj6p"]
Sep 15 10:21:45.199250 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:45.199226    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7f4f5f9486-4qj6p"]
Sep 15 10:21:45.298868 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:21:45.298834    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66f0230d-d695-4187-9861-054fa55ac0a0" path="/var/lib/kubelet/pods/66f0230d-d695-4187-9861-054fa55ac0a0/volumes"
Sep 15 10:22:00.995420 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:00.995384    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:22:01.091239 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:01.091204    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:22:01.618802 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:01.618770    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dnc7s"]
Sep 15 10:22:01.619090 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:01.619077    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66f0230d-d695-4187-9861-054fa55ac0a0" containerName="console"
Sep 15 10:22:01.619140 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:01.619092    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="66f0230d-d695-4187-9861-054fa55ac0a0" containerName="console"
Sep 15 10:22:01.619174 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:01.619140    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="66f0230d-d695-4187-9861-054fa55ac0a0" containerName="console"
Sep 15 10:22:01.621890 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:01.621875    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dnc7s"
Sep 15 10:22:01.624351 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:01.624329    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 15 10:22:01.625111 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:01.625087    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-c9n8v\""
Sep 15 10:22:01.625186 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:01.625144    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 15 10:22:01.630841 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:01.630821    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dnc7s"]
Sep 15 10:22:01.737689 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:01.737648    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac4f67cb-c9b2-4b8f-841c-4581692319b2-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dnc7s\" (UID: \"ac4f67cb-c9b2-4b8f-841c-4581692319b2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dnc7s"
Sep 15 10:22:01.737884 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:01.737705    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4rcb\" (UniqueName: \"kubernetes.io/projected/ac4f67cb-c9b2-4b8f-841c-4581692319b2-kube-api-access-z4rcb\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dnc7s\" (UID: \"ac4f67cb-c9b2-4b8f-841c-4581692319b2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dnc7s"
Sep 15 10:22:01.737884 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:01.737792    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac4f67cb-c9b2-4b8f-841c-4581692319b2-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dnc7s\" (UID: \"ac4f67cb-c9b2-4b8f-841c-4581692319b2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dnc7s"
Sep 15 10:22:01.838503 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:01.838467    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac4f67cb-c9b2-4b8f-841c-4581692319b2-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dnc7s\" (UID: \"ac4f67cb-c9b2-4b8f-841c-4581692319b2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dnc7s"
Sep 15 10:22:01.838654 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:01.838549    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac4f67cb-c9b2-4b8f-841c-4581692319b2-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dnc7s\" (UID: \"ac4f67cb-c9b2-4b8f-841c-4581692319b2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dnc7s"
Sep 15 10:22:01.838654 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:01.838577    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4rcb\" (UniqueName: \"kubernetes.io/projected/ac4f67cb-c9b2-4b8f-841c-4581692319b2-kube-api-access-z4rcb\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dnc7s\" (UID: \"ac4f67cb-c9b2-4b8f-841c-4581692319b2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dnc7s"
Sep 15 10:22:01.838854 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:01.838829    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac4f67cb-c9b2-4b8f-841c-4581692319b2-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dnc7s\" (UID: \"ac4f67cb-c9b2-4b8f-841c-4581692319b2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dnc7s"
Sep 15 10:22:01.838930 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:01.838899    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac4f67cb-c9b2-4b8f-841c-4581692319b2-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dnc7s\" (UID: \"ac4f67cb-c9b2-4b8f-841c-4581692319b2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dnc7s"
Sep 15 10:22:01.846848 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:01.846826    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4rcb\" (UniqueName: \"kubernetes.io/projected/ac4f67cb-c9b2-4b8f-841c-4581692319b2-kube-api-access-z4rcb\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dnc7s\" (UID: \"ac4f67cb-c9b2-4b8f-841c-4581692319b2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dnc7s"
Sep 15 10:22:01.930908 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:01.930876    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dnc7s"
Sep 15 10:22:02.199330 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:02.199303    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dnc7s"]
Sep 15 10:22:02.201874 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:22:02.201844    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac4f67cb_c9b2_4b8f_841c_4581692319b2.slice/crio-2f5c4156d368b2de270d76f0222b720007cd85c718704b7f6a062ab2725dc4f2 WatchSource:0}: Error finding container 2f5c4156d368b2de270d76f0222b720007cd85c718704b7f6a062ab2725dc4f2: Status 404 returned error can't find the container with id 2f5c4156d368b2de270d76f0222b720007cd85c718704b7f6a062ab2725dc4f2
Sep 15 10:22:02.246917 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:02.246880    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dnc7s" event={"ID":"ac4f67cb-c9b2-4b8f-841c-4581692319b2","Type":"ContainerStarted","Data":"2f5c4156d368b2de270d76f0222b720007cd85c718704b7f6a062ab2725dc4f2"}
Sep 15 10:22:10.285180 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:10.285147    2583 generic.go:356] "Generic (PLEG): container finished" podID="ac4f67cb-c9b2-4b8f-841c-4581692319b2" containerID="48a4ad9a97b5c83ecd97b538950f41028dadccfb6d8060ed2a580aa8fdff3d56" exitCode=0
Sep 15 10:22:10.285561 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:10.285196    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dnc7s" event={"ID":"ac4f67cb-c9b2-4b8f-841c-4581692319b2","Type":"ContainerDied","Data":"48a4ad9a97b5c83ecd97b538950f41028dadccfb6d8060ed2a580aa8fdff3d56"}
Sep 15 10:22:12.951697 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:12.951664    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:22:16.307843 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:16.307746    2583 generic.go:356] "Generic (PLEG): container finished" podID="ac4f67cb-c9b2-4b8f-841c-4581692319b2" containerID="a239f72dcc2ffa4afe5f40c6f2dd246091418d939a183e78710719796c5740eb" exitCode=0
Sep 15 10:22:16.307843 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:16.307832    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dnc7s" event={"ID":"ac4f67cb-c9b2-4b8f-841c-4581692319b2","Type":"ContainerDied","Data":"a239f72dcc2ffa4afe5f40c6f2dd246091418d939a183e78710719796c5740eb"}
Sep 15 10:22:23.344513 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:23.344481    2583 generic.go:356] "Generic (PLEG): container finished" podID="ac4f67cb-c9b2-4b8f-841c-4581692319b2" containerID="933fd49ddb3f977e0a61d0b84cf922a01d56ba0e53d30701b94c43f400b53e82" exitCode=0
Sep 15 10:22:23.344872 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:23.344561    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dnc7s" event={"ID":"ac4f67cb-c9b2-4b8f-841c-4581692319b2","Type":"ContainerDied","Data":"933fd49ddb3f977e0a61d0b84cf922a01d56ba0e53d30701b94c43f400b53e82"}
Sep 15 10:22:24.508990 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:24.508959    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dnc7s"
Sep 15 10:22:24.530112 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:24.530082    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac4f67cb-c9b2-4b8f-841c-4581692319b2-util\") pod \"ac4f67cb-c9b2-4b8f-841c-4581692319b2\" (UID: \"ac4f67cb-c9b2-4b8f-841c-4581692319b2\") "
Sep 15 10:22:24.530290 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:24.530119    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z4rcb\" (UniqueName: \"kubernetes.io/projected/ac4f67cb-c9b2-4b8f-841c-4581692319b2-kube-api-access-z4rcb\") pod \"ac4f67cb-c9b2-4b8f-841c-4581692319b2\" (UID: \"ac4f67cb-c9b2-4b8f-841c-4581692319b2\") "
Sep 15 10:22:24.530290 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:24.530149    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac4f67cb-c9b2-4b8f-841c-4581692319b2-bundle\") pod \"ac4f67cb-c9b2-4b8f-841c-4581692319b2\" (UID: \"ac4f67cb-c9b2-4b8f-841c-4581692319b2\") "
Sep 15 10:22:24.530865 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:24.530836    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac4f67cb-c9b2-4b8f-841c-4581692319b2-bundle" (OuterVolumeSpecName: "bundle") pod "ac4f67cb-c9b2-4b8f-841c-4581692319b2" (UID: "ac4f67cb-c9b2-4b8f-841c-4581692319b2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:22:24.532313 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:24.532291    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac4f67cb-c9b2-4b8f-841c-4581692319b2-kube-api-access-z4rcb" (OuterVolumeSpecName: "kube-api-access-z4rcb") pod "ac4f67cb-c9b2-4b8f-841c-4581692319b2" (UID: "ac4f67cb-c9b2-4b8f-841c-4581692319b2"). InnerVolumeSpecName "kube-api-access-z4rcb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:22:24.535863 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:24.535832    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac4f67cb-c9b2-4b8f-841c-4581692319b2-util" (OuterVolumeSpecName: "util") pod "ac4f67cb-c9b2-4b8f-841c-4581692319b2" (UID: "ac4f67cb-c9b2-4b8f-841c-4581692319b2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:22:24.631024 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:24.630922    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac4f67cb-c9b2-4b8f-841c-4581692319b2-util\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:22:24.631024 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:24.630968    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4rcb\" (UniqueName: \"kubernetes.io/projected/ac4f67cb-c9b2-4b8f-841c-4581692319b2-kube-api-access-z4rcb\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:22:24.631024 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:24.630980    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac4f67cb-c9b2-4b8f-841c-4581692319b2-bundle\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:22:25.354250 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:25.354224    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dnc7s"
Sep 15 10:22:25.354250 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:25.354232    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540dnc7s" event={"ID":"ac4f67cb-c9b2-4b8f-841c-4581692319b2","Type":"ContainerDied","Data":"2f5c4156d368b2de270d76f0222b720007cd85c718704b7f6a062ab2725dc4f2"}
Sep 15 10:22:25.354478 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:25.354275    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2f5c4156d368b2de270d76f0222b720007cd85c718704b7f6a062ab2725dc4f2"
Sep 15 10:22:27.449689 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:27.449653    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:22:28.814767 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:28.814735    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:22:29.391112 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:29.391078    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pmw2m"]
Sep 15 10:22:29.391353 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:29.391342    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac4f67cb-c9b2-4b8f-841c-4581692319b2" containerName="pull"
Sep 15 10:22:29.391400 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:29.391354    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac4f67cb-c9b2-4b8f-841c-4581692319b2" containerName="pull"
Sep 15 10:22:29.391400 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:29.391367    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac4f67cb-c9b2-4b8f-841c-4581692319b2" containerName="util"
Sep 15 10:22:29.391400 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:29.391373    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac4f67cb-c9b2-4b8f-841c-4581692319b2" containerName="util"
Sep 15 10:22:29.391400 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:29.391384    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac4f67cb-c9b2-4b8f-841c-4581692319b2" containerName="extract"
Sep 15 10:22:29.391400 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:29.391389    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac4f67cb-c9b2-4b8f-841c-4581692319b2" containerName="extract"
Sep 15 10:22:29.391586 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:29.391438    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac4f67cb-c9b2-4b8f-841c-4581692319b2" containerName="extract"
Sep 15 10:22:29.394154 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:29.394139    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pmw2m"
Sep 15 10:22:29.396681 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:29.396657    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-9gnqr\""
Sep 15 10:22:29.396913 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:29.396895    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 15 10:22:29.397023 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:29.397009    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 15 10:22:29.409208 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:29.409185    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pmw2m"]
Sep 15 10:22:29.468701 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:29.468664    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pphx7\" (UniqueName: \"kubernetes.io/projected/ffbd53f8-efdc-49a7-a242-f396f51a6777-kube-api-access-pphx7\") pod \"cert-manager-operator-controller-manager-55c444bf86-pmw2m\" (UID: \"ffbd53f8-efdc-49a7-a242-f396f51a6777\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pmw2m"
Sep 15 10:22:29.468887 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:29.468721    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ffbd53f8-efdc-49a7-a242-f396f51a6777-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-pmw2m\" (UID: \"ffbd53f8-efdc-49a7-a242-f396f51a6777\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pmw2m"
Sep 15 10:22:29.569077 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:29.569036    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pphx7\" (UniqueName: \"kubernetes.io/projected/ffbd53f8-efdc-49a7-a242-f396f51a6777-kube-api-access-pphx7\") pod \"cert-manager-operator-controller-manager-55c444bf86-pmw2m\" (UID: \"ffbd53f8-efdc-49a7-a242-f396f51a6777\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pmw2m"
Sep 15 10:22:29.569283 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:29.569191    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ffbd53f8-efdc-49a7-a242-f396f51a6777-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-pmw2m\" (UID: \"ffbd53f8-efdc-49a7-a242-f396f51a6777\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pmw2m"
Sep 15 10:22:29.569612 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:29.569591    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ffbd53f8-efdc-49a7-a242-f396f51a6777-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-pmw2m\" (UID: \"ffbd53f8-efdc-49a7-a242-f396f51a6777\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pmw2m"
Sep 15 10:22:29.580698 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:29.580677    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pphx7\" (UniqueName: \"kubernetes.io/projected/ffbd53f8-efdc-49a7-a242-f396f51a6777-kube-api-access-pphx7\") pod \"cert-manager-operator-controller-manager-55c444bf86-pmw2m\" (UID: \"ffbd53f8-efdc-49a7-a242-f396f51a6777\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pmw2m"
Sep 15 10:22:29.703277 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:29.703201    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pmw2m"
Sep 15 10:22:29.976325 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:29.976233    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pmw2m"]
Sep 15 10:22:29.979547 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:22:29.979522    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffbd53f8_efdc_49a7_a242_f396f51a6777.slice/crio-06a7750504e8fce8e25879895cd14414a8532c164b7eaf45137f50cdd7573b5e WatchSource:0}: Error finding container 06a7750504e8fce8e25879895cd14414a8532c164b7eaf45137f50cdd7573b5e: Status 404 returned error can't find the container with id 06a7750504e8fce8e25879895cd14414a8532c164b7eaf45137f50cdd7573b5e
Sep 15 10:22:30.373695 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:30.373606    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pmw2m" event={"ID":"ffbd53f8-efdc-49a7-a242-f396f51a6777","Type":"ContainerStarted","Data":"06a7750504e8fce8e25879895cd14414a8532c164b7eaf45137f50cdd7573b5e"}
Sep 15 10:22:31.062930 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:31.062892    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:22:31.163619 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:31.163588    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:22:32.384749 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:32.384715    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pmw2m" event={"ID":"ffbd53f8-efdc-49a7-a242-f396f51a6777","Type":"ContainerStarted","Data":"d1a5f5e799b78b2017d595b010323d085abee17f6bb8652a27b8db2fa89596ef"}
Sep 15 10:22:32.405974 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:32.405920    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pmw2m" podStartSLOduration=1.482933383 podStartE2EDuration="3.405904122s" totalImagesPullingTime="1.922970739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:22:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:22:32.404045556 +0000 UTC m=+267.651012525" watchObservedRunningTime="2026-09-15 10:22:32.405904122 +0000 UTC m=+267.652871093"
Sep 15 10:22:32.758087 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:32.757978    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:22:32.884922 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:32.883761    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:22:34.068129 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:34.068097    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpl4lx"]
Sep 15 10:22:34.071643 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:34.071625    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpl4lx"
Sep 15 10:22:34.074055 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:34.074028    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 15 10:22:34.074055 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:34.074049    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-c9n8v\""
Sep 15 10:22:34.074231 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:34.074025    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 15 10:22:34.079595 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:34.079573    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpl4lx"]
Sep 15 10:22:34.101733 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:34.101703    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90305e97-2c92-4512-968d-0fb798ebe676-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpl4lx\" (UID: \"90305e97-2c92-4512-968d-0fb798ebe676\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpl4lx"
Sep 15 10:22:34.101733 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:34.101733    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90305e97-2c92-4512-968d-0fb798ebe676-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpl4lx\" (UID: \"90305e97-2c92-4512-968d-0fb798ebe676\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpl4lx"
Sep 15 10:22:34.101939 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:34.101756    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p697b\" (UniqueName: \"kubernetes.io/projected/90305e97-2c92-4512-968d-0fb798ebe676-kube-api-access-p697b\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpl4lx\" (UID: \"90305e97-2c92-4512-968d-0fb798ebe676\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpl4lx"
Sep 15 10:22:34.202818 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:34.202785    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90305e97-2c92-4512-968d-0fb798ebe676-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpl4lx\" (UID: \"90305e97-2c92-4512-968d-0fb798ebe676\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpl4lx"
Sep 15 10:22:34.202818 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:34.202819    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90305e97-2c92-4512-968d-0fb798ebe676-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpl4lx\" (UID: \"90305e97-2c92-4512-968d-0fb798ebe676\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpl4lx"
Sep 15 10:22:34.202971 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:34.202848    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p697b\" (UniqueName: \"kubernetes.io/projected/90305e97-2c92-4512-968d-0fb798ebe676-kube-api-access-p697b\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpl4lx\" (UID: \"90305e97-2c92-4512-968d-0fb798ebe676\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpl4lx"
Sep 15 10:22:34.203185 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:34.203166    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90305e97-2c92-4512-968d-0fb798ebe676-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpl4lx\" (UID: \"90305e97-2c92-4512-968d-0fb798ebe676\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpl4lx"
Sep 15 10:22:34.203234 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:34.203204    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90305e97-2c92-4512-968d-0fb798ebe676-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpl4lx\" (UID: \"90305e97-2c92-4512-968d-0fb798ebe676\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpl4lx"
Sep 15 10:22:34.210330 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:34.210302    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p697b\" (UniqueName: \"kubernetes.io/projected/90305e97-2c92-4512-968d-0fb798ebe676-kube-api-access-p697b\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpl4lx\" (UID: \"90305e97-2c92-4512-968d-0fb798ebe676\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpl4lx"
Sep 15 10:22:34.381766 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:34.381665    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpl4lx"
Sep 15 10:22:34.650305 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:34.650276    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpl4lx"]
Sep 15 10:22:34.653605 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:22:34.653580    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90305e97_2c92_4512_968d_0fb798ebe676.slice/crio-e9d3f5a95e4c1c1b88cc01827e0c79f04aed7679e79fdf7422cea95d8b3b2f49 WatchSource:0}: Error finding container e9d3f5a95e4c1c1b88cc01827e0c79f04aed7679e79fdf7422cea95d8b3b2f49: Status 404 returned error can't find the container with id e9d3f5a95e4c1c1b88cc01827e0c79f04aed7679e79fdf7422cea95d8b3b2f49
Sep 15 10:22:35.117012 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:35.116914    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:22:35.398211 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:35.398179    2583 generic.go:356] "Generic (PLEG): container finished" podID="90305e97-2c92-4512-968d-0fb798ebe676" containerID="fe3212c507ace40d15e77939b5beaa1d690b0a51ebc956c001cef46e2b36e3bd" exitCode=0
Sep 15 10:22:35.398363 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:35.398264    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpl4lx" event={"ID":"90305e97-2c92-4512-968d-0fb798ebe676","Type":"ContainerDied","Data":"fe3212c507ace40d15e77939b5beaa1d690b0a51ebc956c001cef46e2b36e3bd"}
Sep 15 10:22:35.398363 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:35.398309    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpl4lx" event={"ID":"90305e97-2c92-4512-968d-0fb798ebe676","Type":"ContainerStarted","Data":"e9d3f5a95e4c1c1b88cc01827e0c79f04aed7679e79fdf7422cea95d8b3b2f49"}
Sep 15 10:22:35.503804 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:35.503771    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-j8mrd"]
Sep 15 10:22:35.507008 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:35.506983    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-j8mrd"
Sep 15 10:22:35.509191 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:35.509167    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 15 10:22:35.509332 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:35.509194    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-mmx8k\""
Sep 15 10:22:35.509332 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:35.509279    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 15 10:22:35.514345 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:35.514321    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-j8mrd"]
Sep 15 10:22:35.566210 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:35.566180    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:22:35.615713 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:35.615675    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqxkn\" (UniqueName: \"kubernetes.io/projected/3a0d4985-911a-4973-921f-f7ac7f300696-kube-api-access-gqxkn\") pod \"cert-manager-cainjector-559cdd8c87-j8mrd\" (UID: \"3a0d4985-911a-4973-921f-f7ac7f300696\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-j8mrd"
Sep 15 10:22:35.615891 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:35.615741    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3a0d4985-911a-4973-921f-f7ac7f300696-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-j8mrd\" (UID: \"3a0d4985-911a-4973-921f-f7ac7f300696\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-j8mrd"
Sep 15 10:22:35.717177 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:35.717071    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqxkn\" (UniqueName: \"kubernetes.io/projected/3a0d4985-911a-4973-921f-f7ac7f300696-kube-api-access-gqxkn\") pod \"cert-manager-cainjector-559cdd8c87-j8mrd\" (UID: \"3a0d4985-911a-4973-921f-f7ac7f300696\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-j8mrd"
Sep 15 10:22:35.717341 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:35.717196    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3a0d4985-911a-4973-921f-f7ac7f300696-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-j8mrd\" (UID: \"3a0d4985-911a-4973-921f-f7ac7f300696\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-j8mrd"
Sep 15 10:22:35.725128 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:35.725102    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqxkn\" (UniqueName: \"kubernetes.io/projected/3a0d4985-911a-4973-921f-f7ac7f300696-kube-api-access-gqxkn\") pod \"cert-manager-cainjector-559cdd8c87-j8mrd\" (UID: \"3a0d4985-911a-4973-921f-f7ac7f300696\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-j8mrd"
Sep 15 10:22:35.725128 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:35.725116    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3a0d4985-911a-4973-921f-f7ac7f300696-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-j8mrd\" (UID: \"3a0d4985-911a-4973-921f-f7ac7f300696\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-j8mrd"
Sep 15 10:22:35.823819 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:35.823791    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-j8mrd"
Sep 15 10:22:36.099157 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:36.099035    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-j8mrd"]
Sep 15 10:22:36.102339 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:22:36.102301    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3a0d4985_911a_4973_921f_f7ac7f300696.slice/crio-88ff5c44ddf979012c4dc1e634787aa0ef7e6344f7d1ee9c8e25bdf9b9d31d0e WatchSource:0}: Error finding container 88ff5c44ddf979012c4dc1e634787aa0ef7e6344f7d1ee9c8e25bdf9b9d31d0e: Status 404 returned error can't find the container with id 88ff5c44ddf979012c4dc1e634787aa0ef7e6344f7d1ee9c8e25bdf9b9d31d0e
Sep 15 10:22:36.405879 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:36.405838    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-j8mrd" event={"ID":"3a0d4985-911a-4973-921f-f7ac7f300696","Type":"ContainerStarted","Data":"88ff5c44ddf979012c4dc1e634787aa0ef7e6344f7d1ee9c8e25bdf9b9d31d0e"}
Sep 15 10:22:37.414567 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:37.414524    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpl4lx" event={"ID":"90305e97-2c92-4512-968d-0fb798ebe676","Type":"ContainerStarted","Data":"737136eae1195e9a61a2b4ab3ed63250f9dc94ff9c5bdb9aca0784f5f76dc051"}
Sep 15 10:22:37.875648 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:37.875612    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:22:37.933011 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:37.932977    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:22:38.422296 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:38.422257    2583 generic.go:356] "Generic (PLEG): container finished" podID="90305e97-2c92-4512-968d-0fb798ebe676" containerID="737136eae1195e9a61a2b4ab3ed63250f9dc94ff9c5bdb9aca0784f5f76dc051" exitCode=0
Sep 15 10:22:38.422777 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:38.422344    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpl4lx" event={"ID":"90305e97-2c92-4512-968d-0fb798ebe676","Type":"ContainerDied","Data":"737136eae1195e9a61a2b4ab3ed63250f9dc94ff9c5bdb9aca0784f5f76dc051"}
Sep 15 10:22:39.428962 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:39.428865    2583 generic.go:356] "Generic (PLEG): container finished" podID="90305e97-2c92-4512-968d-0fb798ebe676" containerID="f9ac1a6a7e4bc1d5d27276b46d2a3f8c414fea7c746f1cf241fd529517e7002d" exitCode=0
Sep 15 10:22:39.429409 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:39.428957    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpl4lx" event={"ID":"90305e97-2c92-4512-968d-0fb798ebe676","Type":"ContainerDied","Data":"f9ac1a6a7e4bc1d5d27276b46d2a3f8c414fea7c746f1cf241fd529517e7002d"}
Sep 15 10:22:39.430655 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:39.430631    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-j8mrd" event={"ID":"3a0d4985-911a-4973-921f-f7ac7f300696","Type":"ContainerStarted","Data":"2d5f4971fecadff6b139d4f4a366ffd502fd344985beab661a874c36ad9b010f"}
Sep 15 10:22:39.464191 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:39.464139    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-j8mrd" podStartSLOduration=1.3501816770000001 podStartE2EDuration="4.464122879s" totalImagesPullingTime="3.113941202s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:22:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:22:39.463371203 +0000 UTC m=+274.710338173" watchObservedRunningTime="2026-09-15 10:22:39.464122879 +0000 UTC m=+274.711089849"
Sep 15 10:22:40.558832 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:40.558808    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpl4lx"
Sep 15 10:22:40.657754 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:40.657715    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90305e97-2c92-4512-968d-0fb798ebe676-util\") pod \"90305e97-2c92-4512-968d-0fb798ebe676\" (UID: \"90305e97-2c92-4512-968d-0fb798ebe676\") "
Sep 15 10:22:40.657912 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:40.657765    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90305e97-2c92-4512-968d-0fb798ebe676-bundle\") pod \"90305e97-2c92-4512-968d-0fb798ebe676\" (UID: \"90305e97-2c92-4512-968d-0fb798ebe676\") "
Sep 15 10:22:40.657912 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:40.657821    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p697b\" (UniqueName: \"kubernetes.io/projected/90305e97-2c92-4512-968d-0fb798ebe676-kube-api-access-p697b\") pod \"90305e97-2c92-4512-968d-0fb798ebe676\" (UID: \"90305e97-2c92-4512-968d-0fb798ebe676\") "
Sep 15 10:22:40.658244 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:40.658215    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90305e97-2c92-4512-968d-0fb798ebe676-bundle" (OuterVolumeSpecName: "bundle") pod "90305e97-2c92-4512-968d-0fb798ebe676" (UID: "90305e97-2c92-4512-968d-0fb798ebe676"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:22:40.659927 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:40.659903    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90305e97-2c92-4512-968d-0fb798ebe676-kube-api-access-p697b" (OuterVolumeSpecName: "kube-api-access-p697b") pod "90305e97-2c92-4512-968d-0fb798ebe676" (UID: "90305e97-2c92-4512-968d-0fb798ebe676"). InnerVolumeSpecName "kube-api-access-p697b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:22:40.662584 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:40.662564    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90305e97-2c92-4512-968d-0fb798ebe676-util" (OuterVolumeSpecName: "util") pod "90305e97-2c92-4512-968d-0fb798ebe676" (UID: "90305e97-2c92-4512-968d-0fb798ebe676"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:22:40.758424 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:40.758355    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p697b\" (UniqueName: \"kubernetes.io/projected/90305e97-2c92-4512-968d-0fb798ebe676-kube-api-access-p697b\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:22:40.758424 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:40.758380    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90305e97-2c92-4512-968d-0fb798ebe676-util\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:22:40.758424 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:40.758390    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90305e97-2c92-4512-968d-0fb798ebe676-bundle\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:22:41.085696 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:41.085603    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-48pf4"]
Sep 15 10:22:41.086269 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:41.086243    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90305e97-2c92-4512-968d-0fb798ebe676" containerName="extract"
Sep 15 10:22:41.086269 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:41.086269    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="90305e97-2c92-4512-968d-0fb798ebe676" containerName="extract"
Sep 15 10:22:41.086524 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:41.086312    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90305e97-2c92-4512-968d-0fb798ebe676" containerName="util"
Sep 15 10:22:41.086524 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:41.086322    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="90305e97-2c92-4512-968d-0fb798ebe676" containerName="util"
Sep 15 10:22:41.086524 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:41.086338    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90305e97-2c92-4512-968d-0fb798ebe676" containerName="pull"
Sep 15 10:22:41.086524 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:41.086347    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="90305e97-2c92-4512-968d-0fb798ebe676" containerName="pull"
Sep 15 10:22:41.086720 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:41.086541    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="90305e97-2c92-4512-968d-0fb798ebe676" containerName="extract"
Sep 15 10:22:41.091046 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:41.091019    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-48pf4"
Sep 15 10:22:41.093341 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:41.093322    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-fk69m\""
Sep 15 10:22:41.098253 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:41.098089    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-48pf4"]
Sep 15 10:22:41.162921 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:41.162889    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9f88\" (UniqueName: \"kubernetes.io/projected/a088d6b5-d7c8-422c-a6a6-f26935c5129c-kube-api-access-l9f88\") pod \"cert-manager-webhook-6486ffd997-48pf4\" (UID: \"a088d6b5-d7c8-422c-a6a6-f26935c5129c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-48pf4"
Sep 15 10:22:41.163090 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:41.162931    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a088d6b5-d7c8-422c-a6a6-f26935c5129c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-48pf4\" (UID: \"a088d6b5-d7c8-422c-a6a6-f26935c5129c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-48pf4"
Sep 15 10:22:41.263967 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:41.263929    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a088d6b5-d7c8-422c-a6a6-f26935c5129c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-48pf4\" (UID: \"a088d6b5-d7c8-422c-a6a6-f26935c5129c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-48pf4"
Sep 15 10:22:41.264111 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:41.264007    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9f88\" (UniqueName: \"kubernetes.io/projected/a088d6b5-d7c8-422c-a6a6-f26935c5129c-kube-api-access-l9f88\") pod \"cert-manager-webhook-6486ffd997-48pf4\" (UID: \"a088d6b5-d7c8-422c-a6a6-f26935c5129c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-48pf4"
Sep 15 10:22:41.272112 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:41.272091    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a088d6b5-d7c8-422c-a6a6-f26935c5129c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-48pf4\" (UID: \"a088d6b5-d7c8-422c-a6a6-f26935c5129c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-48pf4"
Sep 15 10:22:41.272210 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:41.272163    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9f88\" (UniqueName: \"kubernetes.io/projected/a088d6b5-d7c8-422c-a6a6-f26935c5129c-kube-api-access-l9f88\") pod \"cert-manager-webhook-6486ffd997-48pf4\" (UID: \"a088d6b5-d7c8-422c-a6a6-f26935c5129c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-48pf4"
Sep 15 10:22:41.401621 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:41.401591    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-48pf4"
Sep 15 10:22:41.443307 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:41.443277    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpl4lx" event={"ID":"90305e97-2c92-4512-968d-0fb798ebe676","Type":"ContainerDied","Data":"e9d3f5a95e4c1c1b88cc01827e0c79f04aed7679e79fdf7422cea95d8b3b2f49"}
Sep 15 10:22:41.443506 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:41.443319    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpl4lx"
Sep 15 10:22:41.443506 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:41.443320    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e9d3f5a95e4c1c1b88cc01827e0c79f04aed7679e79fdf7422cea95d8b3b2f49"
Sep 15 10:22:41.672509 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:41.672361    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-48pf4"]
Sep 15 10:22:41.674780 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:22:41.674744    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda088d6b5_d7c8_422c_a6a6_f26935c5129c.slice/crio-a2da7cf4ceed045774d74fab78fd0f1f536d93a812002a527d2c7a02b86d9ed4 WatchSource:0}: Error finding container a2da7cf4ceed045774d74fab78fd0f1f536d93a812002a527d2c7a02b86d9ed4: Status 404 returned error can't find the container with id a2da7cf4ceed045774d74fab78fd0f1f536d93a812002a527d2c7a02b86d9ed4
Sep 15 10:22:42.448904 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:42.448872    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-48pf4" event={"ID":"a088d6b5-d7c8-422c-a6a6-f26935c5129c","Type":"ContainerStarted","Data":"96c51d8812c8286fbf1fc7c096a1c87eb584977347d76cdb7520328c38686b54"}
Sep 15 10:22:42.449070 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:42.448913    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-48pf4"
Sep 15 10:22:42.449070 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:42.448924    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-48pf4" event={"ID":"a088d6b5-d7c8-422c-a6a6-f26935c5129c","Type":"ContainerStarted","Data":"a2da7cf4ceed045774d74fab78fd0f1f536d93a812002a527d2c7a02b86d9ed4"}
Sep 15 10:22:42.465565 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:42.465519    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-48pf4" podStartSLOduration=1.465502179 podStartE2EDuration="1.465502179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:22:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:22:42.464303416 +0000 UTC m=+277.711270385" watchObservedRunningTime="2026-09-15 10:22:42.465502179 +0000 UTC m=+277.712469148"
Sep 15 10:22:42.526778 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:42.526747    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:22:44.145849 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:44.145818    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-hhfm6"]
Sep 15 10:22:44.149253 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:44.149237    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-hhfm6"
Sep 15 10:22:44.151230 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:44.151213    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-zjwln\""
Sep 15 10:22:44.155894 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:44.155750    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-hhfm6"]
Sep 15 10:22:44.292552 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:44.292518    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5068989f-4e18-4994-ae3d-2db5ce8540e1-bound-sa-token\") pod \"cert-manager-55b4d954d5-hhfm6\" (UID: \"5068989f-4e18-4994-ae3d-2db5ce8540e1\") " pod="cert-manager/cert-manager-55b4d954d5-hhfm6"
Sep 15 10:22:44.292742 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:44.292582    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ckcm\" (UniqueName: \"kubernetes.io/projected/5068989f-4e18-4994-ae3d-2db5ce8540e1-kube-api-access-9ckcm\") pod \"cert-manager-55b4d954d5-hhfm6\" (UID: \"5068989f-4e18-4994-ae3d-2db5ce8540e1\") " pod="cert-manager/cert-manager-55b4d954d5-hhfm6"
Sep 15 10:22:44.393223 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:44.393174    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ckcm\" (UniqueName: \"kubernetes.io/projected/5068989f-4e18-4994-ae3d-2db5ce8540e1-kube-api-access-9ckcm\") pod \"cert-manager-55b4d954d5-hhfm6\" (UID: \"5068989f-4e18-4994-ae3d-2db5ce8540e1\") " pod="cert-manager/cert-manager-55b4d954d5-hhfm6"
Sep 15 10:22:44.393367 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:44.393262    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5068989f-4e18-4994-ae3d-2db5ce8540e1-bound-sa-token\") pod \"cert-manager-55b4d954d5-hhfm6\" (UID: \"5068989f-4e18-4994-ae3d-2db5ce8540e1\") " pod="cert-manager/cert-manager-55b4d954d5-hhfm6"
Sep 15 10:22:44.401262 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:44.401195    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5068989f-4e18-4994-ae3d-2db5ce8540e1-bound-sa-token\") pod \"cert-manager-55b4d954d5-hhfm6\" (UID: \"5068989f-4e18-4994-ae3d-2db5ce8540e1\") " pod="cert-manager/cert-manager-55b4d954d5-hhfm6"
Sep 15 10:22:44.401369 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:44.401284    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ckcm\" (UniqueName: \"kubernetes.io/projected/5068989f-4e18-4994-ae3d-2db5ce8540e1-kube-api-access-9ckcm\") pod \"cert-manager-55b4d954d5-hhfm6\" (UID: \"5068989f-4e18-4994-ae3d-2db5ce8540e1\") " pod="cert-manager/cert-manager-55b4d954d5-hhfm6"
Sep 15 10:22:44.458555 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:44.458527    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-hhfm6"
Sep 15 10:22:44.727717 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:44.727694    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-hhfm6"]
Sep 15 10:22:44.730084 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:22:44.730061    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5068989f_4e18_4994_ae3d_2db5ce8540e1.slice/crio-e23d7f207e698c7b5808f137ca088e55ef1ca506ed7c0082a90eaf84654d7937 WatchSource:0}: Error finding container e23d7f207e698c7b5808f137ca088e55ef1ca506ed7c0082a90eaf84654d7937: Status 404 returned error can't find the container with id e23d7f207e698c7b5808f137ca088e55ef1ca506ed7c0082a90eaf84654d7937
Sep 15 10:22:45.462896 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:45.462860    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-hhfm6" event={"ID":"5068989f-4e18-4994-ae3d-2db5ce8540e1","Type":"ContainerStarted","Data":"5ce3a09dfb6f7c53f2a8362d1484426b61cee99cd651a2c60bd3c934ea6c388e"}
Sep 15 10:22:45.462896 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:45.462904    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-hhfm6" event={"ID":"5068989f-4e18-4994-ae3d-2db5ce8540e1","Type":"ContainerStarted","Data":"e23d7f207e698c7b5808f137ca088e55ef1ca506ed7c0082a90eaf84654d7937"}
Sep 15 10:22:45.478351 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:45.478302    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-hhfm6" podStartSLOduration=1.478287029 podStartE2EDuration="1.478287029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:22:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:22:45.476814067 +0000 UTC m=+280.723781038" watchObservedRunningTime="2026-09-15 10:22:45.478287029 +0000 UTC m=+280.725253998"
Sep 15 10:22:47.485046 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:47.485012    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:22:47.585828 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:47.585793    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:22:48.455061 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:48.455027    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-48pf4"
Sep 15 10:22:49.198346 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:49.198316    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g77dt"]
Sep 15 10:22:49.202777 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:49.202760    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g77dt"
Sep 15 10:22:49.205145 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:49.205114    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 15 10:22:49.205145 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:49.205124    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 15 10:22:49.206220 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:49.206204    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-qjcs6\""
Sep 15 10:22:49.211287 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:49.211269    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g77dt"]
Sep 15 10:22:49.336118 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:49.336082    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vddmc\" (UniqueName: \"kubernetes.io/projected/6d971aa8-dd09-44d1-a61e-d89311b49f2f-kube-api-access-vddmc\") pod \"openshift-lws-operator-fd8ccff4c-g77dt\" (UID: \"6d971aa8-dd09-44d1-a61e-d89311b49f2f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g77dt"
Sep 15 10:22:49.336293 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:49.336126    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d971aa8-dd09-44d1-a61e-d89311b49f2f-tmp\") pod \"openshift-lws-operator-fd8ccff4c-g77dt\" (UID: \"6d971aa8-dd09-44d1-a61e-d89311b49f2f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g77dt"
Sep 15 10:22:49.437139 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:49.437098    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d971aa8-dd09-44d1-a61e-d89311b49f2f-tmp\") pod \"openshift-lws-operator-fd8ccff4c-g77dt\" (UID: \"6d971aa8-dd09-44d1-a61e-d89311b49f2f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g77dt"
Sep 15 10:22:49.437301 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:49.437182    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vddmc\" (UniqueName: \"kubernetes.io/projected/6d971aa8-dd09-44d1-a61e-d89311b49f2f-kube-api-access-vddmc\") pod \"openshift-lws-operator-fd8ccff4c-g77dt\" (UID: \"6d971aa8-dd09-44d1-a61e-d89311b49f2f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g77dt"
Sep 15 10:22:49.437540 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:49.437521    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d971aa8-dd09-44d1-a61e-d89311b49f2f-tmp\") pod \"openshift-lws-operator-fd8ccff4c-g77dt\" (UID: \"6d971aa8-dd09-44d1-a61e-d89311b49f2f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g77dt"
Sep 15 10:22:49.444372 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:49.444351    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vddmc\" (UniqueName: \"kubernetes.io/projected/6d971aa8-dd09-44d1-a61e-d89311b49f2f-kube-api-access-vddmc\") pod \"openshift-lws-operator-fd8ccff4c-g77dt\" (UID: \"6d971aa8-dd09-44d1-a61e-d89311b49f2f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g77dt"
Sep 15 10:22:49.513054 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:49.513006    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g77dt"
Sep 15 10:22:49.780040 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:49.780017    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g77dt"]
Sep 15 10:22:49.782320 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:22:49.782291    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6d971aa8_dd09_44d1_a61e_d89311b49f2f.slice/crio-f809ec508fe965f8506a40ff7dd3891238acac343e09959adf2440559e74752e WatchSource:0}: Error finding container f809ec508fe965f8506a40ff7dd3891238acac343e09959adf2440559e74752e: Status 404 returned error can't find the container with id f809ec508fe965f8506a40ff7dd3891238acac343e09959adf2440559e74752e
Sep 15 10:22:50.485602 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:50.485566    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g77dt" event={"ID":"6d971aa8-dd09-44d1-a61e-d89311b49f2f","Type":"ContainerStarted","Data":"f809ec508fe965f8506a40ff7dd3891238acac343e09959adf2440559e74752e"}
Sep 15 10:22:51.492677 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:51.492580    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g77dt" event={"ID":"6d971aa8-dd09-44d1-a61e-d89311b49f2f","Type":"ContainerStarted","Data":"926f345c052c476211924b02fa13ac5c256469c8a184f664db12a71a6e2cd4a0"}
Sep 15 10:22:51.509180 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:51.509124    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g77dt" podStartSLOduration=1.066191618 podStartE2EDuration="2.509110989s" totalImagesPullingTime="1.442919371s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:22:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:22:51.507018299 +0000 UTC m=+286.753985269" watchObservedRunningTime="2026-09-15 10:22:51.509110989 +0000 UTC m=+286.756077959"
Sep 15 10:22:52.189576 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:52.189543    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:22:53.459126 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:53.459099    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:22:54.214802 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:54.214769    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:22:54.288274 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:54.288243    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6vrgt"]
Sep 15 10:22:54.332000 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:54.331973    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6vrgt"]
Sep 15 10:22:54.332140 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:54.332072    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6vrgt"
Sep 15 10:22:54.334489 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:54.334462    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-c9n8v\""
Sep 15 10:22:54.334607 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:54.334495    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 15 10:22:54.335133 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:54.335110    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 15 10:22:54.480796 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:54.480718    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgfr9\" (UniqueName: \"kubernetes.io/projected/ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8-kube-api-access-fgfr9\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6vrgt\" (UID: \"ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6vrgt"
Sep 15 10:22:54.480796 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:54.480770    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6vrgt\" (UID: \"ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6vrgt"
Sep 15 10:22:54.481214 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:54.480875    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6vrgt\" (UID: \"ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6vrgt"
Sep 15 10:22:54.581752 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:54.581715    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgfr9\" (UniqueName: \"kubernetes.io/projected/ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8-kube-api-access-fgfr9\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6vrgt\" (UID: \"ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6vrgt"
Sep 15 10:22:54.581904 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:54.581773    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6vrgt\" (UID: \"ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6vrgt"
Sep 15 10:22:54.581904 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:54.581818    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6vrgt\" (UID: \"ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6vrgt"
Sep 15 10:22:54.582174 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:54.582157    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6vrgt\" (UID: \"ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6vrgt"
Sep 15 10:22:54.582211 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:54.582186    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6vrgt\" (UID: \"ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6vrgt"
Sep 15 10:22:54.589208 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:54.589181    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgfr9\" (UniqueName: \"kubernetes.io/projected/ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8-kube-api-access-fgfr9\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6vrgt\" (UID: \"ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6vrgt"
Sep 15 10:22:54.641457 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:54.641418    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6vrgt"
Sep 15 10:22:54.922179 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:54.922154    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6vrgt"]
Sep 15 10:22:54.923915 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:22:54.923889    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee5c4a02_73f5_4cc8_b4e3_7efcaf6b32e8.slice/crio-7b10c005f73a0e70c9f16e6a150e2573c8ec8a788ec33ae1b73291c4597796cc WatchSource:0}: Error finding container 7b10c005f73a0e70c9f16e6a150e2573c8ec8a788ec33ae1b73291c4597796cc: Status 404 returned error can't find the container with id 7b10c005f73a0e70c9f16e6a150e2573c8ec8a788ec33ae1b73291c4597796cc
Sep 15 10:22:55.510930 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:55.510891    2583 generic.go:356] "Generic (PLEG): container finished" podID="ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8" containerID="5c61e33490d239a3853431e760db151a8baf74a05b98eeff4a1aeb5a84bb5ae7" exitCode=0
Sep 15 10:22:55.511340 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:55.510976    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6vrgt" event={"ID":"ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8","Type":"ContainerDied","Data":"5c61e33490d239a3853431e760db151a8baf74a05b98eeff4a1aeb5a84bb5ae7"}
Sep 15 10:22:55.511340 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:55.511022    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6vrgt" event={"ID":"ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8","Type":"ContainerStarted","Data":"7b10c005f73a0e70c9f16e6a150e2573c8ec8a788ec33ae1b73291c4597796cc"}
Sep 15 10:22:57.521388 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:57.521354    2583 generic.go:356] "Generic (PLEG): container finished" podID="ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8" containerID="95d0c3999ad8647dd46c66a9b46928dab77dbd9d061aec8d4e4e17e1e06dd97c" exitCode=0
Sep 15 10:22:57.521776 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:57.521439    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6vrgt" event={"ID":"ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8","Type":"ContainerDied","Data":"95d0c3999ad8647dd46c66a9b46928dab77dbd9d061aec8d4e4e17e1e06dd97c"}
Sep 15 10:22:58.528394 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:58.528362    2583 generic.go:356] "Generic (PLEG): container finished" podID="ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8" containerID="1a1151c8c5e7d2a7546dc71c5925962815acea739b653156baf3a68d3959ce43" exitCode=0
Sep 15 10:22:58.528807 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:58.528420    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6vrgt" event={"ID":"ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8","Type":"ContainerDied","Data":"1a1151c8c5e7d2a7546dc71c5925962815acea739b653156baf3a68d3959ce43"}
Sep 15 10:22:59.652046 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:59.652022    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6vrgt"
Sep 15 10:22:59.827003 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:59.826918    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fgfr9\" (UniqueName: \"kubernetes.io/projected/ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8-kube-api-access-fgfr9\") pod \"ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8\" (UID: \"ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8\") "
Sep 15 10:22:59.827003 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:59.826956    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8-util\") pod \"ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8\" (UID: \"ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8\") "
Sep 15 10:22:59.827003 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:59.826983    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8-bundle\") pod \"ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8\" (UID: \"ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8\") "
Sep 15 10:22:59.827734 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:59.827703    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8-bundle" (OuterVolumeSpecName: "bundle") pod "ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8" (UID: "ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:22:59.829002 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:59.828981    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8-kube-api-access-fgfr9" (OuterVolumeSpecName: "kube-api-access-fgfr9") pod "ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8" (UID: "ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8"). InnerVolumeSpecName "kube-api-access-fgfr9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:22:59.832398 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:59.832371    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8-util" (OuterVolumeSpecName: "util") pod "ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8" (UID: "ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:22:59.928041 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:59.928008    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgfr9\" (UniqueName: \"kubernetes.io/projected/ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8-kube-api-access-fgfr9\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:22:59.928041 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:59.928037    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8-util\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:22:59.928041 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:22:59.928049    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8-bundle\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:23:00.539713 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:00.539684    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6vrgt"
Sep 15 10:23:00.539882 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:00.539685    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6vrgt" event={"ID":"ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8","Type":"ContainerDied","Data":"7b10c005f73a0e70c9f16e6a150e2573c8ec8a788ec33ae1b73291c4597796cc"}
Sep 15 10:23:00.539882 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:00.539798    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7b10c005f73a0e70c9f16e6a150e2573c8ec8a788ec33ae1b73291c4597796cc"
Sep 15 10:23:03.323384 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:03.323352    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:03.425267 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:03.425234    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:05.189888 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:05.189860    2583 kubelet.go:1648] "Image garbage collection succeeded"
Sep 15 10:23:08.426584 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:08.426551    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:08.483196 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:08.483168    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-85d6978c87-9pkz6"]
Sep 15 10:23:08.483474 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:08.483463    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8" containerName="pull"
Sep 15 10:23:08.483522 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:08.483476    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8" containerName="pull"
Sep 15 10:23:08.483522 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:08.483493    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8" containerName="util"
Sep 15 10:23:08.483522 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:08.483498    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8" containerName="util"
Sep 15 10:23:08.483522 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:08.483507    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8" containerName="extract"
Sep 15 10:23:08.483522 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:08.483512    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8" containerName="extract"
Sep 15 10:23:08.483672 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:08.483560    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee5c4a02-73f5-4cc8-b4e3-7efcaf6b32e8" containerName="extract"
Sep 15 10:23:08.486265 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:08.486251    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-85d6978c87-9pkz6"
Sep 15 10:23:08.488701 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:08.488684    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 15 10:23:08.489603 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:08.489588    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 15 10:23:08.489689 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:08.489628    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-cdltk\""
Sep 15 10:23:08.489729 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:08.489684    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 15 10:23:08.495495 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:08.495468    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cd7d5\" (UniqueName: \"kubernetes.io/projected/eb75d5b6-1850-4fc3-9220-aa6d5d107380-kube-api-access-cd7d5\") pod \"lws-controller-manager-85d6978c87-9pkz6\" (UID: \"eb75d5b6-1850-4fc3-9220-aa6d5d107380\") " pod="openshift-lws-operator/lws-controller-manager-85d6978c87-9pkz6"
Sep 15 10:23:08.495610 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:08.495587    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eb75d5b6-1850-4fc3-9220-aa6d5d107380-cert\") pod \"lws-controller-manager-85d6978c87-9pkz6\" (UID: \"eb75d5b6-1850-4fc3-9220-aa6d5d107380\") " pod="openshift-lws-operator/lws-controller-manager-85d6978c87-9pkz6"
Sep 15 10:23:08.495673 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:08.495651    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/eb75d5b6-1850-4fc3-9220-aa6d5d107380-manager-config\") pod \"lws-controller-manager-85d6978c87-9pkz6\" (UID: \"eb75d5b6-1850-4fc3-9220-aa6d5d107380\") " pod="openshift-lws-operator/lws-controller-manager-85d6978c87-9pkz6"
Sep 15 10:23:08.495716 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:08.495684    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/eb75d5b6-1850-4fc3-9220-aa6d5d107380-metrics-cert\") pod \"lws-controller-manager-85d6978c87-9pkz6\" (UID: \"eb75d5b6-1850-4fc3-9220-aa6d5d107380\") " pod="openshift-lws-operator/lws-controller-manager-85d6978c87-9pkz6"
Sep 15 10:23:08.495759 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:08.495741    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-85d6978c87-9pkz6"]
Sep 15 10:23:08.499215 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:08.499192    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:08.596551 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:08.596527    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eb75d5b6-1850-4fc3-9220-aa6d5d107380-cert\") pod \"lws-controller-manager-85d6978c87-9pkz6\" (UID: \"eb75d5b6-1850-4fc3-9220-aa6d5d107380\") " pod="openshift-lws-operator/lws-controller-manager-85d6978c87-9pkz6"
Sep 15 10:23:08.596677 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:08.596564    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/eb75d5b6-1850-4fc3-9220-aa6d5d107380-manager-config\") pod \"lws-controller-manager-85d6978c87-9pkz6\" (UID: \"eb75d5b6-1850-4fc3-9220-aa6d5d107380\") " pod="openshift-lws-operator/lws-controller-manager-85d6978c87-9pkz6"
Sep 15 10:23:08.596677 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:08.596583    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/eb75d5b6-1850-4fc3-9220-aa6d5d107380-metrics-cert\") pod \"lws-controller-manager-85d6978c87-9pkz6\" (UID: \"eb75d5b6-1850-4fc3-9220-aa6d5d107380\") " pod="openshift-lws-operator/lws-controller-manager-85d6978c87-9pkz6"
Sep 15 10:23:08.596677 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:08.596614    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cd7d5\" (UniqueName: \"kubernetes.io/projected/eb75d5b6-1850-4fc3-9220-aa6d5d107380-kube-api-access-cd7d5\") pod \"lws-controller-manager-85d6978c87-9pkz6\" (UID: \"eb75d5b6-1850-4fc3-9220-aa6d5d107380\") " pod="openshift-lws-operator/lws-controller-manager-85d6978c87-9pkz6"
Sep 15 10:23:08.597249 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:08.597228    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/eb75d5b6-1850-4fc3-9220-aa6d5d107380-manager-config\") pod \"lws-controller-manager-85d6978c87-9pkz6\" (UID: \"eb75d5b6-1850-4fc3-9220-aa6d5d107380\") " pod="openshift-lws-operator/lws-controller-manager-85d6978c87-9pkz6"
Sep 15 10:23:08.599097 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:08.599068    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/eb75d5b6-1850-4fc3-9220-aa6d5d107380-metrics-cert\") pod \"lws-controller-manager-85d6978c87-9pkz6\" (UID: \"eb75d5b6-1850-4fc3-9220-aa6d5d107380\") " pod="openshift-lws-operator/lws-controller-manager-85d6978c87-9pkz6"
Sep 15 10:23:08.599191 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:08.599132    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eb75d5b6-1850-4fc3-9220-aa6d5d107380-cert\") pod \"lws-controller-manager-85d6978c87-9pkz6\" (UID: \"eb75d5b6-1850-4fc3-9220-aa6d5d107380\") " pod="openshift-lws-operator/lws-controller-manager-85d6978c87-9pkz6"
Sep 15 10:23:08.608007 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:08.607988    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cd7d5\" (UniqueName: \"kubernetes.io/projected/eb75d5b6-1850-4fc3-9220-aa6d5d107380-kube-api-access-cd7d5\") pod \"lws-controller-manager-85d6978c87-9pkz6\" (UID: \"eb75d5b6-1850-4fc3-9220-aa6d5d107380\") " pod="openshift-lws-operator/lws-controller-manager-85d6978c87-9pkz6"
Sep 15 10:23:08.795943 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:08.795880    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-85d6978c87-9pkz6"
Sep 15 10:23:09.069826 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:09.069804    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-85d6978c87-9pkz6"]
Sep 15 10:23:09.071802 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:23:09.071779    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb75d5b6_1850_4fc3_9220_aa6d5d107380.slice/crio-e6e1e2906b5b2b31250343217c9f485851a9fb7f0092ab33cdca7d7bca4e56b9 WatchSource:0}: Error finding container e6e1e2906b5b2b31250343217c9f485851a9fb7f0092ab33cdca7d7bca4e56b9: Status 404 returned error can't find the container with id e6e1e2906b5b2b31250343217c9f485851a9fb7f0092ab33cdca7d7bca4e56b9
Sep 15 10:23:09.073797 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:09.073782    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 10:23:09.580435 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:09.580397    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-85d6978c87-9pkz6" event={"ID":"eb75d5b6-1850-4fc3-9220-aa6d5d107380","Type":"ContainerStarted","Data":"e6e1e2906b5b2b31250343217c9f485851a9fb7f0092ab33cdca7d7bca4e56b9"}
Sep 15 10:23:10.586626 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:10.586589    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-85d6978c87-9pkz6" event={"ID":"eb75d5b6-1850-4fc3-9220-aa6d5d107380","Type":"ContainerStarted","Data":"30b7a7637e8013e185608db788610ea563636b4f1cab4f5c78c3d8db69367b69"}
Sep 15 10:23:10.587069 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:10.586710    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-85d6978c87-9pkz6"
Sep 15 10:23:10.603972 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:10.603923    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-85d6978c87-9pkz6" podStartSLOduration=1.233462597 podStartE2EDuration="2.603908997s" totalImagesPullingTime="1.3704464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:23:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:23:10.602869163 +0000 UTC m=+305.849836135" watchObservedRunningTime="2026-09-15 10:23:10.603908997 +0000 UTC m=+305.850876036"
Sep 15 10:23:11.115996 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:11.115960    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0whh82"]
Sep 15 10:23:11.119297 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:11.119280    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0whh82"
Sep 15 10:23:11.122301 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:11.122281    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-c9n8v\""
Sep 15 10:23:11.122426 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:11.122332    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 15 10:23:11.122426 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:11.122371    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 15 10:23:11.129246 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:11.129216    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0whh82"]
Sep 15 10:23:11.217836 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:11.217806    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10c9844d-e6f7-491d-bae5-39368ee16876-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0whh82\" (UID: \"10c9844d-e6f7-491d-bae5-39368ee16876\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0whh82"
Sep 15 10:23:11.218059 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:11.217846    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xq2z\" (UniqueName: \"kubernetes.io/projected/10c9844d-e6f7-491d-bae5-39368ee16876-kube-api-access-7xq2z\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0whh82\" (UID: \"10c9844d-e6f7-491d-bae5-39368ee16876\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0whh82"
Sep 15 10:23:11.218059 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:11.217989    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10c9844d-e6f7-491d-bae5-39368ee16876-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0whh82\" (UID: \"10c9844d-e6f7-491d-bae5-39368ee16876\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0whh82"
Sep 15 10:23:11.263019 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:11.262974    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:11.319073 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:11.319050    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10c9844d-e6f7-491d-bae5-39368ee16876-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0whh82\" (UID: \"10c9844d-e6f7-491d-bae5-39368ee16876\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0whh82"
Sep 15 10:23:11.319214 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:11.319119    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10c9844d-e6f7-491d-bae5-39368ee16876-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0whh82\" (UID: \"10c9844d-e6f7-491d-bae5-39368ee16876\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0whh82"
Sep 15 10:23:11.319214 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:11.319135    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xq2z\" (UniqueName: \"kubernetes.io/projected/10c9844d-e6f7-491d-bae5-39368ee16876-kube-api-access-7xq2z\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0whh82\" (UID: \"10c9844d-e6f7-491d-bae5-39368ee16876\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0whh82"
Sep 15 10:23:11.319515 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:11.319498    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10c9844d-e6f7-491d-bae5-39368ee16876-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0whh82\" (UID: \"10c9844d-e6f7-491d-bae5-39368ee16876\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0whh82"
Sep 15 10:23:11.319580 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:11.319552    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10c9844d-e6f7-491d-bae5-39368ee16876-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0whh82\" (UID: \"10c9844d-e6f7-491d-bae5-39368ee16876\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0whh82"
Sep 15 10:23:11.327763 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:11.327740    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xq2z\" (UniqueName: \"kubernetes.io/projected/10c9844d-e6f7-491d-bae5-39368ee16876-kube-api-access-7xq2z\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0whh82\" (UID: \"10c9844d-e6f7-491d-bae5-39368ee16876\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0whh82"
Sep 15 10:23:11.361411 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:11.361382    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:11.428986 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:11.428969    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0whh82"
Sep 15 10:23:11.699845 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:11.699572    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0whh82"]
Sep 15 10:23:11.702056 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:23:11.702025    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10c9844d_e6f7_491d_bae5_39368ee16876.slice/crio-de74de67ca44c70d434dddec90e7ed62e9f4d6ecb209b856c096a0cd29704386 WatchSource:0}: Error finding container de74de67ca44c70d434dddec90e7ed62e9f4d6ecb209b856c096a0cd29704386: Status 404 returned error can't find the container with id de74de67ca44c70d434dddec90e7ed62e9f4d6ecb209b856c096a0cd29704386
Sep 15 10:23:11.884244 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:11.884213    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:12.127110 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.127028    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:12.230548 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.230517    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:12.272117 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.272085    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6699d7bd9c-bnltb"]
Sep 15 10:23:12.275635 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.275619    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6699d7bd9c-bnltb"
Sep 15 10:23:12.278550 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.278528    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 15 10:23:12.278855 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.278838    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 15 10:23:12.278950 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.278873    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-jkrwv\""
Sep 15 10:23:12.279073 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.279061    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 15 10:23:12.279125 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.279067    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 15 10:23:12.293175 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.293156    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6699d7bd9c-bnltb"]
Sep 15 10:23:12.328549 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.328517    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/105dfa5e-4c1b-4913-bf15-7cda5d01f89b-manifests\") pod \"opendatahub-operator-controller-manager-6699d7bd9c-bnltb\" (UID: \"105dfa5e-4c1b-4913-bf15-7cda5d01f89b\") " pod="opendatahub/opendatahub-operator-controller-manager-6699d7bd9c-bnltb"
Sep 15 10:23:12.328713 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.328563    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/105dfa5e-4c1b-4913-bf15-7cda5d01f89b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6699d7bd9c-bnltb\" (UID: \"105dfa5e-4c1b-4913-bf15-7cda5d01f89b\") " pod="opendatahub/opendatahub-operator-controller-manager-6699d7bd9c-bnltb"
Sep 15 10:23:12.328713 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.328603    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/105dfa5e-4c1b-4913-bf15-7cda5d01f89b-webhook-cert\") pod \"opendatahub-operator-controller-manager-6699d7bd9c-bnltb\" (UID: \"105dfa5e-4c1b-4913-bf15-7cda5d01f89b\") " pod="opendatahub/opendatahub-operator-controller-manager-6699d7bd9c-bnltb"
Sep 15 10:23:12.328800 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.328700    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/105dfa5e-4c1b-4913-bf15-7cda5d01f89b-tmp\") pod \"opendatahub-operator-controller-manager-6699d7bd9c-bnltb\" (UID: \"105dfa5e-4c1b-4913-bf15-7cda5d01f89b\") " pod="opendatahub/opendatahub-operator-controller-manager-6699d7bd9c-bnltb"
Sep 15 10:23:12.328800 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.328780    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2gpg\" (UniqueName: \"kubernetes.io/projected/105dfa5e-4c1b-4913-bf15-7cda5d01f89b-kube-api-access-l2gpg\") pod \"opendatahub-operator-controller-manager-6699d7bd9c-bnltb\" (UID: \"105dfa5e-4c1b-4913-bf15-7cda5d01f89b\") " pod="opendatahub/opendatahub-operator-controller-manager-6699d7bd9c-bnltb"
Sep 15 10:23:12.382309 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.382236    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:12.429310 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.429280    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/105dfa5e-4c1b-4913-bf15-7cda5d01f89b-manifests\") pod \"opendatahub-operator-controller-manager-6699d7bd9c-bnltb\" (UID: \"105dfa5e-4c1b-4913-bf15-7cda5d01f89b\") " pod="opendatahub/opendatahub-operator-controller-manager-6699d7bd9c-bnltb"
Sep 15 10:23:12.429435 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.429321    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/105dfa5e-4c1b-4913-bf15-7cda5d01f89b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6699d7bd9c-bnltb\" (UID: \"105dfa5e-4c1b-4913-bf15-7cda5d01f89b\") " pod="opendatahub/opendatahub-operator-controller-manager-6699d7bd9c-bnltb"
Sep 15 10:23:12.429435 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.429355    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/105dfa5e-4c1b-4913-bf15-7cda5d01f89b-webhook-cert\") pod \"opendatahub-operator-controller-manager-6699d7bd9c-bnltb\" (UID: \"105dfa5e-4c1b-4913-bf15-7cda5d01f89b\") " pod="opendatahub/opendatahub-operator-controller-manager-6699d7bd9c-bnltb"
Sep 15 10:23:12.429435 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.429379    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/105dfa5e-4c1b-4913-bf15-7cda5d01f89b-tmp\") pod \"opendatahub-operator-controller-manager-6699d7bd9c-bnltb\" (UID: \"105dfa5e-4c1b-4913-bf15-7cda5d01f89b\") " pod="opendatahub/opendatahub-operator-controller-manager-6699d7bd9c-bnltb"
Sep 15 10:23:12.429435 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.429404    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2gpg\" (UniqueName: \"kubernetes.io/projected/105dfa5e-4c1b-4913-bf15-7cda5d01f89b-kube-api-access-l2gpg\") pod \"opendatahub-operator-controller-manager-6699d7bd9c-bnltb\" (UID: \"105dfa5e-4c1b-4913-bf15-7cda5d01f89b\") " pod="opendatahub/opendatahub-operator-controller-manager-6699d7bd9c-bnltb"
Sep 15 10:23:12.429699 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.429673    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/105dfa5e-4c1b-4913-bf15-7cda5d01f89b-manifests\") pod \"opendatahub-operator-controller-manager-6699d7bd9c-bnltb\" (UID: \"105dfa5e-4c1b-4913-bf15-7cda5d01f89b\") " pod="opendatahub/opendatahub-operator-controller-manager-6699d7bd9c-bnltb"
Sep 15 10:23:12.431786 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.431761    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/105dfa5e-4c1b-4913-bf15-7cda5d01f89b-tmp\") pod \"opendatahub-operator-controller-manager-6699d7bd9c-bnltb\" (UID: \"105dfa5e-4c1b-4913-bf15-7cda5d01f89b\") " pod="opendatahub/opendatahub-operator-controller-manager-6699d7bd9c-bnltb"
Sep 15 10:23:12.431872 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.431829    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/105dfa5e-4c1b-4913-bf15-7cda5d01f89b-webhook-cert\") pod \"opendatahub-operator-controller-manager-6699d7bd9c-bnltb\" (UID: \"105dfa5e-4c1b-4913-bf15-7cda5d01f89b\") " pod="opendatahub/opendatahub-operator-controller-manager-6699d7bd9c-bnltb"
Sep 15 10:23:12.431872 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.431855    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/105dfa5e-4c1b-4913-bf15-7cda5d01f89b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6699d7bd9c-bnltb\" (UID: \"105dfa5e-4c1b-4913-bf15-7cda5d01f89b\") " pod="opendatahub/opendatahub-operator-controller-manager-6699d7bd9c-bnltb"
Sep 15 10:23:12.438089 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.438064    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2gpg\" (UniqueName: \"kubernetes.io/projected/105dfa5e-4c1b-4913-bf15-7cda5d01f89b-kube-api-access-l2gpg\") pod \"opendatahub-operator-controller-manager-6699d7bd9c-bnltb\" (UID: \"105dfa5e-4c1b-4913-bf15-7cda5d01f89b\") " pod="opendatahub/opendatahub-operator-controller-manager-6699d7bd9c-bnltb"
Sep 15 10:23:12.485880 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.485850    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:12.585825 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.585802    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6699d7bd9c-bnltb"
Sep 15 10:23:12.597081 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.597055    2583 generic.go:356] "Generic (PLEG): container finished" podID="10c9844d-e6f7-491d-bae5-39368ee16876" containerID="9089364d5c4abd35e9dd50c88257f325893bad317aff637f10ef60199748fba8" exitCode=0
Sep 15 10:23:12.597198 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.597092    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0whh82" event={"ID":"10c9844d-e6f7-491d-bae5-39368ee16876","Type":"ContainerDied","Data":"9089364d5c4abd35e9dd50c88257f325893bad317aff637f10ef60199748fba8"}
Sep 15 10:23:12.597198 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.597141    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0whh82" event={"ID":"10c9844d-e6f7-491d-bae5-39368ee16876","Type":"ContainerStarted","Data":"de74de67ca44c70d434dddec90e7ed62e9f4d6ecb209b856c096a0cd29704386"}
Sep 15 10:23:12.613923 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.613896    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:12.691494 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.691459    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:12.792864 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.792839    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:12.862211 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.862089    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6699d7bd9c-bnltb"]
Sep 15 10:23:12.863415 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:12.863391    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:12.864961 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:23:12.864921    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod105dfa5e_4c1b_4913_bf15_7cda5d01f89b.slice/crio-255d2d8186d4b4b0e93f615bc99def29fbab67fe0f8e220d1fb27fcd0e3020ff WatchSource:0}: Error finding container 255d2d8186d4b4b0e93f615bc99def29fbab67fe0f8e220d1fb27fcd0e3020ff: Status 404 returned error can't find the container with id 255d2d8186d4b4b0e93f615bc99def29fbab67fe0f8e220d1fb27fcd0e3020ff
Sep 15 10:23:13.607278 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:13.607195    2583 generic.go:356] "Generic (PLEG): container finished" podID="10c9844d-e6f7-491d-bae5-39368ee16876" containerID="cddbbf284707d0730884e4163f8435ac64b3788cbad561d0012fab3a1d3dddd2" exitCode=0
Sep 15 10:23:13.607278 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:13.607234    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0whh82" event={"ID":"10c9844d-e6f7-491d-bae5-39368ee16876","Type":"ContainerDied","Data":"cddbbf284707d0730884e4163f8435ac64b3788cbad561d0012fab3a1d3dddd2"}
Sep 15 10:23:13.609431 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:13.609407    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6699d7bd9c-bnltb" event={"ID":"105dfa5e-4c1b-4913-bf15-7cda5d01f89b","Type":"ContainerStarted","Data":"255d2d8186d4b4b0e93f615bc99def29fbab67fe0f8e220d1fb27fcd0e3020ff"}
Sep 15 10:23:14.616302 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:14.616265    2583 generic.go:356] "Generic (PLEG): container finished" podID="10c9844d-e6f7-491d-bae5-39368ee16876" containerID="8741246f0c1858b8549d97ffce89e76f42963f9fdfd6143a15ffccfdee41d40e" exitCode=0
Sep 15 10:23:14.616697 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:14.616353    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0whh82" event={"ID":"10c9844d-e6f7-491d-bae5-39368ee16876","Type":"ContainerDied","Data":"8741246f0c1858b8549d97ffce89e76f42963f9fdfd6143a15ffccfdee41d40e"}
Sep 15 10:23:15.834350 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:15.834327    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0whh82"
Sep 15 10:23:15.858896 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:15.858872    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10c9844d-e6f7-491d-bae5-39368ee16876-bundle\") pod \"10c9844d-e6f7-491d-bae5-39368ee16876\" (UID: \"10c9844d-e6f7-491d-bae5-39368ee16876\") "
Sep 15 10:23:15.859024 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:15.858937    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10c9844d-e6f7-491d-bae5-39368ee16876-util\") pod \"10c9844d-e6f7-491d-bae5-39368ee16876\" (UID: \"10c9844d-e6f7-491d-bae5-39368ee16876\") "
Sep 15 10:23:15.859024 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:15.858978    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7xq2z\" (UniqueName: \"kubernetes.io/projected/10c9844d-e6f7-491d-bae5-39368ee16876-kube-api-access-7xq2z\") pod \"10c9844d-e6f7-491d-bae5-39368ee16876\" (UID: \"10c9844d-e6f7-491d-bae5-39368ee16876\") "
Sep 15 10:23:15.859738 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:15.859711    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10c9844d-e6f7-491d-bae5-39368ee16876-bundle" (OuterVolumeSpecName: "bundle") pod "10c9844d-e6f7-491d-bae5-39368ee16876" (UID: "10c9844d-e6f7-491d-bae5-39368ee16876"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:23:15.861254 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:15.861233    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10c9844d-e6f7-491d-bae5-39368ee16876-kube-api-access-7xq2z" (OuterVolumeSpecName: "kube-api-access-7xq2z") pod "10c9844d-e6f7-491d-bae5-39368ee16876" (UID: "10c9844d-e6f7-491d-bae5-39368ee16876"). InnerVolumeSpecName "kube-api-access-7xq2z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:23:15.864299 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:15.864258    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10c9844d-e6f7-491d-bae5-39368ee16876-util" (OuterVolumeSpecName: "util") pod "10c9844d-e6f7-491d-bae5-39368ee16876" (UID: "10c9844d-e6f7-491d-bae5-39368ee16876"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:23:15.959731 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:15.959709    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10c9844d-e6f7-491d-bae5-39368ee16876-util\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:23:15.959731 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:15.959731    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xq2z\" (UniqueName: \"kubernetes.io/projected/10c9844d-e6f7-491d-bae5-39368ee16876-kube-api-access-7xq2z\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:23:15.959848 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:15.959742    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10c9844d-e6f7-491d-bae5-39368ee16876-bundle\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:23:16.628852 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:16.628829    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0whh82"
Sep 15 10:23:16.629019 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:16.628828    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0whh82" event={"ID":"10c9844d-e6f7-491d-bae5-39368ee16876","Type":"ContainerDied","Data":"de74de67ca44c70d434dddec90e7ed62e9f4d6ecb209b856c096a0cd29704386"}
Sep 15 10:23:16.629019 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:16.628953    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="de74de67ca44c70d434dddec90e7ed62e9f4d6ecb209b856c096a0cd29704386"
Sep 15 10:23:16.630605 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:16.630583    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6699d7bd9c-bnltb" event={"ID":"105dfa5e-4c1b-4913-bf15-7cda5d01f89b","Type":"ContainerStarted","Data":"f2ac14a96b370072438520fcc8762b6412c19c6f6744c72e56e3463c9aae462b"}
Sep 15 10:23:17.636554 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:17.636524    2583 generic.go:356] "Generic (PLEG): container finished" podID="105dfa5e-4c1b-4913-bf15-7cda5d01f89b" containerID="f2ac14a96b370072438520fcc8762b6412c19c6f6744c72e56e3463c9aae462b" exitCode=0
Sep 15 10:23:17.636896 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:17.636616    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6699d7bd9c-bnltb" event={"ID":"105dfa5e-4c1b-4913-bf15-7cda5d01f89b","Type":"ContainerDied","Data":"f2ac14a96b370072438520fcc8762b6412c19c6f6744c72e56e3463c9aae462b"}
Sep 15 10:23:18.643492 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:18.643455    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6699d7bd9c-bnltb" event={"ID":"105dfa5e-4c1b-4913-bf15-7cda5d01f89b","Type":"ContainerStarted","Data":"44eefa0530a71ed04b1bcc3d29f490c969405236c62772fabf0ad0a1f6bff69f"}
Sep 15 10:23:18.643948 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:18.643625    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6699d7bd9c-bnltb"
Sep 15 10:23:18.685950 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:18.685904    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6699d7bd9c-bnltb" podStartSLOduration=2.238268278 podStartE2EDuration="6.685889354s" totalImagesPullingTime="3.42925939s" totalInitContainerRuntime="1.018361686s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:23:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:23:18.683978548 +0000 UTC m=+313.930945522" watchObservedRunningTime="2026-09-15 10:23:18.685889354 +0000 UTC m=+313.932856323"
Sep 15 10:23:21.593411 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:21.593379    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-85d6978c87-9pkz6"
Sep 15 10:23:24.283303 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:24.283268    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:24.383002 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:24.382968    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:29.650674 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:29.650634    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6699d7bd9c-bnltb"
Sep 15 10:23:31.347929 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:31.347889    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:31.460593 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:31.460563    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:31.642200 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:31.642172    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:32.006498 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:32.006389    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:33.048643 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:33.048607    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:33.142697 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:33.142670    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:34.016934 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:34.016900    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:38.111328 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.111291    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:38.211685 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.211650    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:38.663729 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.663697    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"]
Sep 15 10:23:38.664006 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.663995    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10c9844d-e6f7-491d-bae5-39368ee16876" containerName="extract"
Sep 15 10:23:38.664050 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.664008    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="10c9844d-e6f7-491d-bae5-39368ee16876" containerName="extract"
Sep 15 10:23:38.664050 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.664018    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10c9844d-e6f7-491d-bae5-39368ee16876" containerName="pull"
Sep 15 10:23:38.664050 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.664024    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="10c9844d-e6f7-491d-bae5-39368ee16876" containerName="pull"
Sep 15 10:23:38.664050 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.664038    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10c9844d-e6f7-491d-bae5-39368ee16876" containerName="util"
Sep 15 10:23:38.664050 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.664043    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="10c9844d-e6f7-491d-bae5-39368ee16876" containerName="util"
Sep 15 10:23:38.664226 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.664096    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="10c9844d-e6f7-491d-bae5-39368ee16876" containerName="extract"
Sep 15 10:23:38.667278 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.667259    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"
Sep 15 10:23:38.669483 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.669436    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 15 10:23:38.669620 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.669551    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 15 10:23:38.669781 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.669760    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 15 10:23:38.669857 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.669760    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-vncgv\""
Sep 15 10:23:38.678152 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.678128    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"]
Sep 15 10:23:38.699334 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.699313    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:38.738403 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.738379    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/585f0d63-e037-4387-b8dd-e4530c18c319-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp69vw\" (UID: \"585f0d63-e037-4387-b8dd-e4530c18c319\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"
Sep 15 10:23:38.738503 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.738422    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/585f0d63-e037-4387-b8dd-e4530c18c319-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp69vw\" (UID: \"585f0d63-e037-4387-b8dd-e4530c18c319\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"
Sep 15 10:23:38.738503 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.738491    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g87zl\" (UniqueName: \"kubernetes.io/projected/585f0d63-e037-4387-b8dd-e4530c18c319-kube-api-access-g87zl\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp69vw\" (UID: \"585f0d63-e037-4387-b8dd-e4530c18c319\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"
Sep 15 10:23:38.738577 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.738517    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/585f0d63-e037-4387-b8dd-e4530c18c319-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp69vw\" (UID: \"585f0d63-e037-4387-b8dd-e4530c18c319\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"
Sep 15 10:23:38.738577 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.738566    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/585f0d63-e037-4387-b8dd-e4530c18c319-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp69vw\" (UID: \"585f0d63-e037-4387-b8dd-e4530c18c319\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"
Sep 15 10:23:38.738643 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.738587    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/585f0d63-e037-4387-b8dd-e4530c18c319-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp69vw\" (UID: \"585f0d63-e037-4387-b8dd-e4530c18c319\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"
Sep 15 10:23:38.738643 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.738619    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/585f0d63-e037-4387-b8dd-e4530c18c319-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp69vw\" (UID: \"585f0d63-e037-4387-b8dd-e4530c18c319\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"
Sep 15 10:23:38.738705 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.738676    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/585f0d63-e037-4387-b8dd-e4530c18c319-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp69vw\" (UID: \"585f0d63-e037-4387-b8dd-e4530c18c319\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"
Sep 15 10:23:38.738705 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.738692    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/585f0d63-e037-4387-b8dd-e4530c18c319-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp69vw\" (UID: \"585f0d63-e037-4387-b8dd-e4530c18c319\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"
Sep 15 10:23:38.839795 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.839767    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/585f0d63-e037-4387-b8dd-e4530c18c319-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp69vw\" (UID: \"585f0d63-e037-4387-b8dd-e4530c18c319\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"
Sep 15 10:23:38.839901 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.839821    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/585f0d63-e037-4387-b8dd-e4530c18c319-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp69vw\" (UID: \"585f0d63-e037-4387-b8dd-e4530c18c319\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"
Sep 15 10:23:38.839901 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.839848    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g87zl\" (UniqueName: \"kubernetes.io/projected/585f0d63-e037-4387-b8dd-e4530c18c319-kube-api-access-g87zl\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp69vw\" (UID: \"585f0d63-e037-4387-b8dd-e4530c18c319\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"
Sep 15 10:23:38.839901 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.839870    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/585f0d63-e037-4387-b8dd-e4530c18c319-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp69vw\" (UID: \"585f0d63-e037-4387-b8dd-e4530c18c319\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"
Sep 15 10:23:38.839901 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.839897    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/585f0d63-e037-4387-b8dd-e4530c18c319-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp69vw\" (UID: \"585f0d63-e037-4387-b8dd-e4530c18c319\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"
Sep 15 10:23:38.840090 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.839913    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/585f0d63-e037-4387-b8dd-e4530c18c319-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp69vw\" (UID: \"585f0d63-e037-4387-b8dd-e4530c18c319\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"
Sep 15 10:23:38.840090 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.839942    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/585f0d63-e037-4387-b8dd-e4530c18c319-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp69vw\" (UID: \"585f0d63-e037-4387-b8dd-e4530c18c319\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"
Sep 15 10:23:38.840192 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.840090    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/585f0d63-e037-4387-b8dd-e4530c18c319-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp69vw\" (UID: \"585f0d63-e037-4387-b8dd-e4530c18c319\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"
Sep 15 10:23:38.840192 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.840136    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/585f0d63-e037-4387-b8dd-e4530c18c319-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp69vw\" (UID: \"585f0d63-e037-4387-b8dd-e4530c18c319\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"
Sep 15 10:23:38.840305 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.840285    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/585f0d63-e037-4387-b8dd-e4530c18c319-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp69vw\" (UID: \"585f0d63-e037-4387-b8dd-e4530c18c319\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"
Sep 15 10:23:38.840362 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.840338    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/585f0d63-e037-4387-b8dd-e4530c18c319-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp69vw\" (UID: \"585f0d63-e037-4387-b8dd-e4530c18c319\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"
Sep 15 10:23:38.840417 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.840401    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/585f0d63-e037-4387-b8dd-e4530c18c319-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp69vw\" (UID: \"585f0d63-e037-4387-b8dd-e4530c18c319\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"
Sep 15 10:23:38.840518 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.840499    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/585f0d63-e037-4387-b8dd-e4530c18c319-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp69vw\" (UID: \"585f0d63-e037-4387-b8dd-e4530c18c319\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"
Sep 15 10:23:38.840843 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.840821    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/585f0d63-e037-4387-b8dd-e4530c18c319-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp69vw\" (UID: \"585f0d63-e037-4387-b8dd-e4530c18c319\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"
Sep 15 10:23:38.842305 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.842283    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/585f0d63-e037-4387-b8dd-e4530c18c319-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp69vw\" (UID: \"585f0d63-e037-4387-b8dd-e4530c18c319\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"
Sep 15 10:23:38.842600 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.842580    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/585f0d63-e037-4387-b8dd-e4530c18c319-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp69vw\" (UID: \"585f0d63-e037-4387-b8dd-e4530c18c319\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"
Sep 15 10:23:38.848650 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.848628    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/585f0d63-e037-4387-b8dd-e4530c18c319-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp69vw\" (UID: \"585f0d63-e037-4387-b8dd-e4530c18c319\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"
Sep 15 10:23:38.849136 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.849113    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g87zl\" (UniqueName: \"kubernetes.io/projected/585f0d63-e037-4387-b8dd-e4530c18c319-kube-api-access-g87zl\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cp69vw\" (UID: \"585f0d63-e037-4387-b8dd-e4530c18c319\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"
Sep 15 10:23:38.923200 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.923138    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:38.977268 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:38.977244    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"
Sep 15 10:23:39.191285 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:39.191203    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:39.250864 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:39.250843    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"]
Sep 15 10:23:39.252861 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:23:39.252833    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod585f0d63_e037_4387_b8dd_e4530c18c319.slice/crio-14bba3074c298a8b63723ab3198ee004c40491c461b27e1093ece1541285f2a6 WatchSource:0}: Error finding container 14bba3074c298a8b63723ab3198ee004c40491c461b27e1093ece1541285f2a6: Status 404 returned error can't find the container with id 14bba3074c298a8b63723ab3198ee004c40491c461b27e1093ece1541285f2a6
Sep 15 10:23:39.739423 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:39.739389    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw" event={"ID":"585f0d63-e037-4387-b8dd-e4530c18c319","Type":"ContainerStarted","Data":"14bba3074c298a8b63723ab3198ee004c40491c461b27e1093ece1541285f2a6"}
Sep 15 10:23:42.073287 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:42.073243    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:23:42.073568 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:42.073342    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:23:42.073568 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:42.073381    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:23:42.227989 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:42.227954    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-bbfhr"]
Sep 15 10:23:42.231877 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:42.231858    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-bbfhr"
Sep 15 10:23:42.234164 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:42.234140    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 15 10:23:42.234702 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:42.234679    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-dd4lc\""
Sep 15 10:23:42.242413 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:42.242389    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-bbfhr"]
Sep 15 10:23:42.273826 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:42.273796    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:42.371578 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:42.371497    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:42.376755 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:42.376733    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e3b4daf0-82f9-4aef-801d-a9ab121f984c-cert\") pod \"odh-model-controller-7998c67dc4-bbfhr\" (UID: \"e3b4daf0-82f9-4aef-801d-a9ab121f984c\") " pod="opendatahub/odh-model-controller-7998c67dc4-bbfhr"
Sep 15 10:23:42.376865 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:42.376796    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dc7pj\" (UniqueName: \"kubernetes.io/projected/e3b4daf0-82f9-4aef-801d-a9ab121f984c-kube-api-access-dc7pj\") pod \"odh-model-controller-7998c67dc4-bbfhr\" (UID: \"e3b4daf0-82f9-4aef-801d-a9ab121f984c\") " pod="opendatahub/odh-model-controller-7998c67dc4-bbfhr"
Sep 15 10:23:42.471887 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:42.471854    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:42.477407 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:42.477384    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e3b4daf0-82f9-4aef-801d-a9ab121f984c-cert\") pod \"odh-model-controller-7998c67dc4-bbfhr\" (UID: \"e3b4daf0-82f9-4aef-801d-a9ab121f984c\") " pod="opendatahub/odh-model-controller-7998c67dc4-bbfhr"
Sep 15 10:23:42.477500 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:42.477435    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dc7pj\" (UniqueName: \"kubernetes.io/projected/e3b4daf0-82f9-4aef-801d-a9ab121f984c-kube-api-access-dc7pj\") pod \"odh-model-controller-7998c67dc4-bbfhr\" (UID: \"e3b4daf0-82f9-4aef-801d-a9ab121f984c\") " pod="opendatahub/odh-model-controller-7998c67dc4-bbfhr"
Sep 15 10:23:42.477548 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:23:42.477536    2583 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 15 10:23:42.477602 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:23:42.477592    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3b4daf0-82f9-4aef-801d-a9ab121f984c-cert podName:e3b4daf0-82f9-4aef-801d-a9ab121f984c nodeName:}" failed. No retries permitted until 2026-09-15 10:23:42.977576312 +0000 UTC m=+338.224543259 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e3b4daf0-82f9-4aef-801d-a9ab121f984c-cert") pod "odh-model-controller-7998c67dc4-bbfhr" (UID: "e3b4daf0-82f9-4aef-801d-a9ab121f984c") : secret "odh-model-controller-webhook-cert" not found
Sep 15 10:23:42.485781 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:42.485757    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dc7pj\" (UniqueName: \"kubernetes.io/projected/e3b4daf0-82f9-4aef-801d-a9ab121f984c-kube-api-access-dc7pj\") pod \"odh-model-controller-7998c67dc4-bbfhr\" (UID: \"e3b4daf0-82f9-4aef-801d-a9ab121f984c\") " pod="opendatahub/odh-model-controller-7998c67dc4-bbfhr"
Sep 15 10:23:42.723010 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:42.722983    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:42.757280 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:42.757249    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw" event={"ID":"585f0d63-e037-4387-b8dd-e4530c18c319","Type":"ContainerStarted","Data":"f4dfb3a0d838cd22be80256169d52d85fda6fffb8d3384f68bd5d57330bcf40a"}
Sep 15 10:23:42.777234 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:42.777183    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw" podStartSLOduration=1.9591952510000001 podStartE2EDuration="4.777168162s" totalImagesPullingTime="2.817972911s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:23:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:23:42.775348646 +0000 UTC m=+338.022315618" watchObservedRunningTime="2026-09-15 10:23:42.777168162 +0000 UTC m=+338.024135131"
Sep 15 10:23:42.828210 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:42.828185    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-xvnnk"]
Sep 15 10:23:42.831558 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:42.831543    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-xvnnk"
Sep 15 10:23:42.834258 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:42.834223    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-9rbjl\""
Sep 15 10:23:42.834365 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:42.834270    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 15 10:23:42.839787 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:42.839756    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:42.840521 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:42.840500    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-xvnnk"]
Sep 15 10:23:42.976547 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:42.976432    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:42.978090 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:42.978075    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"
Sep 15 10:23:42.981478 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:42.981458    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ld4hn\" (UniqueName: \"kubernetes.io/projected/71e3decd-7333-4ed5-82cf-e82de3601c18-kube-api-access-ld4hn\") pod \"model-serving-api-54d89bd5fc-xvnnk\" (UID: \"71e3decd-7333-4ed5-82cf-e82de3601c18\") " pod="opendatahub/model-serving-api-54d89bd5fc-xvnnk"
Sep 15 10:23:42.981585 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:42.981567    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71e3decd-7333-4ed5-82cf-e82de3601c18-tls-certs\") pod \"model-serving-api-54d89bd5fc-xvnnk\" (UID: \"71e3decd-7333-4ed5-82cf-e82de3601c18\") " pod="opendatahub/model-serving-api-54d89bd5fc-xvnnk"
Sep 15 10:23:42.981628 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:42.981611    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e3b4daf0-82f9-4aef-801d-a9ab121f984c-cert\") pod \"odh-model-controller-7998c67dc4-bbfhr\" (UID: \"e3b4daf0-82f9-4aef-801d-a9ab121f984c\") " pod="opendatahub/odh-model-controller-7998c67dc4-bbfhr"
Sep 15 10:23:42.983902 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:42.983886    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e3b4daf0-82f9-4aef-801d-a9ab121f984c-cert\") pod \"odh-model-controller-7998c67dc4-bbfhr\" (UID: \"e3b4daf0-82f9-4aef-801d-a9ab121f984c\") " pod="opendatahub/odh-model-controller-7998c67dc4-bbfhr"
Sep 15 10:23:43.083116 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:43.083079    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71e3decd-7333-4ed5-82cf-e82de3601c18-tls-certs\") pod \"model-serving-api-54d89bd5fc-xvnnk\" (UID: \"71e3decd-7333-4ed5-82cf-e82de3601c18\") " pod="opendatahub/model-serving-api-54d89bd5fc-xvnnk"
Sep 15 10:23:43.083513 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:43.083138    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ld4hn\" (UniqueName: \"kubernetes.io/projected/71e3decd-7333-4ed5-82cf-e82de3601c18-kube-api-access-ld4hn\") pod \"model-serving-api-54d89bd5fc-xvnnk\" (UID: \"71e3decd-7333-4ed5-82cf-e82de3601c18\") " pod="opendatahub/model-serving-api-54d89bd5fc-xvnnk"
Sep 15 10:23:43.085544 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:43.085526    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71e3decd-7333-4ed5-82cf-e82de3601c18-tls-certs\") pod \"model-serving-api-54d89bd5fc-xvnnk\" (UID: \"71e3decd-7333-4ed5-82cf-e82de3601c18\") " pod="opendatahub/model-serving-api-54d89bd5fc-xvnnk"
Sep 15 10:23:43.091731 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:43.091708    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ld4hn\" (UniqueName: \"kubernetes.io/projected/71e3decd-7333-4ed5-82cf-e82de3601c18-kube-api-access-ld4hn\") pod \"model-serving-api-54d89bd5fc-xvnnk\" (UID: \"71e3decd-7333-4ed5-82cf-e82de3601c18\") " pod="opendatahub/model-serving-api-54d89bd5fc-xvnnk"
Sep 15 10:23:43.143751 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:43.143719    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-xvnnk"
Sep 15 10:23:43.143751 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:43.143750    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-bbfhr"
Sep 15 10:23:43.417306 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:43.417282    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-xvnnk"]
Sep 15 10:23:43.419353 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:23:43.419326    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71e3decd_7333_4ed5_82cf_e82de3601c18.slice/crio-d6003760f95a58e1b99de81a6eac1efcee130d3533430494a3114c2ebe370cfa WatchSource:0}: Error finding container d6003760f95a58e1b99de81a6eac1efcee130d3533430494a3114c2ebe370cfa: Status 404 returned error can't find the container with id d6003760f95a58e1b99de81a6eac1efcee130d3533430494a3114c2ebe370cfa
Sep 15 10:23:43.425950 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:43.425916    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-bbfhr"]
Sep 15 10:23:43.428516 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:23:43.428493    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3b4daf0_82f9_4aef_801d_a9ab121f984c.slice/crio-8c12f392a0b8b794f931f68aa68b19fb698cb2c88afc5b9f459779cf65d7b8cc WatchSource:0}: Error finding container 8c12f392a0b8b794f931f68aa68b19fb698cb2c88afc5b9f459779cf65d7b8cc: Status 404 returned error can't find the container with id 8c12f392a0b8b794f931f68aa68b19fb698cb2c88afc5b9f459779cf65d7b8cc
Sep 15 10:23:43.762825 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:43.762737    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-xvnnk" event={"ID":"71e3decd-7333-4ed5-82cf-e82de3601c18","Type":"ContainerStarted","Data":"d6003760f95a58e1b99de81a6eac1efcee130d3533430494a3114c2ebe370cfa"}
Sep 15 10:23:43.763945 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:43.763918    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-bbfhr" event={"ID":"e3b4daf0-82f9-4aef-801d-a9ab121f984c","Type":"ContainerStarted","Data":"8c12f392a0b8b794f931f68aa68b19fb698cb2c88afc5b9f459779cf65d7b8cc"}
Sep 15 10:23:43.982313 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:43.982286    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"
Sep 15 10:23:44.770070 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:44.770042    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"
Sep 15 10:23:44.771160 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:44.771139    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cp69vw"
Sep 15 10:23:47.787033 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:47.786995    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-xvnnk" event={"ID":"71e3decd-7333-4ed5-82cf-e82de3601c18","Type":"ContainerStarted","Data":"af3e528181695ad263434d913ed370870051616053e6e661246c11018808c1ad"}
Sep 15 10:23:47.787468 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:47.787078    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-xvnnk"
Sep 15 10:23:47.788610 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:47.788584    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-bbfhr" event={"ID":"e3b4daf0-82f9-4aef-801d-a9ab121f984c","Type":"ContainerStarted","Data":"8ecafb0cc9b03a29d953ed5d0267873ca52767c2a06bba9a91e0b7f14075a077"}
Sep 15 10:23:47.788701 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:47.788634    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-bbfhr"
Sep 15 10:23:47.804836 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:47.804791    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-xvnnk" podStartSLOduration=1.624490005 podStartE2EDuration="5.804777335s" totalImagesPullingTime="4.18028733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:23:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:23:47.803932944 +0000 UTC m=+343.050899914" watchObservedRunningTime="2026-09-15 10:23:47.804777335 +0000 UTC m=+343.051744305"
Sep 15 10:23:47.819890 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:47.819850    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-bbfhr" podStartSLOduration=1.98775425 podStartE2EDuration="5.819836834s" totalImagesPullingTime="3.832082584s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:23:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:23:47.818357605 +0000 UTC m=+343.065324575" watchObservedRunningTime="2026-09-15 10:23:47.819836834 +0000 UTC m=+343.066803805"
Sep 15 10:23:48.967835 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:48.967803    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-24mmv"]
Sep 15 10:23:48.970272 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:48.970256    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-24mmv"
Sep 15 10:23:48.972580 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:48.972558    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 15 10:23:48.972700 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:48.972671    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 15 10:23:48.972880 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:48.972867    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 15 10:23:48.972940 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:48.972870    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-rfv6l\""
Sep 15 10:23:48.983305 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:48.983283    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-24mmv"]
Sep 15 10:23:49.134699 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:49.134661    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8106e39c-4f35-4890-91eb-1204ca4ccc4b-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-24mmv\" (UID: \"8106e39c-4f35-4890-91eb-1204ca4ccc4b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-24mmv"
Sep 15 10:23:49.134859 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:49.134708    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8106e39c-4f35-4890-91eb-1204ca4ccc4b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-24mmv\" (UID: \"8106e39c-4f35-4890-91eb-1204ca4ccc4b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-24mmv"
Sep 15 10:23:49.134859 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:49.134772    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jb4wc\" (UniqueName: \"kubernetes.io/projected/8106e39c-4f35-4890-91eb-1204ca4ccc4b-kube-api-access-jb4wc\") pod \"istiod-openshift-gateway-5677ccb49b-24mmv\" (UID: \"8106e39c-4f35-4890-91eb-1204ca4ccc4b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-24mmv"
Sep 15 10:23:49.134859 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:49.134840    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8106e39c-4f35-4890-91eb-1204ca4ccc4b-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-24mmv\" (UID: \"8106e39c-4f35-4890-91eb-1204ca4ccc4b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-24mmv"
Sep 15 10:23:49.134989 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:49.134869    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8106e39c-4f35-4890-91eb-1204ca4ccc4b-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-24mmv\" (UID: \"8106e39c-4f35-4890-91eb-1204ca4ccc4b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-24mmv"
Sep 15 10:23:49.134989 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:49.134927    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8106e39c-4f35-4890-91eb-1204ca4ccc4b-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-24mmv\" (UID: \"8106e39c-4f35-4890-91eb-1204ca4ccc4b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-24mmv"
Sep 15 10:23:49.134989 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:49.134958    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8106e39c-4f35-4890-91eb-1204ca4ccc4b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-24mmv\" (UID: \"8106e39c-4f35-4890-91eb-1204ca4ccc4b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-24mmv"
Sep 15 10:23:49.236885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:49.235879    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8106e39c-4f35-4890-91eb-1204ca4ccc4b-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-24mmv\" (UID: \"8106e39c-4f35-4890-91eb-1204ca4ccc4b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-24mmv"
Sep 15 10:23:49.236885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:49.235951    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8106e39c-4f35-4890-91eb-1204ca4ccc4b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-24mmv\" (UID: \"8106e39c-4f35-4890-91eb-1204ca4ccc4b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-24mmv"
Sep 15 10:23:49.236885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:49.235991    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jb4wc\" (UniqueName: \"kubernetes.io/projected/8106e39c-4f35-4890-91eb-1204ca4ccc4b-kube-api-access-jb4wc\") pod \"istiod-openshift-gateway-5677ccb49b-24mmv\" (UID: \"8106e39c-4f35-4890-91eb-1204ca4ccc4b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-24mmv"
Sep 15 10:23:49.236885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:49.236026    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8106e39c-4f35-4890-91eb-1204ca4ccc4b-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-24mmv\" (UID: \"8106e39c-4f35-4890-91eb-1204ca4ccc4b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-24mmv"
Sep 15 10:23:49.236885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:49.236069    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8106e39c-4f35-4890-91eb-1204ca4ccc4b-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-24mmv\" (UID: \"8106e39c-4f35-4890-91eb-1204ca4ccc4b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-24mmv"
Sep 15 10:23:49.236885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:49.236119    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8106e39c-4f35-4890-91eb-1204ca4ccc4b-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-24mmv\" (UID: \"8106e39c-4f35-4890-91eb-1204ca4ccc4b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-24mmv"
Sep 15 10:23:49.236885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:49.236147    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8106e39c-4f35-4890-91eb-1204ca4ccc4b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-24mmv\" (UID: \"8106e39c-4f35-4890-91eb-1204ca4ccc4b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-24mmv"
Sep 15 10:23:49.240224 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:49.239029    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8106e39c-4f35-4890-91eb-1204ca4ccc4b-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-24mmv\" (UID: \"8106e39c-4f35-4890-91eb-1204ca4ccc4b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-24mmv"
Sep 15 10:23:49.240224 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:49.239663    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8106e39c-4f35-4890-91eb-1204ca4ccc4b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-24mmv\" (UID: \"8106e39c-4f35-4890-91eb-1204ca4ccc4b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-24mmv"
Sep 15 10:23:49.241662 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:49.241310    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8106e39c-4f35-4890-91eb-1204ca4ccc4b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-24mmv\" (UID: \"8106e39c-4f35-4890-91eb-1204ca4ccc4b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-24mmv"
Sep 15 10:23:49.243492 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:49.243426    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8106e39c-4f35-4890-91eb-1204ca4ccc4b-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-24mmv\" (UID: \"8106e39c-4f35-4890-91eb-1204ca4ccc4b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-24mmv"
Sep 15 10:23:49.244012 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:49.243986    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8106e39c-4f35-4890-91eb-1204ca4ccc4b-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-24mmv\" (UID: \"8106e39c-4f35-4890-91eb-1204ca4ccc4b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-24mmv"
Sep 15 10:23:49.253422 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:49.253401    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8106e39c-4f35-4890-91eb-1204ca4ccc4b-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-24mmv\" (UID: \"8106e39c-4f35-4890-91eb-1204ca4ccc4b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-24mmv"
Sep 15 10:23:49.253749 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:49.253731    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jb4wc\" (UniqueName: \"kubernetes.io/projected/8106e39c-4f35-4890-91eb-1204ca4ccc4b-kube-api-access-jb4wc\") pod \"istiod-openshift-gateway-5677ccb49b-24mmv\" (UID: \"8106e39c-4f35-4890-91eb-1204ca4ccc4b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-24mmv"
Sep 15 10:23:49.279624 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:49.279606    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-24mmv"
Sep 15 10:23:49.555180 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:49.555157    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-24mmv"]
Sep 15 10:23:49.556894 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:23:49.556866    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8106e39c_4f35_4890_91eb_1204ca4ccc4b.slice/crio-674c72425bae0903962ba2923b4614ef905fc7b7423174ecb233db0c55dba5a2 WatchSource:0}: Error finding container 674c72425bae0903962ba2923b4614ef905fc7b7423174ecb233db0c55dba5a2: Status 404 returned error can't find the container with id 674c72425bae0903962ba2923b4614ef905fc7b7423174ecb233db0c55dba5a2
Sep 15 10:23:49.799834 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:49.799806    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-24mmv" event={"ID":"8106e39c-4f35-4890-91eb-1204ca4ccc4b","Type":"ContainerStarted","Data":"674c72425bae0903962ba2923b4614ef905fc7b7423174ecb233db0c55dba5a2"}
Sep 15 10:23:51.465561 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:51.465515    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:51.563271 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:51.563238    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:51.657084 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:51.657047    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:51.788978 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:51.788893    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:52.266105 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:52.266070    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:23:52.266171 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:52.266149    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:23:52.756385 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:52.756308    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:52.819171 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:52.819132    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-24mmv" event={"ID":"8106e39c-4f35-4890-91eb-1204ca4ccc4b","Type":"ContainerStarted","Data":"2c7e236d46b6bbd70097a44cb6c10b83beb0752ccfa4f79168b44cd3d3ce9e2c"}
Sep 15 10:23:52.819370 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:52.819313    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-24mmv"
Sep 15 10:23:52.844181 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:52.844118    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-24mmv" podStartSLOduration=2.137173339 podStartE2EDuration="4.84409874s" totalImagesPullingTime="2.706925401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:23:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:23:52.841512711 +0000 UTC m=+348.088479681" watchObservedRunningTime="2026-09-15 10:23:52.84409874 +0000 UTC m=+348.091065711"
Sep 15 10:23:52.849800 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:52.849768    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:52.998323 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:52.998290    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:23:53.826684 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:53.826660    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-24mmv"
Sep 15 10:23:58.795808 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:58.795727    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-bbfhr"
Sep 15 10:23:58.797892 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:23:58.797876    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-xvnnk"
Sep 15 10:24:25.074679 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.074642    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d6995cf76-f9jf2"]
Sep 15 10:24:25.085358 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.085330    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d6995cf76-f9jf2"
Sep 15 10:24:25.089595 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.089570    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d6995cf76-f9jf2"]
Sep 15 10:24:25.135260 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.135234    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:24:25.189159 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.189135    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:24:25.213338 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.213304    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzlh7\" (UniqueName: \"kubernetes.io/projected/7fa13ed5-9d37-40e8-b621-62c60369ba9a-kube-api-access-tzlh7\") pod \"console-6d6995cf76-f9jf2\" (UID: \"7fa13ed5-9d37-40e8-b621-62c60369ba9a\") " pod="openshift-console/console-6d6995cf76-f9jf2"
Sep 15 10:24:25.213506 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.213356    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7fa13ed5-9d37-40e8-b621-62c60369ba9a-console-oauth-config\") pod \"console-6d6995cf76-f9jf2\" (UID: \"7fa13ed5-9d37-40e8-b621-62c60369ba9a\") " pod="openshift-console/console-6d6995cf76-f9jf2"
Sep 15 10:24:25.213506 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.213378    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7fa13ed5-9d37-40e8-b621-62c60369ba9a-service-ca\") pod \"console-6d6995cf76-f9jf2\" (UID: \"7fa13ed5-9d37-40e8-b621-62c60369ba9a\") " pod="openshift-console/console-6d6995cf76-f9jf2"
Sep 15 10:24:25.213506 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.213399    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7fa13ed5-9d37-40e8-b621-62c60369ba9a-trusted-ca-bundle\") pod \"console-6d6995cf76-f9jf2\" (UID: \"7fa13ed5-9d37-40e8-b621-62c60369ba9a\") " pod="openshift-console/console-6d6995cf76-f9jf2"
Sep 15 10:24:25.213506 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.213493    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7fa13ed5-9d37-40e8-b621-62c60369ba9a-console-serving-cert\") pod \"console-6d6995cf76-f9jf2\" (UID: \"7fa13ed5-9d37-40e8-b621-62c60369ba9a\") " pod="openshift-console/console-6d6995cf76-f9jf2"
Sep 15 10:24:25.213704 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.213509    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7fa13ed5-9d37-40e8-b621-62c60369ba9a-oauth-serving-cert\") pod \"console-6d6995cf76-f9jf2\" (UID: \"7fa13ed5-9d37-40e8-b621-62c60369ba9a\") " pod="openshift-console/console-6d6995cf76-f9jf2"
Sep 15 10:24:25.213704 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.213537    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7fa13ed5-9d37-40e8-b621-62c60369ba9a-console-config\") pod \"console-6d6995cf76-f9jf2\" (UID: \"7fa13ed5-9d37-40e8-b621-62c60369ba9a\") " pod="openshift-console/console-6d6995cf76-f9jf2"
Sep 15 10:24:25.279359 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.279332    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:24:25.313925 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.313893    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzlh7\" (UniqueName: \"kubernetes.io/projected/7fa13ed5-9d37-40e8-b621-62c60369ba9a-kube-api-access-tzlh7\") pod \"console-6d6995cf76-f9jf2\" (UID: \"7fa13ed5-9d37-40e8-b621-62c60369ba9a\") " pod="openshift-console/console-6d6995cf76-f9jf2"
Sep 15 10:24:25.313925 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.313929    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7fa13ed5-9d37-40e8-b621-62c60369ba9a-console-oauth-config\") pod \"console-6d6995cf76-f9jf2\" (UID: \"7fa13ed5-9d37-40e8-b621-62c60369ba9a\") " pod="openshift-console/console-6d6995cf76-f9jf2"
Sep 15 10:24:25.314167 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.313953    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7fa13ed5-9d37-40e8-b621-62c60369ba9a-service-ca\") pod \"console-6d6995cf76-f9jf2\" (UID: \"7fa13ed5-9d37-40e8-b621-62c60369ba9a\") " pod="openshift-console/console-6d6995cf76-f9jf2"
Sep 15 10:24:25.314167 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.313978    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7fa13ed5-9d37-40e8-b621-62c60369ba9a-trusted-ca-bundle\") pod \"console-6d6995cf76-f9jf2\" (UID: \"7fa13ed5-9d37-40e8-b621-62c60369ba9a\") " pod="openshift-console/console-6d6995cf76-f9jf2"
Sep 15 10:24:25.314167 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.314057    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7fa13ed5-9d37-40e8-b621-62c60369ba9a-console-serving-cert\") pod \"console-6d6995cf76-f9jf2\" (UID: \"7fa13ed5-9d37-40e8-b621-62c60369ba9a\") " pod="openshift-console/console-6d6995cf76-f9jf2"
Sep 15 10:24:25.314167 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.314079    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7fa13ed5-9d37-40e8-b621-62c60369ba9a-oauth-serving-cert\") pod \"console-6d6995cf76-f9jf2\" (UID: \"7fa13ed5-9d37-40e8-b621-62c60369ba9a\") " pod="openshift-console/console-6d6995cf76-f9jf2"
Sep 15 10:24:25.314167 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.314116    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7fa13ed5-9d37-40e8-b621-62c60369ba9a-console-config\") pod \"console-6d6995cf76-f9jf2\" (UID: \"7fa13ed5-9d37-40e8-b621-62c60369ba9a\") " pod="openshift-console/console-6d6995cf76-f9jf2"
Sep 15 10:24:25.314745 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.314714    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7fa13ed5-9d37-40e8-b621-62c60369ba9a-service-ca\") pod \"console-6d6995cf76-f9jf2\" (UID: \"7fa13ed5-9d37-40e8-b621-62c60369ba9a\") " pod="openshift-console/console-6d6995cf76-f9jf2"
Sep 15 10:24:25.314911 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.314884    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7fa13ed5-9d37-40e8-b621-62c60369ba9a-trusted-ca-bundle\") pod \"console-6d6995cf76-f9jf2\" (UID: \"7fa13ed5-9d37-40e8-b621-62c60369ba9a\") " pod="openshift-console/console-6d6995cf76-f9jf2"
Sep 15 10:24:25.314994 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.314920    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7fa13ed5-9d37-40e8-b621-62c60369ba9a-console-config\") pod \"console-6d6995cf76-f9jf2\" (UID: \"7fa13ed5-9d37-40e8-b621-62c60369ba9a\") " pod="openshift-console/console-6d6995cf76-f9jf2"
Sep 15 10:24:25.315202 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.315173    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7fa13ed5-9d37-40e8-b621-62c60369ba9a-oauth-serving-cert\") pod \"console-6d6995cf76-f9jf2\" (UID: \"7fa13ed5-9d37-40e8-b621-62c60369ba9a\") " pod="openshift-console/console-6d6995cf76-f9jf2"
Sep 15 10:24:25.316743 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.316721    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7fa13ed5-9d37-40e8-b621-62c60369ba9a-console-oauth-config\") pod \"console-6d6995cf76-f9jf2\" (UID: \"7fa13ed5-9d37-40e8-b621-62c60369ba9a\") " pod="openshift-console/console-6d6995cf76-f9jf2"
Sep 15 10:24:25.317756 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.317734    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7fa13ed5-9d37-40e8-b621-62c60369ba9a-console-serving-cert\") pod \"console-6d6995cf76-f9jf2\" (UID: \"7fa13ed5-9d37-40e8-b621-62c60369ba9a\") " pod="openshift-console/console-6d6995cf76-f9jf2"
Sep 15 10:24:25.321538 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.321514    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzlh7\" (UniqueName: \"kubernetes.io/projected/7fa13ed5-9d37-40e8-b621-62c60369ba9a-kube-api-access-tzlh7\") pod \"console-6d6995cf76-f9jf2\" (UID: \"7fa13ed5-9d37-40e8-b621-62c60369ba9a\") " pod="openshift-console/console-6d6995cf76-f9jf2"
Sep 15 10:24:25.338202 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.338151    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:24:25.395296 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.395271    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d6995cf76-f9jf2"
Sep 15 10:24:25.664120 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.664097    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d6995cf76-f9jf2"]
Sep 15 10:24:25.666278 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:24:25.666251    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fa13ed5_9d37_40e8_b621_62c60369ba9a.slice/crio-644eac8cf7e4eec50e8d2b8c2d5c8e32524feed6eb31413af882439a26956e35 WatchSource:0}: Error finding container 644eac8cf7e4eec50e8d2b8c2d5c8e32524feed6eb31413af882439a26956e35: Status 404 returned error can't find the container with id 644eac8cf7e4eec50e8d2b8c2d5c8e32524feed6eb31413af882439a26956e35
Sep 15 10:24:25.874396 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.874369    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:24:25.927531 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.927460    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:24:25.983328 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.983300    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d6995cf76-f9jf2" event={"ID":"7fa13ed5-9d37-40e8-b621-62c60369ba9a","Type":"ContainerStarted","Data":"4d83610c65a22c232a3cc87f23a3d229cb68557ace9ae5476fbda324a7a393a5"}
Sep 15 10:24:25.983436 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.983346    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d6995cf76-f9jf2" event={"ID":"7fa13ed5-9d37-40e8-b621-62c60369ba9a","Type":"ContainerStarted","Data":"644eac8cf7e4eec50e8d2b8c2d5c8e32524feed6eb31413af882439a26956e35"}
Sep 15 10:24:25.999908 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:25.999866    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6d6995cf76-f9jf2" podStartSLOduration=0.99984923 podStartE2EDuration="999.84923ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:24:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:24:25.999205179 +0000 UTC m=+381.246172149" watchObservedRunningTime="2026-09-15 10:24:25.99984923 +0000 UTC m=+381.246816200"
Sep 15 10:24:35.395721 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:35.395689    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d6995cf76-f9jf2"
Sep 15 10:24:35.396206 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:35.395753    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6d6995cf76-f9jf2"
Sep 15 10:24:35.400402 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:35.400380    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6d6995cf76-f9jf2"
Sep 15 10:24:36.037972 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:36.037934    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6d6995cf76-f9jf2"
Sep 15 10:24:36.081380 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:36.081347    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f4bdf6bf5-6clb8"]
Sep 15 10:24:36.183456 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:36.183424    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:24:36.235922 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:36.235894    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:24:36.291541 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:36.291474    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:24:43.265623 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:43.265595    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:24:43.363847 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:43.363814    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:24:46.745325 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:46.745293    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pd9x9"]
Sep 15 10:24:46.748993 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:46.748978    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pd9x9"
Sep 15 10:24:46.751390 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:46.751360    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-c9n8v\""
Sep 15 10:24:46.751390 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:46.751378    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 15 10:24:46.752063 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:46.752049    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 15 10:24:46.756698 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:46.756678    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pd9x9"]
Sep 15 10:24:46.843421 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:46.843384    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69gc6m"]
Sep 15 10:24:46.846934 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:46.846917    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69gc6m"
Sep 15 10:24:46.853863 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:46.853749    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69gc6m"]
Sep 15 10:24:46.903583 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:46.903547    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7e31e0f-4739-482b-88e2-8e7a6f609edf-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pd9x9\" (UID: \"f7e31e0f-4739-482b-88e2-8e7a6f609edf\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pd9x9"
Sep 15 10:24:46.903760 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:46.903606    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxhjj\" (UniqueName: \"kubernetes.io/projected/f7e31e0f-4739-482b-88e2-8e7a6f609edf-kube-api-access-fxhjj\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pd9x9\" (UID: \"f7e31e0f-4739-482b-88e2-8e7a6f609edf\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pd9x9"
Sep 15 10:24:46.903760 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:46.903691    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7e31e0f-4739-482b-88e2-8e7a6f609edf-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pd9x9\" (UID: \"f7e31e0f-4739-482b-88e2-8e7a6f609edf\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pd9x9"
Sep 15 10:24:46.929360 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:46.929319    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:24:46.939622 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:46.939595    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7hqz2"]
Sep 15 10:24:46.943142 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:46.943124    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7hqz2"
Sep 15 10:24:46.951005 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:46.950983    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7hqz2"]
Sep 15 10:24:47.005019 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.004929    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7e31e0f-4739-482b-88e2-8e7a6f609edf-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pd9x9\" (UID: \"f7e31e0f-4739-482b-88e2-8e7a6f609edf\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pd9x9"
Sep 15 10:24:47.005019 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.004977    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1afa8705-52b1-480b-9db9-68ba1800f62f-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69gc6m\" (UID: \"1afa8705-52b1-480b-9db9-68ba1800f62f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69gc6m"
Sep 15 10:24:47.005250 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.005041    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnltq\" (UniqueName: \"kubernetes.io/projected/1afa8705-52b1-480b-9db9-68ba1800f62f-kube-api-access-xnltq\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69gc6m\" (UID: \"1afa8705-52b1-480b-9db9-68ba1800f62f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69gc6m"
Sep 15 10:24:47.005250 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.005066    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7e31e0f-4739-482b-88e2-8e7a6f609edf-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pd9x9\" (UID: \"f7e31e0f-4739-482b-88e2-8e7a6f609edf\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pd9x9"
Sep 15 10:24:47.005250 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.005144    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxhjj\" (UniqueName: \"kubernetes.io/projected/f7e31e0f-4739-482b-88e2-8e7a6f609edf-kube-api-access-fxhjj\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pd9x9\" (UID: \"f7e31e0f-4739-482b-88e2-8e7a6f609edf\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pd9x9"
Sep 15 10:24:47.005250 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.005181    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1afa8705-52b1-480b-9db9-68ba1800f62f-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69gc6m\" (UID: \"1afa8705-52b1-480b-9db9-68ba1800f62f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69gc6m"
Sep 15 10:24:47.005494 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.005376    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7e31e0f-4739-482b-88e2-8e7a6f609edf-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pd9x9\" (UID: \"f7e31e0f-4739-482b-88e2-8e7a6f609edf\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pd9x9"
Sep 15 10:24:47.005494 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.005380    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7e31e0f-4739-482b-88e2-8e7a6f609edf-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pd9x9\" (UID: \"f7e31e0f-4739-482b-88e2-8e7a6f609edf\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pd9x9"
Sep 15 10:24:47.013597 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.013578    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxhjj\" (UniqueName: \"kubernetes.io/projected/f7e31e0f-4739-482b-88e2-8e7a6f609edf-kube-api-access-fxhjj\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pd9x9\" (UID: \"f7e31e0f-4739-482b-88e2-8e7a6f609edf\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pd9x9"
Sep 15 10:24:47.029638 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.029609    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:24:47.047034 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.047009    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajh8k4"]
Sep 15 10:24:47.050653 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.050639    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajh8k4"
Sep 15 10:24:47.060390 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.060364    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pd9x9"
Sep 15 10:24:47.062118 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.062098    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajh8k4"]
Sep 15 10:24:47.106086 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.106048    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1afa8705-52b1-480b-9db9-68ba1800f62f-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69gc6m\" (UID: \"1afa8705-52b1-480b-9db9-68ba1800f62f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69gc6m"
Sep 15 10:24:47.106086 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.106089    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f561c96d-1b78-4c40-891d-ae150a4592c0-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7hqz2\" (UID: \"f561c96d-1b78-4c40-891d-ae150a4592c0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7hqz2"
Sep 15 10:24:47.106338 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.106110    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qfj9\" (UniqueName: \"kubernetes.io/projected/f561c96d-1b78-4c40-891d-ae150a4592c0-kube-api-access-4qfj9\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7hqz2\" (UID: \"f561c96d-1b78-4c40-891d-ae150a4592c0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7hqz2"
Sep 15 10:24:47.106338 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.106239    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f561c96d-1b78-4c40-891d-ae150a4592c0-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7hqz2\" (UID: \"f561c96d-1b78-4c40-891d-ae150a4592c0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7hqz2"
Sep 15 10:24:47.106338 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.106317    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnltq\" (UniqueName: \"kubernetes.io/projected/1afa8705-52b1-480b-9db9-68ba1800f62f-kube-api-access-xnltq\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69gc6m\" (UID: \"1afa8705-52b1-480b-9db9-68ba1800f62f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69gc6m"
Sep 15 10:24:47.106514 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.106395    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1afa8705-52b1-480b-9db9-68ba1800f62f-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69gc6m\" (UID: \"1afa8705-52b1-480b-9db9-68ba1800f62f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69gc6m"
Sep 15 10:24:47.106560 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.106530    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1afa8705-52b1-480b-9db9-68ba1800f62f-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69gc6m\" (UID: \"1afa8705-52b1-480b-9db9-68ba1800f62f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69gc6m"
Sep 15 10:24:47.106695 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.106677    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1afa8705-52b1-480b-9db9-68ba1800f62f-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69gc6m\" (UID: \"1afa8705-52b1-480b-9db9-68ba1800f62f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69gc6m"
Sep 15 10:24:47.115226 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.115206    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnltq\" (UniqueName: \"kubernetes.io/projected/1afa8705-52b1-480b-9db9-68ba1800f62f-kube-api-access-xnltq\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69gc6m\" (UID: \"1afa8705-52b1-480b-9db9-68ba1800f62f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69gc6m"
Sep 15 10:24:47.156329 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.156294    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69gc6m"
Sep 15 10:24:47.207877 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.207806    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajh8k4\" (UID: \"7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajh8k4"
Sep 15 10:24:47.208025 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.207893    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f561c96d-1b78-4c40-891d-ae150a4592c0-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7hqz2\" (UID: \"f561c96d-1b78-4c40-891d-ae150a4592c0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7hqz2"
Sep 15 10:24:47.208025 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.207926    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qfj9\" (UniqueName: \"kubernetes.io/projected/f561c96d-1b78-4c40-891d-ae150a4592c0-kube-api-access-4qfj9\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7hqz2\" (UID: \"f561c96d-1b78-4c40-891d-ae150a4592c0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7hqz2"
Sep 15 10:24:47.208025 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.207993    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f561c96d-1b78-4c40-891d-ae150a4592c0-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7hqz2\" (UID: \"f561c96d-1b78-4c40-891d-ae150a4592c0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7hqz2"
Sep 15 10:24:47.208338 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.208026    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs4bw\" (UniqueName: \"kubernetes.io/projected/7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9-kube-api-access-gs4bw\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajh8k4\" (UID: \"7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajh8k4"
Sep 15 10:24:47.208338 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.208065    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajh8k4\" (UID: \"7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajh8k4"
Sep 15 10:24:47.208597 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.208395    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f561c96d-1b78-4c40-891d-ae150a4592c0-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7hqz2\" (UID: \"f561c96d-1b78-4c40-891d-ae150a4592c0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7hqz2"
Sep 15 10:24:47.208742 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.208700    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f561c96d-1b78-4c40-891d-ae150a4592c0-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7hqz2\" (UID: \"f561c96d-1b78-4c40-891d-ae150a4592c0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7hqz2"
Sep 15 10:24:47.217371 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.217348    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qfj9\" (UniqueName: \"kubernetes.io/projected/f561c96d-1b78-4c40-891d-ae150a4592c0-kube-api-access-4qfj9\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7hqz2\" (UID: \"f561c96d-1b78-4c40-891d-ae150a4592c0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7hqz2"
Sep 15 10:24:47.253355 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.253321    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7hqz2"
Sep 15 10:24:47.309806 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.309242    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajh8k4\" (UID: \"7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajh8k4"
Sep 15 10:24:47.309806 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.309308    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs4bw\" (UniqueName: \"kubernetes.io/projected/7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9-kube-api-access-gs4bw\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajh8k4\" (UID: \"7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajh8k4"
Sep 15 10:24:47.309806 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.309342    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajh8k4\" (UID: \"7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajh8k4"
Sep 15 10:24:47.309806 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.309688    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajh8k4\" (UID: \"7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajh8k4"
Sep 15 10:24:47.310079 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.309959    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajh8k4\" (UID: \"7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajh8k4"
Sep 15 10:24:47.318958 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.318931    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs4bw\" (UniqueName: \"kubernetes.io/projected/7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9-kube-api-access-gs4bw\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajh8k4\" (UID: \"7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajh8k4"
Sep 15 10:24:47.336093 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.336066    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pd9x9"]
Sep 15 10:24:47.336315 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:24:47.336295    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7e31e0f_4739_482b_88e2_8e7a6f609edf.slice/crio-1ec46f688a47c9ac3d2ae0563e713ab2d3026a14ca9e11cde8bd817bca7b34db WatchSource:0}: Error finding container 1ec46f688a47c9ac3d2ae0563e713ab2d3026a14ca9e11cde8bd817bca7b34db: Status 404 returned error can't find the container with id 1ec46f688a47c9ac3d2ae0563e713ab2d3026a14ca9e11cde8bd817bca7b34db
Sep 15 10:24:47.360499 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.360476    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajh8k4"
Sep 15 10:24:47.407706 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.407654    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:24:47.431931 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.431893    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69gc6m"]
Sep 15 10:24:47.432619 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:24:47.432585    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1afa8705_52b1_480b_9db9_68ba1800f62f.slice/crio-1417c0fdb17e38cf4a07c41ec7e2ccbbdd70e6ddb3c1065bbfdf7e6c9c8e60d6 WatchSource:0}: Error finding container 1417c0fdb17e38cf4a07c41ec7e2ccbbdd70e6ddb3c1065bbfdf7e6c9c8e60d6: Status 404 returned error can't find the container with id 1417c0fdb17e38cf4a07c41ec7e2ccbbdd70e6ddb3c1065bbfdf7e6c9c8e60d6
Sep 15 10:24:47.535328 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.529051    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7hqz2"]
Sep 15 10:24:47.632127 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:47.632105    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajh8k4"]
Sep 15 10:24:47.633958 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:24:47.633928    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bad47d0_dfe4_42c7_ab09_3e66b0ae75f9.slice/crio-a6bbbff4862c062072941410f53c990056d8d418d3d193e6132cb6046046bde3 WatchSource:0}: Error finding container a6bbbff4862c062072941410f53c990056d8d418d3d193e6132cb6046046bde3: Status 404 returned error can't find the container with id a6bbbff4862c062072941410f53c990056d8d418d3d193e6132cb6046046bde3
Sep 15 10:24:48.100052 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:48.100016    2583 generic.go:356] "Generic (PLEG): container finished" podID="1afa8705-52b1-480b-9db9-68ba1800f62f" containerID="96806685d579954ef15af55d3bb0201d672eff99fb593f171f230e9f24646c3c" exitCode=0
Sep 15 10:24:48.100489 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:48.100102    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69gc6m" event={"ID":"1afa8705-52b1-480b-9db9-68ba1800f62f","Type":"ContainerDied","Data":"96806685d579954ef15af55d3bb0201d672eff99fb593f171f230e9f24646c3c"}
Sep 15 10:24:48.100489 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:48.100169    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69gc6m" event={"ID":"1afa8705-52b1-480b-9db9-68ba1800f62f","Type":"ContainerStarted","Data":"1417c0fdb17e38cf4a07c41ec7e2ccbbdd70e6ddb3c1065bbfdf7e6c9c8e60d6"}
Sep 15 10:24:48.101736 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:48.101710    2583 generic.go:356] "Generic (PLEG): container finished" podID="f561c96d-1b78-4c40-891d-ae150a4592c0" containerID="1d59944974999b2f134dd4cd5c73ea12e620527ec600ef44674d79a98ae66178" exitCode=0
Sep 15 10:24:48.101963 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:48.101761    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7hqz2" event={"ID":"f561c96d-1b78-4c40-891d-ae150a4592c0","Type":"ContainerDied","Data":"1d59944974999b2f134dd4cd5c73ea12e620527ec600ef44674d79a98ae66178"}
Sep 15 10:24:48.101963 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:48.101809    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7hqz2" event={"ID":"f561c96d-1b78-4c40-891d-ae150a4592c0","Type":"ContainerStarted","Data":"a6ae84b04c1221575aadf2ac5635cf4d401c9642f2ebe669f3615f07b78e9ec7"}
Sep 15 10:24:48.103327 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:48.103303    2583 generic.go:356] "Generic (PLEG): container finished" podID="f7e31e0f-4739-482b-88e2-8e7a6f609edf" containerID="d59e7ea7dbc8d02a4426a1565d67f89c2e00b55d83cea0fb664f8336173f776b" exitCode=0
Sep 15 10:24:48.103407 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:48.103387    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pd9x9" event={"ID":"f7e31e0f-4739-482b-88e2-8e7a6f609edf","Type":"ContainerDied","Data":"d59e7ea7dbc8d02a4426a1565d67f89c2e00b55d83cea0fb664f8336173f776b"}
Sep 15 10:24:48.103483 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:48.103432    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pd9x9" event={"ID":"f7e31e0f-4739-482b-88e2-8e7a6f609edf","Type":"ContainerStarted","Data":"1ec46f688a47c9ac3d2ae0563e713ab2d3026a14ca9e11cde8bd817bca7b34db"}
Sep 15 10:24:48.104984 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:48.104968    2583 generic.go:356] "Generic (PLEG): container finished" podID="7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9" containerID="70ce5ab16391eb539528e799f97236d2b4ec7d1ddaa76b536c7438b526dabb2e" exitCode=0
Sep 15 10:24:48.105055 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:48.105021    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajh8k4" event={"ID":"7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9","Type":"ContainerDied","Data":"70ce5ab16391eb539528e799f97236d2b4ec7d1ddaa76b536c7438b526dabb2e"}
Sep 15 10:24:48.105055 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:48.105052    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajh8k4" event={"ID":"7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9","Type":"ContainerStarted","Data":"a6bbbff4862c062072941410f53c990056d8d418d3d193e6132cb6046046bde3"}
Sep 15 10:24:48.876381 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:48.876347    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:24:48.926570 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:48.926540    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:24:49.112152 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:49.112117    2583 generic.go:356] "Generic (PLEG): container finished" podID="1afa8705-52b1-480b-9db9-68ba1800f62f" containerID="3285bf64b434efdda122047f0b44470aeaf67da84db70823210a9707d920f1c2" exitCode=0
Sep 15 10:24:49.112506 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:49.112187    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69gc6m" event={"ID":"1afa8705-52b1-480b-9db9-68ba1800f62f","Type":"ContainerDied","Data":"3285bf64b434efdda122047f0b44470aeaf67da84db70823210a9707d920f1c2"}
Sep 15 10:24:49.114044 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:49.114021    2583 generic.go:356] "Generic (PLEG): container finished" podID="f561c96d-1b78-4c40-891d-ae150a4592c0" containerID="6e9ebf4a88651c98cfd80a9bbc20b94af3eb161590cd41e78e8065f4a140abd9" exitCode=0
Sep 15 10:24:49.114133 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:49.114090    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7hqz2" event={"ID":"f561c96d-1b78-4c40-891d-ae150a4592c0","Type":"ContainerDied","Data":"6e9ebf4a88651c98cfd80a9bbc20b94af3eb161590cd41e78e8065f4a140abd9"}
Sep 15 10:24:49.115806 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:49.115784    2583 generic.go:356] "Generic (PLEG): container finished" podID="f7e31e0f-4739-482b-88e2-8e7a6f609edf" containerID="75a1c823c208ba49d60d00c80c7348727735ae21bd327e5181a456ef28fe8419" exitCode=0
Sep 15 10:24:49.115884 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:49.115815    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pd9x9" event={"ID":"f7e31e0f-4739-482b-88e2-8e7a6f609edf","Type":"ContainerDied","Data":"75a1c823c208ba49d60d00c80c7348727735ae21bd327e5181a456ef28fe8419"}
Sep 15 10:24:50.127725 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:50.127689    2583 generic.go:356] "Generic (PLEG): container finished" podID="1afa8705-52b1-480b-9db9-68ba1800f62f" containerID="1a726f2356a2c1348efb8f2d37f00b1cf374367e022fc466e9ac03d6b2a10cfd" exitCode=0
Sep 15 10:24:50.128140 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:50.127776    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69gc6m" event={"ID":"1afa8705-52b1-480b-9db9-68ba1800f62f","Type":"ContainerDied","Data":"1a726f2356a2c1348efb8f2d37f00b1cf374367e022fc466e9ac03d6b2a10cfd"}
Sep 15 10:24:50.129881 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:50.129860    2583 generic.go:356] "Generic (PLEG): container finished" podID="f561c96d-1b78-4c40-891d-ae150a4592c0" containerID="90fd549244b001d0d4d2f56c51fc080de4c18d4e5934e99a73a9bf4a5df6ebfe" exitCode=0
Sep 15 10:24:50.129971 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:50.129922    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7hqz2" event={"ID":"f561c96d-1b78-4c40-891d-ae150a4592c0","Type":"ContainerDied","Data":"90fd549244b001d0d4d2f56c51fc080de4c18d4e5934e99a73a9bf4a5df6ebfe"}
Sep 15 10:24:50.131942 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:50.131923    2583 generic.go:356] "Generic (PLEG): container finished" podID="f7e31e0f-4739-482b-88e2-8e7a6f609edf" containerID="59d1a9562ceffcb165a0ff25abf6cdebce85c797fc18c4cd5aec7a06f06f8e42" exitCode=0
Sep 15 10:24:50.132043 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:50.131946    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pd9x9" event={"ID":"f7e31e0f-4739-482b-88e2-8e7a6f609edf","Type":"ContainerDied","Data":"59d1a9562ceffcb165a0ff25abf6cdebce85c797fc18c4cd5aec7a06f06f8e42"}
Sep 15 10:24:50.133711 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:50.133694    2583 generic.go:356] "Generic (PLEG): container finished" podID="7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9" containerID="d92ae7066518c348ac3699a5f62267cd2859411e1179fc99e6531ff1a5a60634" exitCode=0
Sep 15 10:24:50.133790 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:50.133722    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajh8k4" event={"ID":"7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9","Type":"ContainerDied","Data":"d92ae7066518c348ac3699a5f62267cd2859411e1179fc99e6531ff1a5a60634"}
Sep 15 10:24:51.140596 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:51.140559    2583 generic.go:356] "Generic (PLEG): container finished" podID="7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9" containerID="c220fb5a7b5b55ba9a96b6bbb3723436b5d602e35a7c0d2587d5bd632c9e749b" exitCode=0
Sep 15 10:24:51.140957 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:51.140649    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajh8k4" event={"ID":"7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9","Type":"ContainerDied","Data":"c220fb5a7b5b55ba9a96b6bbb3723436b5d602e35a7c0d2587d5bd632c9e749b"}
Sep 15 10:24:51.278121 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:51.278097    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69gc6m"
Sep 15 10:24:51.323518 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:51.323495    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pd9x9"
Sep 15 10:24:51.326347 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:51.326327    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7hqz2"
Sep 15 10:24:51.448791 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:51.448705    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f561c96d-1b78-4c40-891d-ae150a4592c0-bundle\") pod \"f561c96d-1b78-4c40-891d-ae150a4592c0\" (UID: \"f561c96d-1b78-4c40-891d-ae150a4592c0\") "
Sep 15 10:24:51.448791 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:51.448773    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7e31e0f-4739-482b-88e2-8e7a6f609edf-util\") pod \"f7e31e0f-4739-482b-88e2-8e7a6f609edf\" (UID: \"f7e31e0f-4739-482b-88e2-8e7a6f609edf\") "
Sep 15 10:24:51.449015 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:51.448805    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7e31e0f-4739-482b-88e2-8e7a6f609edf-bundle\") pod \"f7e31e0f-4739-482b-88e2-8e7a6f609edf\" (UID: \"f7e31e0f-4739-482b-88e2-8e7a6f609edf\") "
Sep 15 10:24:51.449015 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:51.448831    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4qfj9\" (UniqueName: \"kubernetes.io/projected/f561c96d-1b78-4c40-891d-ae150a4592c0-kube-api-access-4qfj9\") pod \"f561c96d-1b78-4c40-891d-ae150a4592c0\" (UID: \"f561c96d-1b78-4c40-891d-ae150a4592c0\") "
Sep 15 10:24:51.449015 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:51.448875    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnltq\" (UniqueName: \"kubernetes.io/projected/1afa8705-52b1-480b-9db9-68ba1800f62f-kube-api-access-xnltq\") pod \"1afa8705-52b1-480b-9db9-68ba1800f62f\" (UID: \"1afa8705-52b1-480b-9db9-68ba1800f62f\") "
Sep 15 10:24:51.449015 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:51.448900    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f561c96d-1b78-4c40-891d-ae150a4592c0-util\") pod \"f561c96d-1b78-4c40-891d-ae150a4592c0\" (UID: \"f561c96d-1b78-4c40-891d-ae150a4592c0\") "
Sep 15 10:24:51.449015 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:51.448947    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1afa8705-52b1-480b-9db9-68ba1800f62f-bundle\") pod \"1afa8705-52b1-480b-9db9-68ba1800f62f\" (UID: \"1afa8705-52b1-480b-9db9-68ba1800f62f\") "
Sep 15 10:24:51.449257 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:51.449139    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1afa8705-52b1-480b-9db9-68ba1800f62f-util\") pod \"1afa8705-52b1-480b-9db9-68ba1800f62f\" (UID: \"1afa8705-52b1-480b-9db9-68ba1800f62f\") "
Sep 15 10:24:51.449257 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:51.449169    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxhjj\" (UniqueName: \"kubernetes.io/projected/f7e31e0f-4739-482b-88e2-8e7a6f609edf-kube-api-access-fxhjj\") pod \"f7e31e0f-4739-482b-88e2-8e7a6f609edf\" (UID: \"f7e31e0f-4739-482b-88e2-8e7a6f609edf\") "
Sep 15 10:24:51.449482 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:51.449429    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f561c96d-1b78-4c40-891d-ae150a4592c0-bundle" (OuterVolumeSpecName: "bundle") pod "f561c96d-1b78-4c40-891d-ae150a4592c0" (UID: "f561c96d-1b78-4c40-891d-ae150a4592c0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:24:51.449573 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:51.449478    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7e31e0f-4739-482b-88e2-8e7a6f609edf-bundle" (OuterVolumeSpecName: "bundle") pod "f7e31e0f-4739-482b-88e2-8e7a6f609edf" (UID: "f7e31e0f-4739-482b-88e2-8e7a6f609edf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:24:51.449661 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:51.449617    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f561c96d-1b78-4c40-891d-ae150a4592c0-bundle\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:24:51.449661 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:51.449635    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7e31e0f-4739-482b-88e2-8e7a6f609edf-bundle\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:24:51.450004 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:51.449968    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1afa8705-52b1-480b-9db9-68ba1800f62f-bundle" (OuterVolumeSpecName: "bundle") pod "1afa8705-52b1-480b-9db9-68ba1800f62f" (UID: "1afa8705-52b1-480b-9db9-68ba1800f62f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:24:51.451523 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:51.451472    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f561c96d-1b78-4c40-891d-ae150a4592c0-kube-api-access-4qfj9" (OuterVolumeSpecName: "kube-api-access-4qfj9") pod "f561c96d-1b78-4c40-891d-ae150a4592c0" (UID: "f561c96d-1b78-4c40-891d-ae150a4592c0"). InnerVolumeSpecName "kube-api-access-4qfj9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:24:51.451624 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:51.451585    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1afa8705-52b1-480b-9db9-68ba1800f62f-kube-api-access-xnltq" (OuterVolumeSpecName: "kube-api-access-xnltq") pod "1afa8705-52b1-480b-9db9-68ba1800f62f" (UID: "1afa8705-52b1-480b-9db9-68ba1800f62f"). InnerVolumeSpecName "kube-api-access-xnltq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:24:51.452019 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:51.451996    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7e31e0f-4739-482b-88e2-8e7a6f609edf-kube-api-access-fxhjj" (OuterVolumeSpecName: "kube-api-access-fxhjj") pod "f7e31e0f-4739-482b-88e2-8e7a6f609edf" (UID: "f7e31e0f-4739-482b-88e2-8e7a6f609edf"). InnerVolumeSpecName "kube-api-access-fxhjj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:24:51.455269 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:51.455229    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f561c96d-1b78-4c40-891d-ae150a4592c0-util" (OuterVolumeSpecName: "util") pod "f561c96d-1b78-4c40-891d-ae150a4592c0" (UID: "f561c96d-1b78-4c40-891d-ae150a4592c0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:24:51.455364 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:51.455324    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7e31e0f-4739-482b-88e2-8e7a6f609edf-util" (OuterVolumeSpecName: "util") pod "f7e31e0f-4739-482b-88e2-8e7a6f609edf" (UID: "f7e31e0f-4739-482b-88e2-8e7a6f609edf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:24:51.455861 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:51.455841    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1afa8705-52b1-480b-9db9-68ba1800f62f-util" (OuterVolumeSpecName: "util") pod "1afa8705-52b1-480b-9db9-68ba1800f62f" (UID: "1afa8705-52b1-480b-9db9-68ba1800f62f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:24:51.550742 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:51.550703    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qfj9\" (UniqueName: \"kubernetes.io/projected/f561c96d-1b78-4c40-891d-ae150a4592c0-kube-api-access-4qfj9\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:24:51.550742 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:51.550731    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnltq\" (UniqueName: \"kubernetes.io/projected/1afa8705-52b1-480b-9db9-68ba1800f62f-kube-api-access-xnltq\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:24:51.550742 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:51.550744    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f561c96d-1b78-4c40-891d-ae150a4592c0-util\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:24:51.550969 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:51.550756    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1afa8705-52b1-480b-9db9-68ba1800f62f-bundle\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:24:51.550969 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:51.550767    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1afa8705-52b1-480b-9db9-68ba1800f62f-util\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:24:51.550969 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:51.550777    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxhjj\" (UniqueName: \"kubernetes.io/projected/f7e31e0f-4739-482b-88e2-8e7a6f609edf-kube-api-access-fxhjj\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:24:51.550969 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:51.550787    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7e31e0f-4739-482b-88e2-8e7a6f609edf-util\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:24:52.148089 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:52.148057    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69gc6m" event={"ID":"1afa8705-52b1-480b-9db9-68ba1800f62f","Type":"ContainerDied","Data":"1417c0fdb17e38cf4a07c41ec7e2ccbbdd70e6ddb3c1065bbfdf7e6c9c8e60d6"}
Sep 15 10:24:52.148506 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:52.148109    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1417c0fdb17e38cf4a07c41ec7e2ccbbdd70e6ddb3c1065bbfdf7e6c9c8e60d6"
Sep 15 10:24:52.148506 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:52.148070    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69gc6m"
Sep 15 10:24:52.150147 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:52.150115    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7hqz2" event={"ID":"f561c96d-1b78-4c40-891d-ae150a4592c0","Type":"ContainerDied","Data":"a6ae84b04c1221575aadf2ac5635cf4d401c9642f2ebe669f3615f07b78e9ec7"}
Sep 15 10:24:52.150260 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:52.150153    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a6ae84b04c1221575aadf2ac5635cf4d401c9642f2ebe669f3615f07b78e9ec7"
Sep 15 10:24:52.150260 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:52.150120    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c7hqz2"
Sep 15 10:24:52.152015 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:52.151997    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pd9x9"
Sep 15 10:24:52.152102 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:52.152030    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pd9x9" event={"ID":"f7e31e0f-4739-482b-88e2-8e7a6f609edf","Type":"ContainerDied","Data":"1ec46f688a47c9ac3d2ae0563e713ab2d3026a14ca9e11cde8bd817bca7b34db"}
Sep 15 10:24:52.152102 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:52.152072    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1ec46f688a47c9ac3d2ae0563e713ab2d3026a14ca9e11cde8bd817bca7b34db"
Sep 15 10:24:52.289159 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:52.289137    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajh8k4"
Sep 15 10:24:52.457864 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:52.457761    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9-util\") pod \"7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9\" (UID: \"7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9\") "
Sep 15 10:24:52.457864 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:52.457825    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gs4bw\" (UniqueName: \"kubernetes.io/projected/7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9-kube-api-access-gs4bw\") pod \"7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9\" (UID: \"7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9\") "
Sep 15 10:24:52.458068 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:52.457877    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9-bundle\") pod \"7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9\" (UID: \"7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9\") "
Sep 15 10:24:52.458742 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:52.458700    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9-bundle" (OuterVolumeSpecName: "bundle") pod "7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9" (UID: "7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:24:52.460004 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:52.459974    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9-kube-api-access-gs4bw" (OuterVolumeSpecName: "kube-api-access-gs4bw") pod "7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9" (UID: "7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9"). InnerVolumeSpecName "kube-api-access-gs4bw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:24:52.463453 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:52.463406    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9-util" (OuterVolumeSpecName: "util") pod "7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9" (UID: "7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:24:52.559217 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:52.559182    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gs4bw\" (UniqueName: \"kubernetes.io/projected/7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9-kube-api-access-gs4bw\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:24:52.559217 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:52.559209    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9-bundle\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:24:52.559217 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:52.559218    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9-util\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:24:53.164896 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:53.164865    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajh8k4"
Sep 15 10:24:53.164896 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:53.164881    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ajh8k4" event={"ID":"7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9","Type":"ContainerDied","Data":"a6bbbff4862c062072941410f53c990056d8d418d3d193e6132cb6046046bde3"}
Sep 15 10:24:53.165464 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:53.164945    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a6bbbff4862c062072941410f53c990056d8d418d3d193e6132cb6046046bde3"
Sep 15 10:24:57.078627 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:57.078591    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:24:59.465218 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:59.465182    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:24:59.863141 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:24:59.863058    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:01.100282 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:01.100246    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5f4bdf6bf5-6clb8" podUID="fb5990c4-0a70-4f63-a145-0d81cf3e35c7" containerName="console" containerID="cri-o://e0fa95c50352dfc1643e71185164fe5c6105a6b7031edb3cd41a2e1ffef5a3d8" gracePeriod=15
Sep 15 10:25:01.289867 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:01.289832    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:01.343733 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:01.343703    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f4bdf6bf5-6clb8"
Sep 15 10:25:01.438829 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:01.438793    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-trusted-ca-bundle\") pod \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\" (UID: \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\") "
Sep 15 10:25:01.438829 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:01.438834    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-console-serving-cert\") pod \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\" (UID: \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\") "
Sep 15 10:25:01.439065 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:01.438892    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-service-ca\") pod \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\" (UID: \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\") "
Sep 15 10:25:01.439065 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:01.438928    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-console-config\") pod \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\" (UID: \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\") "
Sep 15 10:25:01.439065 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:01.438950    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-console-oauth-config\") pod \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\" (UID: \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\") "
Sep 15 10:25:01.439065 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:01.438966    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-oauth-serving-cert\") pod \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\" (UID: \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\") "
Sep 15 10:25:01.439253 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:01.439058    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-skdmk\" (UniqueName: \"kubernetes.io/projected/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-kube-api-access-skdmk\") pod \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\" (UID: \"fb5990c4-0a70-4f63-a145-0d81cf3e35c7\") "
Sep 15 10:25:01.439370 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:01.439346    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fb5990c4-0a70-4f63-a145-0d81cf3e35c7" (UID: "fb5990c4-0a70-4f63-a145-0d81cf3e35c7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:25:01.439370 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:01.439353    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-service-ca" (OuterVolumeSpecName: "service-ca") pod "fb5990c4-0a70-4f63-a145-0d81cf3e35c7" (UID: "fb5990c4-0a70-4f63-a145-0d81cf3e35c7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:25:01.439495 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:01.439357    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fb5990c4-0a70-4f63-a145-0d81cf3e35c7" (UID: "fb5990c4-0a70-4f63-a145-0d81cf3e35c7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:25:01.439495 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:01.439436    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-console-config" (OuterVolumeSpecName: "console-config") pod "fb5990c4-0a70-4f63-a145-0d81cf3e35c7" (UID: "fb5990c4-0a70-4f63-a145-0d81cf3e35c7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:25:01.441194 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:01.441169    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fb5990c4-0a70-4f63-a145-0d81cf3e35c7" (UID: "fb5990c4-0a70-4f63-a145-0d81cf3e35c7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:25:01.441194 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:01.441167    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-kube-api-access-skdmk" (OuterVolumeSpecName: "kube-api-access-skdmk") pod "fb5990c4-0a70-4f63-a145-0d81cf3e35c7" (UID: "fb5990c4-0a70-4f63-a145-0d81cf3e35c7"). InnerVolumeSpecName "kube-api-access-skdmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:25:01.441322 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:01.441232    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fb5990c4-0a70-4f63-a145-0d81cf3e35c7" (UID: "fb5990c4-0a70-4f63-a145-0d81cf3e35c7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:25:01.540707 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:01.540676    2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-service-ca\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:25:01.540707 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:01.540701    2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-console-config\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:25:01.540707 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:01.540710    2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-console-oauth-config\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:25:01.540935 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:01.540720    2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-oauth-serving-cert\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:25:01.540935 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:01.540730    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-skdmk\" (UniqueName: \"kubernetes.io/projected/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-kube-api-access-skdmk\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:25:01.540935 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:01.540741    2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-trusted-ca-bundle\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:25:01.540935 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:01.540751    2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb5990c4-0a70-4f63-a145-0d81cf3e35c7-console-serving-cert\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:25:02.212593 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:02.212557    2583 generic.go:356] "Generic (PLEG): container finished" podID="fb5990c4-0a70-4f63-a145-0d81cf3e35c7" containerID="e0fa95c50352dfc1643e71185164fe5c6105a6b7031edb3cd41a2e1ffef5a3d8" exitCode=0
Sep 15 10:25:02.213042 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:02.212605    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f4bdf6bf5-6clb8"
Sep 15 10:25:02.213042 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:02.212656    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f4bdf6bf5-6clb8" event={"ID":"fb5990c4-0a70-4f63-a145-0d81cf3e35c7","Type":"ContainerDied","Data":"e0fa95c50352dfc1643e71185164fe5c6105a6b7031edb3cd41a2e1ffef5a3d8"}
Sep 15 10:25:02.213042 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:02.212716    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f4bdf6bf5-6clb8" event={"ID":"fb5990c4-0a70-4f63-a145-0d81cf3e35c7","Type":"ContainerDied","Data":"6fdb950a6b8a649644b53eb009e33bb892ddec2651a9bf98258b05ca94ab39c9"}
Sep 15 10:25:02.213042 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:02.212752    2583 scope.go:117] "RemoveContainer" containerID="e0fa95c50352dfc1643e71185164fe5c6105a6b7031edb3cd41a2e1ffef5a3d8"
Sep 15 10:25:02.222485 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:02.222428    2583 scope.go:117] "RemoveContainer" containerID="e0fa95c50352dfc1643e71185164fe5c6105a6b7031edb3cd41a2e1ffef5a3d8"
Sep 15 10:25:02.222716 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:25:02.222699    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0fa95c50352dfc1643e71185164fe5c6105a6b7031edb3cd41a2e1ffef5a3d8\": container with ID starting with e0fa95c50352dfc1643e71185164fe5c6105a6b7031edb3cd41a2e1ffef5a3d8 not found: ID does not exist" containerID="e0fa95c50352dfc1643e71185164fe5c6105a6b7031edb3cd41a2e1ffef5a3d8"
Sep 15 10:25:02.222759 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:02.222723    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0fa95c50352dfc1643e71185164fe5c6105a6b7031edb3cd41a2e1ffef5a3d8"} err="failed to get container status \"e0fa95c50352dfc1643e71185164fe5c6105a6b7031edb3cd41a2e1ffef5a3d8\": rpc error: code = NotFound desc = could not find container \"e0fa95c50352dfc1643e71185164fe5c6105a6b7031edb3cd41a2e1ffef5a3d8\": container with ID starting with e0fa95c50352dfc1643e71185164fe5c6105a6b7031edb3cd41a2e1ffef5a3d8 not found: ID does not exist"
Sep 15 10:25:02.235570 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:02.235550    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f4bdf6bf5-6clb8"]
Sep 15 10:25:02.241792 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:02.241774    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f4bdf6bf5-6clb8"]
Sep 15 10:25:02.463971 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:02.463899    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:02.715222 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:02.715146    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:03.300179 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:03.300140    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb5990c4-0a70-4f63-a145-0d81cf3e35c7" path="/var/lib/kubelet/pods/fb5990c4-0a70-4f63-a145-0d81cf3e35c7/volumes"
Sep 15 10:25:04.173567 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:04.173534    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:04.663645 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:04.663613    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:04.902163 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:04.902129    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:05.055057 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:05.054971    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:05.166178 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:05.166140    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:05.459087 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:05.459057    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:05.571617 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:05.571584    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:10.489042 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:10.489006    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:10.581885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:10.581850    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:14.925841 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:14.925811    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:15.109303 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:15.109275    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:15.210783 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:15.210696    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:15.456331 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:15.456277    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:18.102791 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:18.102750    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:18.179168 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:18.179129    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:22.838292 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:22.838242    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:22.947965 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:22.947933    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:23.048777 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:23.048748    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:29.511377 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:29.511303    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:29.735076 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:29.735043    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:34.897712 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:34.897681    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:34.950361 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:34.950322    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:35.295419 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:35.295341    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:35.358992 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:35.358966    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:38.606583 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:38.606550    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:38.742068 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:38.742012    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:38.845105 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:38.845056    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:42.459160 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:42.459124    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:42.558417 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:42.558379    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:42.659277 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:42.659240    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:54.410817 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:54.410765    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:54.465948 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:54.465915    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:54.605578 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:54.605544    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:25:55.160018 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:25:55.159977    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:26:08.313200 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:26:08.313164    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:26:08.416027 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:26:08.415998    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:26:11.008386 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:26:11.008338    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:26:11.115312 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:26:11.115280    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:26:11.176268 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:26:11.176239    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:26:11.824978 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:26:11.824947    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:26:11.914368 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:26:11.914335    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:26:30.783213 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:26:30.783178    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:26:30.886705 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:26:30.886670    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:26:31.224856 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:26:31.224828    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:26:42.949147 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:26:42.949111    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:26:43.820765 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:26:43.820728    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:26:49.557954 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:26:49.557916    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:26:49.656388 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:26:49.656362    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:26:49.749801 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:26:49.749770    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:26:49.807431 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:26:49.807400    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:26:49.866889 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:26:49.866797    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:26:52.997474 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:26:52.997422    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:26:53.095469 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:26:53.095409    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:27:24.473919 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:24.473875    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:27:24.573694 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:24.573660    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:27:25.856883 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.856828    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:27:25.954936 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.954897    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:27:25.961859 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.961833    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5b77bbc99d-756d5"]
Sep 15 10:27:25.962172 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.962161    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f561c96d-1b78-4c40-891d-ae150a4592c0" containerName="extract"
Sep 15 10:27:25.962219 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.962173    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f561c96d-1b78-4c40-891d-ae150a4592c0" containerName="extract"
Sep 15 10:27:25.962219 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.962181    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9" containerName="pull"
Sep 15 10:27:25.962219 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.962186    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9" containerName="pull"
Sep 15 10:27:25.962219 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.962196    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f561c96d-1b78-4c40-891d-ae150a4592c0" containerName="util"
Sep 15 10:27:25.962219 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.962202    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f561c96d-1b78-4c40-891d-ae150a4592c0" containerName="util"
Sep 15 10:27:25.962219 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.962213    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1afa8705-52b1-480b-9db9-68ba1800f62f" containerName="util"
Sep 15 10:27:25.962219 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.962218    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1afa8705-52b1-480b-9db9-68ba1800f62f" containerName="util"
Sep 15 10:27:25.962432 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.962224    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f561c96d-1b78-4c40-891d-ae150a4592c0" containerName="pull"
Sep 15 10:27:25.962432 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.962229    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f561c96d-1b78-4c40-891d-ae150a4592c0" containerName="pull"
Sep 15 10:27:25.962432 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.962238    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7e31e0f-4739-482b-88e2-8e7a6f609edf" containerName="extract"
Sep 15 10:27:25.962432 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.962243    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7e31e0f-4739-482b-88e2-8e7a6f609edf" containerName="extract"
Sep 15 10:27:25.962432 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.962249    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1afa8705-52b1-480b-9db9-68ba1800f62f" containerName="extract"
Sep 15 10:27:25.962432 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.962254    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1afa8705-52b1-480b-9db9-68ba1800f62f" containerName="extract"
Sep 15 10:27:25.962432 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.962260    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9" containerName="extract"
Sep 15 10:27:25.962432 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.962265    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9" containerName="extract"
Sep 15 10:27:25.962432 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.962272    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9" containerName="util"
Sep 15 10:27:25.962432 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.962279    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9" containerName="util"
Sep 15 10:27:25.962432 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.962288    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb5990c4-0a70-4f63-a145-0d81cf3e35c7" containerName="console"
Sep 15 10:27:25.962432 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.962293    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb5990c4-0a70-4f63-a145-0d81cf3e35c7" containerName="console"
Sep 15 10:27:25.962432 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.962298    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7e31e0f-4739-482b-88e2-8e7a6f609edf" containerName="util"
Sep 15 10:27:25.962432 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.962303    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7e31e0f-4739-482b-88e2-8e7a6f609edf" containerName="util"
Sep 15 10:27:25.962432 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.962312    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7e31e0f-4739-482b-88e2-8e7a6f609edf" containerName="pull"
Sep 15 10:27:25.962432 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.962317    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7e31e0f-4739-482b-88e2-8e7a6f609edf" containerName="pull"
Sep 15 10:27:25.962432 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.962324    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1afa8705-52b1-480b-9db9-68ba1800f62f" containerName="pull"
Sep 15 10:27:25.962432 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.962328    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1afa8705-52b1-480b-9db9-68ba1800f62f" containerName="pull"
Sep 15 10:27:25.962432 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.962391    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb5990c4-0a70-4f63-a145-0d81cf3e35c7" containerName="console"
Sep 15 10:27:25.962432 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.962397    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f561c96d-1b78-4c40-891d-ae150a4592c0" containerName="extract"
Sep 15 10:27:25.962432 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.962402    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bad47d0-dfe4-42c7-ab09-3e66b0ae75f9" containerName="extract"
Sep 15 10:27:25.962432 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.962408    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1afa8705-52b1-480b-9db9-68ba1800f62f" containerName="extract"
Sep 15 10:27:25.962432 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.962413    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7e31e0f-4739-482b-88e2-8e7a6f609edf" containerName="extract"
Sep 15 10:27:25.965494 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.965476    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5b77bbc99d-756d5"
Sep 15 10:27:25.967724 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.967707    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 15 10:27:25.968322 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.968300    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-dkr42\""
Sep 15 10:27:25.969172 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.969152    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 15 10:27:25.969284 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.969269    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 15 10:27:25.969358 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.969295    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 15 10:27:25.976290 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:25.976259    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5b77bbc99d-756d5"]
Sep 15 10:27:26.015662 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:26.015627    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6s9f\" (UniqueName: \"kubernetes.io/projected/29bb1f2b-9b06-4d79-935a-c7a99dc43dd7-kube-api-access-c6s9f\") pod \"maas-api-5b77bbc99d-756d5\" (UID: \"29bb1f2b-9b06-4d79-935a-c7a99dc43dd7\") " pod="odh-ai-gateway-infra/maas-api-5b77bbc99d-756d5"
Sep 15 10:27:26.015818 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:26.015703    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29bb1f2b-9b06-4d79-935a-c7a99dc43dd7-metrics-certs\") pod \"maas-api-5b77bbc99d-756d5\" (UID: \"29bb1f2b-9b06-4d79-935a-c7a99dc43dd7\") " pod="odh-ai-gateway-infra/maas-api-5b77bbc99d-756d5"
Sep 15 10:27:26.015818 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:26.015756    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29bb1f2b-9b06-4d79-935a-c7a99dc43dd7-openshift-service-ca\") pod \"maas-api-5b77bbc99d-756d5\" (UID: \"29bb1f2b-9b06-4d79-935a-c7a99dc43dd7\") " pod="odh-ai-gateway-infra/maas-api-5b77bbc99d-756d5"
Sep 15 10:27:26.015818 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:26.015791    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29bb1f2b-9b06-4d79-935a-c7a99dc43dd7-maas-api-tls\") pod \"maas-api-5b77bbc99d-756d5\" (UID: \"29bb1f2b-9b06-4d79-935a-c7a99dc43dd7\") " pod="odh-ai-gateway-infra/maas-api-5b77bbc99d-756d5"
Sep 15 10:27:26.033313 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:26.033289    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:27:26.117290 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:26.117205    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6s9f\" (UniqueName: \"kubernetes.io/projected/29bb1f2b-9b06-4d79-935a-c7a99dc43dd7-kube-api-access-c6s9f\") pod \"maas-api-5b77bbc99d-756d5\" (UID: \"29bb1f2b-9b06-4d79-935a-c7a99dc43dd7\") " pod="odh-ai-gateway-infra/maas-api-5b77bbc99d-756d5"
Sep 15 10:27:26.117290 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:26.117273    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29bb1f2b-9b06-4d79-935a-c7a99dc43dd7-metrics-certs\") pod \"maas-api-5b77bbc99d-756d5\" (UID: \"29bb1f2b-9b06-4d79-935a-c7a99dc43dd7\") " pod="odh-ai-gateway-infra/maas-api-5b77bbc99d-756d5"
Sep 15 10:27:26.117569 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:26.117305    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29bb1f2b-9b06-4d79-935a-c7a99dc43dd7-openshift-service-ca\") pod \"maas-api-5b77bbc99d-756d5\" (UID: \"29bb1f2b-9b06-4d79-935a-c7a99dc43dd7\") " pod="odh-ai-gateway-infra/maas-api-5b77bbc99d-756d5"
Sep 15 10:27:26.117569 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:26.117343    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29bb1f2b-9b06-4d79-935a-c7a99dc43dd7-maas-api-tls\") pod \"maas-api-5b77bbc99d-756d5\" (UID: \"29bb1f2b-9b06-4d79-935a-c7a99dc43dd7\") " pod="odh-ai-gateway-infra/maas-api-5b77bbc99d-756d5"
Sep 15 10:27:26.117569 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:27:26.117436    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Sep 15 10:27:26.117569 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:27:26.117542    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29bb1f2b-9b06-4d79-935a-c7a99dc43dd7-maas-api-tls podName:29bb1f2b-9b06-4d79-935a-c7a99dc43dd7 nodeName:}" failed. No retries permitted until 2026-09-15 10:27:26.617520079 +0000 UTC m=+561.864487043 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/29bb1f2b-9b06-4d79-935a-c7a99dc43dd7-maas-api-tls") pod "maas-api-5b77bbc99d-756d5" (UID: "29bb1f2b-9b06-4d79-935a-c7a99dc43dd7") : secret "maas-api-serving-cert" not found
Sep 15 10:27:26.118139 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:26.118112    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29bb1f2b-9b06-4d79-935a-c7a99dc43dd7-openshift-service-ca\") pod \"maas-api-5b77bbc99d-756d5\" (UID: \"29bb1f2b-9b06-4d79-935a-c7a99dc43dd7\") " pod="odh-ai-gateway-infra/maas-api-5b77bbc99d-756d5"
Sep 15 10:27:26.119703 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:26.119682    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29bb1f2b-9b06-4d79-935a-c7a99dc43dd7-metrics-certs\") pod \"maas-api-5b77bbc99d-756d5\" (UID: \"29bb1f2b-9b06-4d79-935a-c7a99dc43dd7\") " pod="odh-ai-gateway-infra/maas-api-5b77bbc99d-756d5"
Sep 15 10:27:26.127562 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:26.127537    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6s9f\" (UniqueName: \"kubernetes.io/projected/29bb1f2b-9b06-4d79-935a-c7a99dc43dd7-kube-api-access-c6s9f\") pod \"maas-api-5b77bbc99d-756d5\" (UID: \"29bb1f2b-9b06-4d79-935a-c7a99dc43dd7\") " pod="odh-ai-gateway-infra/maas-api-5b77bbc99d-756d5"
Sep 15 10:27:26.220365 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:26.220331    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:27:26.327221 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:26.327194    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:27:26.426312 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:26.426275    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:27:26.621526 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:26.621489    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29bb1f2b-9b06-4d79-935a-c7a99dc43dd7-maas-api-tls\") pod \"maas-api-5b77bbc99d-756d5\" (UID: \"29bb1f2b-9b06-4d79-935a-c7a99dc43dd7\") " pod="odh-ai-gateway-infra/maas-api-5b77bbc99d-756d5"
Sep 15 10:27:26.623897 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:26.623871    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29bb1f2b-9b06-4d79-935a-c7a99dc43dd7-maas-api-tls\") pod \"maas-api-5b77bbc99d-756d5\" (UID: \"29bb1f2b-9b06-4d79-935a-c7a99dc43dd7\") " pod="odh-ai-gateway-infra/maas-api-5b77bbc99d-756d5"
Sep 15 10:27:26.877278 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:26.877187    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5b77bbc99d-756d5"
Sep 15 10:27:27.148930 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:27.148907    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5b77bbc99d-756d5"]
Sep 15 10:27:27.150915 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:27:27.150879    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29bb1f2b_9b06_4d79_935a_c7a99dc43dd7.slice/crio-e6ee94d1d954c79f34f32808ebcfcbfdb4966dd82b0897c4d6601e25db88bc0f WatchSource:0}: Error finding container e6ee94d1d954c79f34f32808ebcfcbfdb4966dd82b0897c4d6601e25db88bc0f: Status 404 returned error can't find the container with id e6ee94d1d954c79f34f32808ebcfcbfdb4966dd82b0897c4d6601e25db88bc0f
Sep 15 10:27:27.965897 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:27.965863    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5b77bbc99d-756d5" event={"ID":"29bb1f2b-9b06-4d79-935a-c7a99dc43dd7","Type":"ContainerStarted","Data":"e6ee94d1d954c79f34f32808ebcfcbfdb4966dd82b0897c4d6601e25db88bc0f"}
Sep 15 10:27:30.983579 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:30.983540    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5b77bbc99d-756d5" event={"ID":"29bb1f2b-9b06-4d79-935a-c7a99dc43dd7","Type":"ContainerStarted","Data":"ffecff1f56a64bb97f95b46decbf7932abbdd91e79457d41724122b788ee719a"}
Sep 15 10:27:30.983951 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:30.983691    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5b77bbc99d-756d5"
Sep 15 10:27:30.999873 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:30.999803    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5b77bbc99d-756d5" podStartSLOduration=3.221546221 podStartE2EDuration="5.999788843s" totalImagesPullingTime="2.778242622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:27:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:27:30.998784342 +0000 UTC m=+566.245751305" watchObservedRunningTime="2026-09-15 10:27:30.999788843 +0000 UTC m=+566.246755813"
Sep 15 10:27:36.994946 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:36.994919    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5b77bbc99d-756d5"
Sep 15 10:27:37.954167 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:37.954132    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:27:38.083920 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:38.083887    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:27:59.180268 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:27:59.180238    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:28:04.943613 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:04.943581    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-wttp4"]
Sep 15 10:28:04.946843 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:04.946823    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wttp4"
Sep 15 10:28:04.949561 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:04.949536    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 15 10:28:04.950361 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:04.950338    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 15 10:28:04.950507 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:04.950363    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-fwljq\""
Sep 15 10:28:04.950507 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:04.950369    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 15 10:28:04.954043 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:04.954020    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2gfj\" (UniqueName: \"kubernetes.io/projected/0622f16d-0989-453a-bb5c-0e7ed6669318-kube-api-access-w2gfj\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wttp4\" (UID: \"0622f16d-0989-453a-bb5c-0e7ed6669318\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wttp4"
Sep 15 10:28:04.954162 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:04.954070    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0622f16d-0989-453a-bb5c-0e7ed6669318-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wttp4\" (UID: \"0622f16d-0989-453a-bb5c-0e7ed6669318\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wttp4"
Sep 15 10:28:04.954162 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:04.954097    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0622f16d-0989-453a-bb5c-0e7ed6669318-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wttp4\" (UID: \"0622f16d-0989-453a-bb5c-0e7ed6669318\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wttp4"
Sep 15 10:28:04.954162 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:04.954129    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0622f16d-0989-453a-bb5c-0e7ed6669318-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wttp4\" (UID: \"0622f16d-0989-453a-bb5c-0e7ed6669318\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wttp4"
Sep 15 10:28:04.954324 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:04.954183    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0622f16d-0989-453a-bb5c-0e7ed6669318-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wttp4\" (UID: \"0622f16d-0989-453a-bb5c-0e7ed6669318\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wttp4"
Sep 15 10:28:04.954324 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:04.954249    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0622f16d-0989-453a-bb5c-0e7ed6669318-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wttp4\" (UID: \"0622f16d-0989-453a-bb5c-0e7ed6669318\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wttp4"
Sep 15 10:28:04.955861 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:04.955842    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-wttp4"]
Sep 15 10:28:04.984457 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:04.984421    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:28:05.054737 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:05.054711    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0622f16d-0989-453a-bb5c-0e7ed6669318-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wttp4\" (UID: \"0622f16d-0989-453a-bb5c-0e7ed6669318\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wttp4"
Sep 15 10:28:05.054872 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:05.054761    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0622f16d-0989-453a-bb5c-0e7ed6669318-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wttp4\" (UID: \"0622f16d-0989-453a-bb5c-0e7ed6669318\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wttp4"
Sep 15 10:28:05.054872 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:05.054820    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2gfj\" (UniqueName: \"kubernetes.io/projected/0622f16d-0989-453a-bb5c-0e7ed6669318-kube-api-access-w2gfj\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wttp4\" (UID: \"0622f16d-0989-453a-bb5c-0e7ed6669318\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wttp4"
Sep 15 10:28:05.054955 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:05.054876    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0622f16d-0989-453a-bb5c-0e7ed6669318-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wttp4\" (UID: \"0622f16d-0989-453a-bb5c-0e7ed6669318\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wttp4"
Sep 15 10:28:05.054955 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:05.054899    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0622f16d-0989-453a-bb5c-0e7ed6669318-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wttp4\" (UID: \"0622f16d-0989-453a-bb5c-0e7ed6669318\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wttp4"
Sep 15 10:28:05.054955 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:05.054926    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0622f16d-0989-453a-bb5c-0e7ed6669318-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wttp4\" (UID: \"0622f16d-0989-453a-bb5c-0e7ed6669318\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wttp4"
Sep 15 10:28:05.055274 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:05.055249    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0622f16d-0989-453a-bb5c-0e7ed6669318-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wttp4\" (UID: \"0622f16d-0989-453a-bb5c-0e7ed6669318\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wttp4"
Sep 15 10:28:05.055387 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:05.055298    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0622f16d-0989-453a-bb5c-0e7ed6669318-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wttp4\" (UID: \"0622f16d-0989-453a-bb5c-0e7ed6669318\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wttp4"
Sep 15 10:28:05.055387 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:05.055356    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0622f16d-0989-453a-bb5c-0e7ed6669318-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wttp4\" (UID: \"0622f16d-0989-453a-bb5c-0e7ed6669318\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wttp4"
Sep 15 10:28:05.057063 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:05.057034    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0622f16d-0989-453a-bb5c-0e7ed6669318-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wttp4\" (UID: \"0622f16d-0989-453a-bb5c-0e7ed6669318\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wttp4"
Sep 15 10:28:05.057285 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:05.057270    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0622f16d-0989-453a-bb5c-0e7ed6669318-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wttp4\" (UID: \"0622f16d-0989-453a-bb5c-0e7ed6669318\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wttp4"
Sep 15 10:28:05.062944 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:05.062924    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2gfj\" (UniqueName: \"kubernetes.io/projected/0622f16d-0989-453a-bb5c-0e7ed6669318-kube-api-access-w2gfj\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wttp4\" (UID: \"0622f16d-0989-453a-bb5c-0e7ed6669318\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wttp4"
Sep 15 10:28:05.260760 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:05.260690    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-fwljq\""
Sep 15 10:28:05.268584 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:05.268564    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wttp4"
Sep 15 10:28:05.539775 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:05.539752    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-wttp4"]
Sep 15 10:28:05.542014 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:28:05.541977    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0622f16d_0989_453a_bb5c_0e7ed6669318.slice/crio-c5780bd22984d398a957556b9d2d00c1d459afdb503aef01a19fe5d94648578e WatchSource:0}: Error finding container c5780bd22984d398a957556b9d2d00c1d459afdb503aef01a19fe5d94648578e: Status 404 returned error can't find the container with id c5780bd22984d398a957556b9d2d00c1d459afdb503aef01a19fe5d94648578e
Sep 15 10:28:06.176107 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:06.176070    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wttp4" event={"ID":"0622f16d-0989-453a-bb5c-0e7ed6669318","Type":"ContainerStarted","Data":"c5780bd22984d398a957556b9d2d00c1d459afdb503aef01a19fe5d94648578e"}
Sep 15 10:28:07.182970 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:07.182925    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wttp4" event={"ID":"0622f16d-0989-453a-bb5c-0e7ed6669318","Type":"ContainerStarted","Data":"f03543f6af3dbe5523d1be877b9fa4b0c3598f196e1da73b2233f34527053ac3"}
Sep 15 10:28:07.200158 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:07.200112    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wttp4" podStartSLOduration=2.226660076 podStartE2EDuration="3.200087597s" totalImagesPullingTime="973.427521ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:28:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:28:07.198153949 +0000 UTC m=+602.445120920" watchObservedRunningTime="2026-09-15 10:28:07.200087597 +0000 UTC m=+602.447054600"
Sep 15 10:28:10.853098 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:10.853063    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt"]
Sep 15 10:28:10.856502 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:10.856483    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt"
Sep 15 10:28:10.858915 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:10.858893    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 15 10:28:10.868711 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:10.868685    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt"]
Sep 15 10:28:10.878346 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:10.878321    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:28:10.904836 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:10.904807    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf199338-8a03-45c6-be1c-aca427903dcc-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt\" (UID: \"cf199338-8a03-45c6-be1c-aca427903dcc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt"
Sep 15 10:28:10.904945 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:10.904850    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf199338-8a03-45c6-be1c-aca427903dcc-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt\" (UID: \"cf199338-8a03-45c6-be1c-aca427903dcc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt"
Sep 15 10:28:10.904945 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:10.904873    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf199338-8a03-45c6-be1c-aca427903dcc-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt\" (UID: \"cf199338-8a03-45c6-be1c-aca427903dcc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt"
Sep 15 10:28:10.905043 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:10.905005    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf199338-8a03-45c6-be1c-aca427903dcc-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt\" (UID: \"cf199338-8a03-45c6-be1c-aca427903dcc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt"
Sep 15 10:28:10.905092 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:10.905040    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf199338-8a03-45c6-be1c-aca427903dcc-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt\" (UID: \"cf199338-8a03-45c6-be1c-aca427903dcc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt"
Sep 15 10:28:10.905092 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:10.905064    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2rrr\" (UniqueName: \"kubernetes.io/projected/cf199338-8a03-45c6-be1c-aca427903dcc-kube-api-access-m2rrr\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt\" (UID: \"cf199338-8a03-45c6-be1c-aca427903dcc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt"
Sep 15 10:28:11.006293 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:11.006256    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf199338-8a03-45c6-be1c-aca427903dcc-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt\" (UID: \"cf199338-8a03-45c6-be1c-aca427903dcc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt"
Sep 15 10:28:11.006293 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:11.006299    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf199338-8a03-45c6-be1c-aca427903dcc-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt\" (UID: \"cf199338-8a03-45c6-be1c-aca427903dcc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt"
Sep 15 10:28:11.006578 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:11.006320    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2rrr\" (UniqueName: \"kubernetes.io/projected/cf199338-8a03-45c6-be1c-aca427903dcc-kube-api-access-m2rrr\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt\" (UID: \"cf199338-8a03-45c6-be1c-aca427903dcc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt"
Sep 15 10:28:11.006578 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:11.006362    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf199338-8a03-45c6-be1c-aca427903dcc-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt\" (UID: \"cf199338-8a03-45c6-be1c-aca427903dcc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt"
Sep 15 10:28:11.006578 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:11.006382    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf199338-8a03-45c6-be1c-aca427903dcc-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt\" (UID: \"cf199338-8a03-45c6-be1c-aca427903dcc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt"
Sep 15 10:28:11.006578 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:11.006396    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf199338-8a03-45c6-be1c-aca427903dcc-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt\" (UID: \"cf199338-8a03-45c6-be1c-aca427903dcc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt"
Sep 15 10:28:11.006810 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:11.006774    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf199338-8a03-45c6-be1c-aca427903dcc-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt\" (UID: \"cf199338-8a03-45c6-be1c-aca427903dcc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt"
Sep 15 10:28:11.006810 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:11.006794    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf199338-8a03-45c6-be1c-aca427903dcc-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt\" (UID: \"cf199338-8a03-45c6-be1c-aca427903dcc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt"
Sep 15 10:28:11.006884 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:11.006827    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf199338-8a03-45c6-be1c-aca427903dcc-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt\" (UID: \"cf199338-8a03-45c6-be1c-aca427903dcc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt"
Sep 15 10:28:11.008706 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:11.008685    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf199338-8a03-45c6-be1c-aca427903dcc-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt\" (UID: \"cf199338-8a03-45c6-be1c-aca427903dcc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt"
Sep 15 10:28:11.008812 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:11.008797    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf199338-8a03-45c6-be1c-aca427903dcc-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt\" (UID: \"cf199338-8a03-45c6-be1c-aca427903dcc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt"
Sep 15 10:28:11.014321 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:11.014298    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2rrr\" (UniqueName: \"kubernetes.io/projected/cf199338-8a03-45c6-be1c-aca427903dcc-kube-api-access-m2rrr\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt\" (UID: \"cf199338-8a03-45c6-be1c-aca427903dcc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt"
Sep 15 10:28:11.166777 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:11.166746    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt"
Sep 15 10:28:11.436658 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:11.436630    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt"]
Sep 15 10:28:11.437870 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:28:11.437842    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf199338_8a03_45c6_be1c_aca427903dcc.slice/crio-18083ae9a32b65380cc6254c9c82481302880b5eeeeabd1a4f449857631c635a WatchSource:0}: Error finding container 18083ae9a32b65380cc6254c9c82481302880b5eeeeabd1a4f449857631c635a: Status 404 returned error can't find the container with id 18083ae9a32b65380cc6254c9c82481302880b5eeeeabd1a4f449857631c635a
Sep 15 10:28:11.439246 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:11.439226    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 10:28:12.212106 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:12.212069    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt" event={"ID":"cf199338-8a03-45c6-be1c-aca427903dcc","Type":"ContainerStarted","Data":"3578228b8867763ce75d962326d3256cb8fe8b082df374d03e3bcd0a20e236b1"}
Sep 15 10:28:12.212483 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:12.212152    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt" event={"ID":"cf199338-8a03-45c6-be1c-aca427903dcc","Type":"ContainerStarted","Data":"18083ae9a32b65380cc6254c9c82481302880b5eeeeabd1a4f449857631c635a"}
Sep 15 10:28:12.230225 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:12.230172    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt" podStartSLOduration=2.041988349 podStartE2EDuration="2.230158141s" totalImagesPullingTime="188.169792ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:28:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:28:12.2273938 +0000 UTC m=+607.474360769" watchObservedRunningTime="2026-09-15 10:28:12.230158141 +0000 UTC m=+607.477125110"
Sep 15 10:28:15.269645 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:15.269605    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wttp4"
Sep 15 10:28:15.270048 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:15.269668    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wttp4"
Sep 15 10:28:15.281763 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:15.281740    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wttp4"
Sep 15 10:28:16.245302 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:16.245270    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wttp4"
Sep 15 10:28:21.167536 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:21.167506    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt"
Sep 15 10:28:21.167935 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:21.167597    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt"
Sep 15 10:28:21.179965 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:21.179943    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt"
Sep 15 10:28:21.270346 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:21.270318    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt"
Sep 15 10:28:22.906660 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:22.906628    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:28:27.188876 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:27.188841    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:28:41.549523 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:41.549418    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8ptwc"]
Sep 15 10:28:41.552531 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:41.552510    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8ptwc"
Sep 15 10:28:41.554837 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:41.554814    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 15 10:28:41.564266 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:41.564241    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8ptwc"]
Sep 15 10:28:41.580366 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:41.580342    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:28:41.665196 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:41.665172    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ceb4a31-6489-4363-b35e-f5aa3adacd43-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8ptwc\" (UID: \"0ceb4a31-6489-4363-b35e-f5aa3adacd43\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8ptwc"
Sep 15 10:28:41.665313 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:41.665204    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ceb4a31-6489-4363-b35e-f5aa3adacd43-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8ptwc\" (UID: \"0ceb4a31-6489-4363-b35e-f5aa3adacd43\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8ptwc"
Sep 15 10:28:41.665313 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:41.665242    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ceb4a31-6489-4363-b35e-f5aa3adacd43-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8ptwc\" (UID: \"0ceb4a31-6489-4363-b35e-f5aa3adacd43\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8ptwc"
Sep 15 10:28:41.665407 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:41.665367    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slgqx\" (UniqueName: \"kubernetes.io/projected/0ceb4a31-6489-4363-b35e-f5aa3adacd43-kube-api-access-slgqx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8ptwc\" (UID: \"0ceb4a31-6489-4363-b35e-f5aa3adacd43\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8ptwc"
Sep 15 10:28:41.665407 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:41.665396    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ceb4a31-6489-4363-b35e-f5aa3adacd43-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8ptwc\" (UID: \"0ceb4a31-6489-4363-b35e-f5aa3adacd43\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8ptwc"
Sep 15 10:28:41.665499 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:41.665413    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ceb4a31-6489-4363-b35e-f5aa3adacd43-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8ptwc\" (UID: \"0ceb4a31-6489-4363-b35e-f5aa3adacd43\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8ptwc"
Sep 15 10:28:41.766637 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:41.766604    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ceb4a31-6489-4363-b35e-f5aa3adacd43-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8ptwc\" (UID: \"0ceb4a31-6489-4363-b35e-f5aa3adacd43\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8ptwc"
Sep 15 10:28:41.766753 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:41.766703    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slgqx\" (UniqueName: \"kubernetes.io/projected/0ceb4a31-6489-4363-b35e-f5aa3adacd43-kube-api-access-slgqx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8ptwc\" (UID: \"0ceb4a31-6489-4363-b35e-f5aa3adacd43\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8ptwc"
Sep 15 10:28:41.766753 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:41.766733    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ceb4a31-6489-4363-b35e-f5aa3adacd43-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8ptwc\" (UID: \"0ceb4a31-6489-4363-b35e-f5aa3adacd43\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8ptwc"
Sep 15 10:28:41.766840 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:41.766757    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ceb4a31-6489-4363-b35e-f5aa3adacd43-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8ptwc\" (UID: \"0ceb4a31-6489-4363-b35e-f5aa3adacd43\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8ptwc"
Sep 15 10:28:41.766840 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:41.766804    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ceb4a31-6489-4363-b35e-f5aa3adacd43-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8ptwc\" (UID: \"0ceb4a31-6489-4363-b35e-f5aa3adacd43\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8ptwc"
Sep 15 10:28:41.766938 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:41.766836    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ceb4a31-6489-4363-b35e-f5aa3adacd43-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8ptwc\" (UID: \"0ceb4a31-6489-4363-b35e-f5aa3adacd43\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8ptwc"
Sep 15 10:28:41.767071 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:41.767050    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ceb4a31-6489-4363-b35e-f5aa3adacd43-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8ptwc\" (UID: \"0ceb4a31-6489-4363-b35e-f5aa3adacd43\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8ptwc"
Sep 15 10:28:41.767140 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:41.767067    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ceb4a31-6489-4363-b35e-f5aa3adacd43-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8ptwc\" (UID: \"0ceb4a31-6489-4363-b35e-f5aa3adacd43\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8ptwc"
Sep 15 10:28:41.767140 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:41.767101    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ceb4a31-6489-4363-b35e-f5aa3adacd43-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8ptwc\" (UID: \"0ceb4a31-6489-4363-b35e-f5aa3adacd43\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8ptwc"
Sep 15 10:28:41.769016 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:41.768990    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ceb4a31-6489-4363-b35e-f5aa3adacd43-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8ptwc\" (UID: \"0ceb4a31-6489-4363-b35e-f5aa3adacd43\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8ptwc"
Sep 15 10:28:41.769244 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:41.769229    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ceb4a31-6489-4363-b35e-f5aa3adacd43-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8ptwc\" (UID: \"0ceb4a31-6489-4363-b35e-f5aa3adacd43\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8ptwc"
Sep 15 10:28:41.774030 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:41.774008    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slgqx\" (UniqueName: \"kubernetes.io/projected/0ceb4a31-6489-4363-b35e-f5aa3adacd43-kube-api-access-slgqx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8ptwc\" (UID: \"0ceb4a31-6489-4363-b35e-f5aa3adacd43\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8ptwc"
Sep 15 10:28:41.864169 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:41.864098    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8ptwc"
Sep 15 10:28:42.131786 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:42.131758    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8ptwc"]
Sep 15 10:28:42.132390 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:28:42.132360    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ceb4a31_6489_4363_b35e_f5aa3adacd43.slice/crio-941cd7c530890c9f5ed4091fcae7a79f707425c696aa5d531e6e4a65528c7ec0 WatchSource:0}: Error finding container 941cd7c530890c9f5ed4091fcae7a79f707425c696aa5d531e6e4a65528c7ec0: Status 404 returned error can't find the container with id 941cd7c530890c9f5ed4091fcae7a79f707425c696aa5d531e6e4a65528c7ec0
Sep 15 10:28:42.376316 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:42.376293    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8ptwc" event={"ID":"0ceb4a31-6489-4363-b35e-f5aa3adacd43","Type":"ContainerStarted","Data":"941cd7c530890c9f5ed4091fcae7a79f707425c696aa5d531e6e4a65528c7ec0"}
Sep 15 10:28:43.382521 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:43.382484    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8ptwc" event={"ID":"0ceb4a31-6489-4363-b35e-f5aa3adacd43","Type":"ContainerStarted","Data":"c90257334cb137df21babfae828072e929ad923b6b60c8d81195cbcebb9a4aa2"}
Sep 15 10:28:43.398099 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:43.398051    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8ptwc" podStartSLOduration=2.200756107 podStartE2EDuration="2.39803747s" totalImagesPullingTime="197.281363ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:28:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:28:43.39628046 +0000 UTC m=+638.643247454" watchObservedRunningTime="2026-09-15 10:28:43.39803747 +0000 UTC m=+638.645004491"
Sep 15 10:28:49.351279 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:49.351240    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:28:49.452438 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:49.452410    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:28:51.864689 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:51.864660    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8ptwc"
Sep 15 10:28:51.865060 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:51.864710    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8ptwc"
Sep 15 10:28:51.877389 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:51.877363    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8ptwc"
Sep 15 10:28:52.441924 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:52.441898    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8ptwc"
Sep 15 10:28:56.780620 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:28:56.780588    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:29:57.245604 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:29:57.245569    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:29:57.345467 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:29:57.345412    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:29:58.341548 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:29:58.341467    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:29:58.444929 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:29:58.444895    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:14.598483 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:14.598408    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:14.655140 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:14.655114    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:14.720543 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:14.720502    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:14.926421 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:14.926393    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:15.023090 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.023060    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:15.033687 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.033663    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"]
Sep 15 10:30:15.036197 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.036176    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:15.038660 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.038641    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-86a8f5-openshift-default-dockercfg-cpxh8\""
Sep 15 10:30:15.053951 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.053928    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"]
Sep 15 10:30:15.118922 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.118888    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:15.143951 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.143920    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cead169-6453-4e87-8ab8-d1d47e6502d3-workload-socket\") pod \"e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") " pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:15.144098 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.143963    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3cead169-6453-4e87-8ab8-d1d47e6502d3-istio-podinfo\") pod \"e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") " pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:15.144098 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.144004    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cead169-6453-4e87-8ab8-d1d47e6502d3-credential-socket\") pod \"e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") " pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:15.144098 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.144025    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3cead169-6453-4e87-8ab8-d1d47e6502d3-istio-token\") pod \"e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") " pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:15.144098 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.144045    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3cead169-6453-4e87-8ab8-d1d47e6502d3-istiod-ca-cert\") pod \"e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") " pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:15.144098 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.144061    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3cead169-6453-4e87-8ab8-d1d47e6502d3-istio-data\") pod \"e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") " pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:15.144275 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.144156    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3cead169-6453-4e87-8ab8-d1d47e6502d3-istio-envoy\") pod \"e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") " pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:15.144275 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.144189    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkz99\" (UniqueName: \"kubernetes.io/projected/3cead169-6453-4e87-8ab8-d1d47e6502d3-kube-api-access-xkz99\") pod \"e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") " pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:15.144275 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.144226    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3cead169-6453-4e87-8ab8-d1d47e6502d3-workload-certs\") pod \"e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") " pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:15.245488 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.245372    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cead169-6453-4e87-8ab8-d1d47e6502d3-credential-socket\") pod \"e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") " pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:15.245488 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.245415    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3cead169-6453-4e87-8ab8-d1d47e6502d3-istio-token\") pod \"e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") " pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:15.245488 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.245470    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3cead169-6453-4e87-8ab8-d1d47e6502d3-istiod-ca-cert\") pod \"e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") " pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:15.245772 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.245498    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3cead169-6453-4e87-8ab8-d1d47e6502d3-istio-data\") pod \"e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") " pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:15.245772 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.245563    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3cead169-6453-4e87-8ab8-d1d47e6502d3-istio-envoy\") pod \"e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") " pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:15.245772 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.245594    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkz99\" (UniqueName: \"kubernetes.io/projected/3cead169-6453-4e87-8ab8-d1d47e6502d3-kube-api-access-xkz99\") pod \"e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") " pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:15.245772 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.245642    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3cead169-6453-4e87-8ab8-d1d47e6502d3-workload-certs\") pod \"e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") " pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:15.245772 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.245685    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cead169-6453-4e87-8ab8-d1d47e6502d3-workload-socket\") pod \"e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") " pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:15.245772 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.245739    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3cead169-6453-4e87-8ab8-d1d47e6502d3-istio-podinfo\") pod \"e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") " pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:15.246095 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.246005    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cead169-6453-4e87-8ab8-d1d47e6502d3-credential-socket\") pod \"e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") " pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:15.246095 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.246085    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3cead169-6453-4e87-8ab8-d1d47e6502d3-istio-data\") pod \"e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") " pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:15.246207 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.246172    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cead169-6453-4e87-8ab8-d1d47e6502d3-workload-socket\") pod \"e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") " pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:15.246263 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.246205    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3cead169-6453-4e87-8ab8-d1d47e6502d3-workload-certs\") pod \"e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") " pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:15.246350 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.246330    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3cead169-6453-4e87-8ab8-d1d47e6502d3-istiod-ca-cert\") pod \"e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") " pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:15.247818 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.247789    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3cead169-6453-4e87-8ab8-d1d47e6502d3-istio-envoy\") pod \"e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") " pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:15.248106 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.248090    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3cead169-6453-4e87-8ab8-d1d47e6502d3-istio-podinfo\") pod \"e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") " pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:15.252582 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.252560    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkz99\" (UniqueName: \"kubernetes.io/projected/3cead169-6453-4e87-8ab8-d1d47e6502d3-kube-api-access-xkz99\") pod \"e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") " pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:15.253056 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.253036    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3cead169-6453-4e87-8ab8-d1d47e6502d3-istio-token\") pod \"e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") " pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:15.346670 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.346645    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:15.381532 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.381505    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:15.642517 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.642482    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"]
Sep 15 10:30:15.643509 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:30:15.643480    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cead169_6453_4e87_8ab8_d1d47e6502d3.slice/crio-2ca1266b161e93a474fb156329719c6cf6f7c365937886ad97bc9104e934970d WatchSource:0}: Error finding container 2ca1266b161e93a474fb156329719c6cf6f7c365937886ad97bc9104e934970d: Status 404 returned error can't find the container with id 2ca1266b161e93a474fb156329719c6cf6f7c365937886ad97bc9104e934970d
Sep 15 10:30:15.645707 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.645672    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:30:15.645781 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.645737    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:30:15.645781 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.645768    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:30:15.868610 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.868572    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx" event={"ID":"3cead169-6453-4e87-8ab8-d1d47e6502d3","Type":"ContainerStarted","Data":"9a2e012a54a8ee76ab5206e79a33d5033ad77b03ec384037906a11e09f263cf4"}
Sep 15 10:30:15.868772 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.868640    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx" event={"ID":"3cead169-6453-4e87-8ab8-d1d47e6502d3","Type":"ContainerStarted","Data":"2ca1266b161e93a474fb156329719c6cf6f7c365937886ad97bc9104e934970d"}
Sep 15 10:30:15.888154 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:15.888098    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx" podStartSLOduration=0.888076948 podStartE2EDuration="888.076948ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:30:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:30:15.886059643 +0000 UTC m=+731.133026637" watchObservedRunningTime="2026-09-15 10:30:15.888076948 +0000 UTC m=+731.135043920"
Sep 15 10:30:16.347012 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:16.346977    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:16.351583 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:16.351559    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:16.841816 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:16.841780    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:16.873722 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:16.873691    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:16.874677 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:16.874657    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:16.907876 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:16.907843    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:18.554224 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:18.554191    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:19.165147 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:19.165113    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc"]
Sep 15 10:30:19.168071 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:19.168048    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc"
Sep 15 10:30:19.170222 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:19.170194    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-8c1047-serving-cert\""
Sep 15 10:30:19.178508 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:19.178484    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc"]
Sep 15 10:30:19.280460 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:19.280409    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48677bfc-860f-49f9-87b3-a83cce52f760-maas-api-tls\") pod \"maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc\" (UID: \"48677bfc-860f-49f9-87b3-a83cce52f760\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc"
Sep 15 10:30:19.280625 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:19.280469    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5m5j\" (UniqueName: \"kubernetes.io/projected/48677bfc-860f-49f9-87b3-a83cce52f760-kube-api-access-k5m5j\") pod \"maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc\" (UID: \"48677bfc-860f-49f9-87b3-a83cce52f760\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc"
Sep 15 10:30:19.280625 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:19.280587    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48677bfc-860f-49f9-87b3-a83cce52f760-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc\" (UID: \"48677bfc-860f-49f9-87b3-a83cce52f760\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc"
Sep 15 10:30:19.280715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:19.280637    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48677bfc-860f-49f9-87b3-a83cce52f760-metrics-certs\") pod \"maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc\" (UID: \"48677bfc-860f-49f9-87b3-a83cce52f760\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc"
Sep 15 10:30:19.381574 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:19.381535    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48677bfc-860f-49f9-87b3-a83cce52f760-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc\" (UID: \"48677bfc-860f-49f9-87b3-a83cce52f760\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc"
Sep 15 10:30:19.381760 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:19.381715    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48677bfc-860f-49f9-87b3-a83cce52f760-metrics-certs\") pod \"maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc\" (UID: \"48677bfc-860f-49f9-87b3-a83cce52f760\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc"
Sep 15 10:30:19.381872 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:19.381844    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48677bfc-860f-49f9-87b3-a83cce52f760-maas-api-tls\") pod \"maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc\" (UID: \"48677bfc-860f-49f9-87b3-a83cce52f760\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc"
Sep 15 10:30:19.381943 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:19.381877    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5m5j\" (UniqueName: \"kubernetes.io/projected/48677bfc-860f-49f9-87b3-a83cce52f760-kube-api-access-k5m5j\") pod \"maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc\" (UID: \"48677bfc-860f-49f9-87b3-a83cce52f760\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc"
Sep 15 10:30:19.382008 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:30:19.381991    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-8c1047-serving-cert: secret "maas-api-e2e-worker-w0-8c1047-serving-cert" not found
Sep 15 10:30:19.382077 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:30:19.382059    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48677bfc-860f-49f9-87b3-a83cce52f760-maas-api-tls podName:48677bfc-860f-49f9-87b3-a83cce52f760 nodeName:}" failed. No retries permitted until 2026-09-15 10:30:19.882037878 +0000 UTC m=+735.129004829 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/48677bfc-860f-49f9-87b3-a83cce52f760-maas-api-tls") pod "maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc" (UID: "48677bfc-860f-49f9-87b3-a83cce52f760") : secret "maas-api-e2e-worker-w0-8c1047-serving-cert" not found
Sep 15 10:30:19.382330 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:19.382312    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48677bfc-860f-49f9-87b3-a83cce52f760-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc\" (UID: \"48677bfc-860f-49f9-87b3-a83cce52f760\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc"
Sep 15 10:30:19.384523 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:19.384502    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48677bfc-860f-49f9-87b3-a83cce52f760-metrics-certs\") pod \"maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc\" (UID: \"48677bfc-860f-49f9-87b3-a83cce52f760\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc"
Sep 15 10:30:19.395113 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:19.395088    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5m5j\" (UniqueName: \"kubernetes.io/projected/48677bfc-860f-49f9-87b3-a83cce52f760-kube-api-access-k5m5j\") pod \"maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc\" (UID: \"48677bfc-860f-49f9-87b3-a83cce52f760\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc"
Sep 15 10:30:19.886295 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:19.886267    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48677bfc-860f-49f9-87b3-a83cce52f760-maas-api-tls\") pod \"maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc\" (UID: \"48677bfc-860f-49f9-87b3-a83cce52f760\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc"
Sep 15 10:30:19.888808 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:19.888784    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48677bfc-860f-49f9-87b3-a83cce52f760-maas-api-tls\") pod \"maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc\" (UID: \"48677bfc-860f-49f9-87b3-a83cce52f760\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc"
Sep 15 10:30:20.080508 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:20.080479    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc"
Sep 15 10:30:20.166528 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:20.166435    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d"]
Sep 15 10:30:20.169553 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:20.169532    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d"
Sep 15 10:30:20.172069 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:20.172045    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-ff0655-serving-cert\""
Sep 15 10:30:20.184590 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:20.184569    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d"]
Sep 15 10:30:20.290142 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:20.290105    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0eabace0-ccad-44ee-a69f-ab95ed30a1c4-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d\" (UID: \"0eabace0-ccad-44ee-a69f-ab95ed30a1c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d"
Sep 15 10:30:20.290142 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:20.290149    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfxdc\" (UniqueName: \"kubernetes.io/projected/0eabace0-ccad-44ee-a69f-ab95ed30a1c4-kube-api-access-tfxdc\") pod \"maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d\" (UID: \"0eabace0-ccad-44ee-a69f-ab95ed30a1c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d"
Sep 15 10:30:20.290365 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:20.290215    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0eabace0-ccad-44ee-a69f-ab95ed30a1c4-maas-api-tls\") pod \"maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d\" (UID: \"0eabace0-ccad-44ee-a69f-ab95ed30a1c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d"
Sep 15 10:30:20.290365 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:20.290304    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0eabace0-ccad-44ee-a69f-ab95ed30a1c4-metrics-certs\") pod \"maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d\" (UID: \"0eabace0-ccad-44ee-a69f-ab95ed30a1c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d"
Sep 15 10:30:20.350254 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:20.350227    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc"]
Sep 15 10:30:20.351319 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:30:20.351295    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48677bfc_860f_49f9_87b3_a83cce52f760.slice/crio-b1b91fef02d7e6d06791932acd2f8cdaac0a77432ba2ce2fca09f2020933d4c3 WatchSource:0}: Error finding container b1b91fef02d7e6d06791932acd2f8cdaac0a77432ba2ce2fca09f2020933d4c3: Status 404 returned error can't find the container with id b1b91fef02d7e6d06791932acd2f8cdaac0a77432ba2ce2fca09f2020933d4c3
Sep 15 10:30:20.391707 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:20.391678    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0eabace0-ccad-44ee-a69f-ab95ed30a1c4-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d\" (UID: \"0eabace0-ccad-44ee-a69f-ab95ed30a1c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d"
Sep 15 10:30:20.391903 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:20.391854    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfxdc\" (UniqueName: \"kubernetes.io/projected/0eabace0-ccad-44ee-a69f-ab95ed30a1c4-kube-api-access-tfxdc\") pod \"maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d\" (UID: \"0eabace0-ccad-44ee-a69f-ab95ed30a1c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d"
Sep 15 10:30:20.392013 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:20.391947    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0eabace0-ccad-44ee-a69f-ab95ed30a1c4-maas-api-tls\") pod \"maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d\" (UID: \"0eabace0-ccad-44ee-a69f-ab95ed30a1c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d"
Sep 15 10:30:20.392228 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:30:20.392069    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-ff0655-serving-cert: secret "maas-api-e2e-worker-w4-ff0655-serving-cert" not found
Sep 15 10:30:20.392228 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:20.392087    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0eabace0-ccad-44ee-a69f-ab95ed30a1c4-metrics-certs\") pod \"maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d\" (UID: \"0eabace0-ccad-44ee-a69f-ab95ed30a1c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d"
Sep 15 10:30:20.392366 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:30:20.392350    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0eabace0-ccad-44ee-a69f-ab95ed30a1c4-maas-api-tls podName:0eabace0-ccad-44ee-a69f-ab95ed30a1c4 nodeName:}" failed. No retries permitted until 2026-09-15 10:30:20.892326621 +0000 UTC m=+736.139293573 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0eabace0-ccad-44ee-a69f-ab95ed30a1c4-maas-api-tls") pod "maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d" (UID: "0eabace0-ccad-44ee-a69f-ab95ed30a1c4") : secret "maas-api-e2e-worker-w4-ff0655-serving-cert" not found
Sep 15 10:30:20.392435 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:20.392402    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0eabace0-ccad-44ee-a69f-ab95ed30a1c4-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d\" (UID: \"0eabace0-ccad-44ee-a69f-ab95ed30a1c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d"
Sep 15 10:30:20.394604 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:20.394584    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0eabace0-ccad-44ee-a69f-ab95ed30a1c4-metrics-certs\") pod \"maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d\" (UID: \"0eabace0-ccad-44ee-a69f-ab95ed30a1c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d"
Sep 15 10:30:20.401778 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:20.401755    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfxdc\" (UniqueName: \"kubernetes.io/projected/0eabace0-ccad-44ee-a69f-ab95ed30a1c4-kube-api-access-tfxdc\") pod \"maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d\" (UID: \"0eabace0-ccad-44ee-a69f-ab95ed30a1c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d"
Sep 15 10:30:20.481837 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:20.481766    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:20.552424 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:20.552392    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:20.896127 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:20.896093    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0eabace0-ccad-44ee-a69f-ab95ed30a1c4-maas-api-tls\") pod \"maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d\" (UID: \"0eabace0-ccad-44ee-a69f-ab95ed30a1c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d"
Sep 15 10:30:20.898986 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:20.898963    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0eabace0-ccad-44ee-a69f-ab95ed30a1c4-maas-api-tls\") pod \"maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d\" (UID: \"0eabace0-ccad-44ee-a69f-ab95ed30a1c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d"
Sep 15 10:30:20.899355 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:20.899332    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc" event={"ID":"48677bfc-860f-49f9-87b3-a83cce52f760","Type":"ContainerStarted","Data":"c5b0f378bea1395c41e143e671aaec7756dd41eb09619661f202e262f9a076f9"}
Sep 15 10:30:20.899438 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:20.899379    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc" event={"ID":"48677bfc-860f-49f9-87b3-a83cce52f760","Type":"ContainerStarted","Data":"b1b91fef02d7e6d06791932acd2f8cdaac0a77432ba2ce2fca09f2020933d4c3"}
Sep 15 10:30:20.899535 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:20.899473    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc"
Sep 15 10:30:20.919400 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:20.919356    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc" podStartSLOduration=1.542027526 podStartE2EDuration="1.91933968s" totalImagesPullingTime="377.312154ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:30:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:30:20.918728444 +0000 UTC m=+736.165695412" watchObservedRunningTime="2026-09-15 10:30:20.91933968 +0000 UTC m=+736.166306651"
Sep 15 10:30:21.082131 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:21.082051    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d"
Sep 15 10:30:21.253949 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:21.253917    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj"]
Sep 15 10:30:21.257569 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:21.257541    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj"
Sep 15 10:30:21.262144 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:21.262117    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-86a8f5-serving-cert\""
Sep 15 10:30:21.270614 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:21.270594    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj"]
Sep 15 10:30:21.302390 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:21.302357    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60fe0698-4c41-4f62-9389-f8b03cd9086a-maas-api-tls\") pod \"maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj\" (UID: \"60fe0698-4c41-4f62-9389-f8b03cd9086a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj"
Sep 15 10:30:21.302574 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:21.302438    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqh4l\" (UniqueName: \"kubernetes.io/projected/60fe0698-4c41-4f62-9389-f8b03cd9086a-kube-api-access-wqh4l\") pod \"maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj\" (UID: \"60fe0698-4c41-4f62-9389-f8b03cd9086a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj"
Sep 15 10:30:21.302574 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:21.302514    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60fe0698-4c41-4f62-9389-f8b03cd9086a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj\" (UID: \"60fe0698-4c41-4f62-9389-f8b03cd9086a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj"
Sep 15 10:30:21.302574 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:21.302538    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60fe0698-4c41-4f62-9389-f8b03cd9086a-metrics-certs\") pod \"maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj\" (UID: \"60fe0698-4c41-4f62-9389-f8b03cd9086a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj"
Sep 15 10:30:21.360117 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:21.360073    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d"]
Sep 15 10:30:21.362229 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:30:21.362203    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0eabace0_ccad_44ee_a69f_ab95ed30a1c4.slice/crio-e51c1cb163f13777d49c94e6198a90c5ad5cd987f3eedd49189d740097017c84 WatchSource:0}: Error finding container e51c1cb163f13777d49c94e6198a90c5ad5cd987f3eedd49189d740097017c84: Status 404 returned error can't find the container with id e51c1cb163f13777d49c94e6198a90c5ad5cd987f3eedd49189d740097017c84
Sep 15 10:30:21.393030 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:21.393006    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:21.403857 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:21.403835    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqh4l\" (UniqueName: \"kubernetes.io/projected/60fe0698-4c41-4f62-9389-f8b03cd9086a-kube-api-access-wqh4l\") pod \"maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj\" (UID: \"60fe0698-4c41-4f62-9389-f8b03cd9086a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj"
Sep 15 10:30:21.403979 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:21.403889    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60fe0698-4c41-4f62-9389-f8b03cd9086a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj\" (UID: \"60fe0698-4c41-4f62-9389-f8b03cd9086a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj"
Sep 15 10:30:21.403979 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:21.403909    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60fe0698-4c41-4f62-9389-f8b03cd9086a-metrics-certs\") pod \"maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj\" (UID: \"60fe0698-4c41-4f62-9389-f8b03cd9086a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj"
Sep 15 10:30:21.403979 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:21.403972    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60fe0698-4c41-4f62-9389-f8b03cd9086a-maas-api-tls\") pod \"maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj\" (UID: \"60fe0698-4c41-4f62-9389-f8b03cd9086a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj"
Sep 15 10:30:21.404622 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:21.404600    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60fe0698-4c41-4f62-9389-f8b03cd9086a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj\" (UID: \"60fe0698-4c41-4f62-9389-f8b03cd9086a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj"
Sep 15 10:30:21.406714 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:21.406657    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60fe0698-4c41-4f62-9389-f8b03cd9086a-maas-api-tls\") pod \"maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj\" (UID: \"60fe0698-4c41-4f62-9389-f8b03cd9086a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj"
Sep 15 10:30:21.406795 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:21.406739    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60fe0698-4c41-4f62-9389-f8b03cd9086a-metrics-certs\") pod \"maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj\" (UID: \"60fe0698-4c41-4f62-9389-f8b03cd9086a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj"
Sep 15 10:30:21.413713 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:21.413673    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqh4l\" (UniqueName: \"kubernetes.io/projected/60fe0698-4c41-4f62-9389-f8b03cd9086a-kube-api-access-wqh4l\") pod \"maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj\" (UID: \"60fe0698-4c41-4f62-9389-f8b03cd9086a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj"
Sep 15 10:30:21.491599 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:21.491565    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:21.570243 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:21.570210    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj"
Sep 15 10:30:21.728710 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:21.728679    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:21.832061 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:21.832022    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:21.846683 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:21.846655    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj"]
Sep 15 10:30:21.847487 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:30:21.847435    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60fe0698_4c41_4f62_9389_f8b03cd9086a.slice/crio-4efd2d1c84bb1fb37349d6a1cb176ff747bc0ed130b87c6762f8843b3b548a1d WatchSource:0}: Error finding container 4efd2d1c84bb1fb37349d6a1cb176ff747bc0ed130b87c6762f8843b3b548a1d: Status 404 returned error can't find the container with id 4efd2d1c84bb1fb37349d6a1cb176ff747bc0ed130b87c6762f8843b3b548a1d
Sep 15 10:30:21.905874 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:21.905841    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj" event={"ID":"60fe0698-4c41-4f62-9389-f8b03cd9086a","Type":"ContainerStarted","Data":"4efd2d1c84bb1fb37349d6a1cb176ff747bc0ed130b87c6762f8843b3b548a1d"}
Sep 15 10:30:21.906976 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:21.906949    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d" event={"ID":"0eabace0-ccad-44ee-a69f-ab95ed30a1c4","Type":"ContainerStarted","Data":"e51c1cb163f13777d49c94e6198a90c5ad5cd987f3eedd49189d740097017c84"}
Sep 15 10:30:22.103581 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:22.103545    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:22.875583 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:22.875552    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:22.915461 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:22.915413    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj" event={"ID":"60fe0698-4c41-4f62-9389-f8b03cd9086a","Type":"ContainerStarted","Data":"c5631c76951d92cc875a3c52acca970c3d7f8fbea5699338b7d9278760ddb4b8"}
Sep 15 10:30:22.915811 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:22.915528    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj"
Sep 15 10:30:22.938697 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:22.938654    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj" podStartSLOduration=1.601497455 podStartE2EDuration="1.938639581s" totalImagesPullingTime="337.142126ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:30:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:30:22.935501004 +0000 UTC m=+738.182467976" watchObservedRunningTime="2026-09-15 10:30:22.938639581 +0000 UTC m=+738.185606551"
Sep 15 10:30:22.948168 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:22.948145    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:23.024399 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:23.024372    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:23.124606 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:23.124572    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:23.406014 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:23.405979    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:23.587762 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:23.587729    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:23.791787 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:23.791696    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:23.923109 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:23.923072    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d" event={"ID":"0eabace0-ccad-44ee-a69f-ab95ed30a1c4","Type":"ContainerStarted","Data":"1137483d61c11df6a10fb6d17b058d9c982efb10b8e438491f49ebaba00ac204"}
Sep 15 10:30:23.923535 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:23.923210    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d"
Sep 15 10:30:23.942507 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:23.942433    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d" podStartSLOduration=2.383761568 podStartE2EDuration="3.942415741s" totalImagesPullingTime="1.558654173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:30:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:30:23.939958609 +0000 UTC m=+739.186925578" watchObservedRunningTime="2026-09-15 10:30:23.942415741 +0000 UTC m=+739.189382754"
Sep 15 10:30:25.643541 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:25.643504    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:26.352567 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.352536    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"]
Sep 15 10:30:26.355149 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.355127    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:26.357782 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.357761    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-8110bc-openshift-default-dockercfg-q8x8p\""
Sep 15 10:30:26.369866 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.369832    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"]
Sep 15 10:30:26.455592 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.455554    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-credential-socket\") pod \"e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") " pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:26.455789 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.455605    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-workload-socket\") pod \"e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") " pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:26.455789 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.455633    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d752d\" (UniqueName: \"kubernetes.io/projected/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-kube-api-access-d752d\") pod \"e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") " pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:26.455789 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.455661    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-istiod-ca-cert\") pod \"e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") " pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:26.455789 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.455698    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-istio-token\") pod \"e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") " pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:26.455789 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.455729    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-istio-envoy\") pod \"e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") " pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:26.455789 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.455755    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-workload-certs\") pod \"e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") " pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:26.455789 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.455775    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-istio-data\") pod \"e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") " pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:26.456242 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.455800    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-istio-podinfo\") pod \"e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") " pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:26.557164 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.557126    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-istiod-ca-cert\") pod \"e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") " pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:26.557164 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.557167    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-istio-token\") pod \"e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") " pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:26.557412 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.557211    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-istio-envoy\") pod \"e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") " pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:26.557412 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.557241    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-workload-certs\") pod \"e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") " pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:26.557412 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.557272    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-istio-data\") pod \"e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") " pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:26.557412 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.557299    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-istio-podinfo\") pod \"e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") " pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:26.557652 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.557405    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-credential-socket\") pod \"e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") " pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:26.557767 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.557742    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-istio-data\") pod \"e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") " pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:26.557826 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.557774    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-workload-certs\") pod \"e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") " pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:26.557826 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.557810    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-credential-socket\") pod \"e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") " pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:26.557966 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.557948    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-workload-socket\") pod \"e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") " pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:26.558011 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.557982    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d752d\" (UniqueName: \"kubernetes.io/projected/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-kube-api-access-d752d\") pod \"e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") " pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:26.558011 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.557985    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-istiod-ca-cert\") pod \"e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") " pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:26.558308 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.558266    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-workload-socket\") pod \"e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") " pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:26.560168 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.560151    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-istio-envoy\") pod \"e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") " pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:26.560364 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.560345    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-istio-podinfo\") pod \"e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") " pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:26.566151 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.566128    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d752d\" (UniqueName: \"kubernetes.io/projected/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-kube-api-access-d752d\") pod \"e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") " pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:26.566263 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.566252    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-istio-token\") pod \"e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") " pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:26.601602 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.601574    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:26.668455 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.668419    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:26.704414 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.704381    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:26.811282 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.811250    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:26.912386 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.912350    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc"
Sep 15 10:30:26.956642 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.956555    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"]
Sep 15 10:30:26.961127 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:30:26.961081    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcffc93c4_c0e4_480c_b91f_5fd9cfab8234.slice/crio-20e22210ff09a622530578ae5a687afe901937add37a3371e1069fbdcf9b933d WatchSource:0}: Error finding container 20e22210ff09a622530578ae5a687afe901937add37a3371e1069fbdcf9b933d: Status 404 returned error can't find the container with id 20e22210ff09a622530578ae5a687afe901937add37a3371e1069fbdcf9b933d
Sep 15 10:30:26.964672 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.964632    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:30:26.964778 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.964725    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:30:26.964778 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:26.964770    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:30:27.953371 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:27.953327    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp" event={"ID":"cffc93c4-c0e4-480c-b91f-5fd9cfab8234","Type":"ContainerStarted","Data":"1eb8eec838a3efcfaae5e42127981f9c5090db9c998a413841c514fa6c96c2b5"}
Sep 15 10:30:27.954001 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:27.953398    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp" event={"ID":"cffc93c4-c0e4-480c-b91f-5fd9cfab8234","Type":"ContainerStarted","Data":"20e22210ff09a622530578ae5a687afe901937add37a3371e1069fbdcf9b933d"}
Sep 15 10:30:27.976360 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:27.976303    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp" podStartSLOduration=1.976284047 podStartE2EDuration="1.976284047s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:30:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:30:27.975155336 +0000 UTC m=+743.222122305" watchObservedRunningTime="2026-09-15 10:30:27.976284047 +0000 UTC m=+743.223251017"
Sep 15 10:30:28.231849 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:28.231760    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:28.337342 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:28.337303    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:28.669519 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:28.669472    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:28.675459 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:28.675415    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:28.929214 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:28.929135    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj"
Sep 15 10:30:28.963411 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:28.963382    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:28.965720 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:28.965679    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:29.185721 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:29.185637    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:29.935155 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:29.935127    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d"
Sep 15 10:30:31.462845 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:31.462780    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:31.552963 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:31.552929    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:32.141394 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:32.141333    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:32.572057 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:32.571968    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:34.027104 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:34.027057    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:34.128552 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:34.128521    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:34.454522 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:34.454410    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp"]
Sep 15 10:30:34.458355 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:34.458319    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp"
Sep 15 10:30:34.460898 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:34.460869    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-8110bc-serving-cert\""
Sep 15 10:30:34.468874 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:34.468839    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp"]
Sep 15 10:30:34.481772 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:34.481742    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:34.542957 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:34.542915    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/985b3d6b-3519-4a9c-b3c4-515b31bf6b75-openshift-service-ca\") pod \"maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp\" (UID: \"985b3d6b-3519-4a9c-b3c4-515b31bf6b75\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp"
Sep 15 10:30:34.543123 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:34.543083    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/985b3d6b-3519-4a9c-b3c4-515b31bf6b75-metrics-certs\") pod \"maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp\" (UID: \"985b3d6b-3519-4a9c-b3c4-515b31bf6b75\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp"
Sep 15 10:30:34.543193 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:34.543146    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/985b3d6b-3519-4a9c-b3c4-515b31bf6b75-maas-api-tls\") pod \"maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp\" (UID: \"985b3d6b-3519-4a9c-b3c4-515b31bf6b75\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp"
Sep 15 10:30:34.543229 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:34.543205    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86mkp\" (UniqueName: \"kubernetes.io/projected/985b3d6b-3519-4a9c-b3c4-515b31bf6b75-kube-api-access-86mkp\") pod \"maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp\" (UID: \"985b3d6b-3519-4a9c-b3c4-515b31bf6b75\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp"
Sep 15 10:30:34.574112 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:34.574077    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:34.644727 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:34.644686    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/985b3d6b-3519-4a9c-b3c4-515b31bf6b75-openshift-service-ca\") pod \"maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp\" (UID: \"985b3d6b-3519-4a9c-b3c4-515b31bf6b75\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp"
Sep 15 10:30:34.644921 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:34.644798    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/985b3d6b-3519-4a9c-b3c4-515b31bf6b75-metrics-certs\") pod \"maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp\" (UID: \"985b3d6b-3519-4a9c-b3c4-515b31bf6b75\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp"
Sep 15 10:30:34.644921 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:34.644844    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/985b3d6b-3519-4a9c-b3c4-515b31bf6b75-maas-api-tls\") pod \"maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp\" (UID: \"985b3d6b-3519-4a9c-b3c4-515b31bf6b75\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp"
Sep 15 10:30:34.644921 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:34.644890    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86mkp\" (UniqueName: \"kubernetes.io/projected/985b3d6b-3519-4a9c-b3c4-515b31bf6b75-kube-api-access-86mkp\") pod \"maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp\" (UID: \"985b3d6b-3519-4a9c-b3c4-515b31bf6b75\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp"
Sep 15 10:30:34.646033 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:34.646003    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/985b3d6b-3519-4a9c-b3c4-515b31bf6b75-openshift-service-ca\") pod \"maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp\" (UID: \"985b3d6b-3519-4a9c-b3c4-515b31bf6b75\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp"
Sep 15 10:30:34.647664 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:34.647624    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/985b3d6b-3519-4a9c-b3c4-515b31bf6b75-maas-api-tls\") pod \"maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp\" (UID: \"985b3d6b-3519-4a9c-b3c4-515b31bf6b75\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp"
Sep 15 10:30:34.648045 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:34.648021    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/985b3d6b-3519-4a9c-b3c4-515b31bf6b75-metrics-certs\") pod \"maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp\" (UID: \"985b3d6b-3519-4a9c-b3c4-515b31bf6b75\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp"
Sep 15 10:30:34.653630 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:34.653602    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86mkp\" (UniqueName: \"kubernetes.io/projected/985b3d6b-3519-4a9c-b3c4-515b31bf6b75-kube-api-access-86mkp\") pod \"maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp\" (UID: \"985b3d6b-3519-4a9c-b3c4-515b31bf6b75\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp"
Sep 15 10:30:34.675259 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:34.674877    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:34.773947 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:34.773859    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp"
Sep 15 10:30:35.051885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:35.051856    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp"]
Sep 15 10:30:35.054517 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:30:35.054485    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod985b3d6b_3519_4a9c_b3c4_515b31bf6b75.slice/crio-b2399138a1a1fe63bdf802d137aab6938f0c3e20ce3861bb5cf850a39549db70 WatchSource:0}: Error finding container b2399138a1a1fe63bdf802d137aab6938f0c3e20ce3861bb5cf850a39549db70: Status 404 returned error can't find the container with id b2399138a1a1fe63bdf802d137aab6938f0c3e20ce3861bb5cf850a39549db70
Sep 15 10:30:35.077709 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:35.077672    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:35.175691 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:35.175649    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:35.521407 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:35.521376    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:36.017707 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:36.017674    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp" event={"ID":"985b3d6b-3519-4a9c-b3c4-515b31bf6b75","Type":"ContainerStarted","Data":"b2399138a1a1fe63bdf802d137aab6938f0c3e20ce3861bb5cf850a39549db70"}
Sep 15 10:30:36.490484 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:36.490383    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:36.760817 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:36.760785    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:37.025954 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:37.025864    2583 generic.go:356] "Generic (PLEG): container finished" podID="3cead169-6453-4e87-8ab8-d1d47e6502d3" containerID="9a2e012a54a8ee76ab5206e79a33d5033ad77b03ec384037906a11e09f263cf4" exitCode=0
Sep 15 10:30:37.025954 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:37.025934    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx" event={"ID":"3cead169-6453-4e87-8ab8-d1d47e6502d3","Type":"ContainerDied","Data":"9a2e012a54a8ee76ab5206e79a33d5033ad77b03ec384037906a11e09f263cf4"}
Sep 15 10:30:37.026469 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:37.026431    2583 scope.go:117] "RemoveContainer" containerID="9a2e012a54a8ee76ab5206e79a33d5033ad77b03ec384037906a11e09f263cf4"
Sep 15 10:30:37.027749 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:37.027721    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:30:37.027838 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:37.027792    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp" event={"ID":"985b3d6b-3519-4a9c-b3c4-515b31bf6b75","Type":"ContainerStarted","Data":"053ecd05bbdfb90b7d4b67ef54d6347781fa5c026e30697e7fa3caa0ee0e4012"}
Sep 15 10:30:37.027886 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:37.027839    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp"
Sep 15 10:30:37.027886 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:37.027796    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:30:37.027964 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:37.027890    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:30:37.066544 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:37.066500    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp" podStartSLOduration=2.191844932 podStartE2EDuration="3.066484762s" totalImagesPullingTime="874.63983ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:30:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:30:37.064047502 +0000 UTC m=+752.311014473" watchObservedRunningTime="2026-09-15 10:30:37.066484762 +0000 UTC m=+752.313451731"
Sep 15 10:30:37.181982 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:37.181955    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:37.281665 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:37.281578    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:37.677555 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:37.677521    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:37.778071 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:37.778030    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:38.037188 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:38.037104    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx" event={"ID":"3cead169-6453-4e87-8ab8-d1d47e6502d3","Type":"ContainerStarted","Data":"7fe6ab28c7b017a26b128d7675f6f12de8b05e45c0a6522a1a7a459a2fa1254e"}
Sep 15 10:30:38.117764 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:38.117728    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:38.219515 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:38.219482    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:38.347553 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:38.347437    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:38.347737 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:38.347590    2583 patch_prober.go:28] interesting pod/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 15 10:30:38.347737 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:38.347650    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx" podUID="3cead169-6453-4e87-8ab8-d1d47e6502d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 15 10:30:38.403342 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:38.403301    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:38.512586 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:38.512552    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:39.347555 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:39.347513    2583 patch_prober.go:28] interesting pod/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 15 10:30:39.348042 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:39.347592    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx" podUID="3cead169-6453-4e87-8ab8-d1d47e6502d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 15 10:30:39.628600 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:39.628520    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg"]
Sep 15 10:30:39.632135 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:39.632120    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg"
Sep 15 10:30:39.634541 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:39.634515    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-234840-serving-cert\""
Sep 15 10:30:39.642301 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:39.642276    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg"]
Sep 15 10:30:39.683466 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:39.683422    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:39.781133 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:39.781105    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:39.800709 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:39.800676    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2db6e97f-345b-4fa3-9b48-127cd97bb0d4-maas-api-tls\") pod \"maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg\" (UID: \"2db6e97f-345b-4fa3-9b48-127cd97bb0d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg"
Sep 15 10:30:39.800709 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:39.800710    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2db6e97f-345b-4fa3-9b48-127cd97bb0d4-openshift-service-ca\") pod \"maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg\" (UID: \"2db6e97f-345b-4fa3-9b48-127cd97bb0d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg"
Sep 15 10:30:39.800886 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:39.800795    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2db6e97f-345b-4fa3-9b48-127cd97bb0d4-metrics-certs\") pod \"maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg\" (UID: \"2db6e97f-345b-4fa3-9b48-127cd97bb0d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg"
Sep 15 10:30:39.800886 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:39.800822    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sz5d6\" (UniqueName: \"kubernetes.io/projected/2db6e97f-345b-4fa3-9b48-127cd97bb0d4-kube-api-access-sz5d6\") pod \"maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg\" (UID: \"2db6e97f-345b-4fa3-9b48-127cd97bb0d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg"
Sep 15 10:30:39.901885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:39.901847    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2db6e97f-345b-4fa3-9b48-127cd97bb0d4-metrics-certs\") pod \"maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg\" (UID: \"2db6e97f-345b-4fa3-9b48-127cd97bb0d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg"
Sep 15 10:30:39.901885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:39.901890    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz5d6\" (UniqueName: \"kubernetes.io/projected/2db6e97f-345b-4fa3-9b48-127cd97bb0d4-kube-api-access-sz5d6\") pod \"maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg\" (UID: \"2db6e97f-345b-4fa3-9b48-127cd97bb0d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg"
Sep 15 10:30:39.902154 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:39.901991    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2db6e97f-345b-4fa3-9b48-127cd97bb0d4-maas-api-tls\") pod \"maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg\" (UID: \"2db6e97f-345b-4fa3-9b48-127cd97bb0d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg"
Sep 15 10:30:39.902154 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:39.902013    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2db6e97f-345b-4fa3-9b48-127cd97bb0d4-openshift-service-ca\") pod \"maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg\" (UID: \"2db6e97f-345b-4fa3-9b48-127cd97bb0d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg"
Sep 15 10:30:39.902154 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:30:39.902123    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-234840-serving-cert: secret "maas-api-e2e-inf-b-234840-serving-cert" not found
Sep 15 10:30:39.902318 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:30:39.902188    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2db6e97f-345b-4fa3-9b48-127cd97bb0d4-maas-api-tls podName:2db6e97f-345b-4fa3-9b48-127cd97bb0d4 nodeName:}" failed. No retries permitted until 2026-09-15 10:30:40.402170177 +0000 UTC m=+755.649137125 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2db6e97f-345b-4fa3-9b48-127cd97bb0d4-maas-api-tls") pod "maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg" (UID: "2db6e97f-345b-4fa3-9b48-127cd97bb0d4") : secret "maas-api-e2e-inf-b-234840-serving-cert" not found
Sep 15 10:30:39.902689 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:39.902667    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2db6e97f-345b-4fa3-9b48-127cd97bb0d4-openshift-service-ca\") pod \"maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg\" (UID: \"2db6e97f-345b-4fa3-9b48-127cd97bb0d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg"
Sep 15 10:30:39.904324 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:39.904305    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2db6e97f-345b-4fa3-9b48-127cd97bb0d4-metrics-certs\") pod \"maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg\" (UID: \"2db6e97f-345b-4fa3-9b48-127cd97bb0d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg"
Sep 15 10:30:39.911063 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:39.911045    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sz5d6\" (UniqueName: \"kubernetes.io/projected/2db6e97f-345b-4fa3-9b48-127cd97bb0d4-kube-api-access-sz5d6\") pod \"maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg\" (UID: \"2db6e97f-345b-4fa3-9b48-127cd97bb0d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg"
Sep 15 10:30:40.001384 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:40.001330    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:40.347455 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:40.347344    2583 patch_prober.go:28] interesting pod/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 15 10:30:40.347455 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:40.347414    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx" podUID="3cead169-6453-4e87-8ab8-d1d47e6502d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 15 10:30:40.405225 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:40.405191    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:40.405754 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:40.405730    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2db6e97f-345b-4fa3-9b48-127cd97bb0d4-maas-api-tls\") pod \"maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg\" (UID: \"2db6e97f-345b-4fa3-9b48-127cd97bb0d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg"
Sep 15 10:30:40.405881 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:30:40.405864    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-234840-serving-cert: secret "maas-api-e2e-inf-b-234840-serving-cert" not found
Sep 15 10:30:40.405934 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:30:40.405920    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2db6e97f-345b-4fa3-9b48-127cd97bb0d4-maas-api-tls podName:2db6e97f-345b-4fa3-9b48-127cd97bb0d4 nodeName:}" failed. No retries permitted until 2026-09-15 10:30:41.405906231 +0000 UTC m=+756.652873180 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2db6e97f-345b-4fa3-9b48-127cd97bb0d4-maas-api-tls") pod "maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg" (UID: "2db6e97f-345b-4fa3-9b48-127cd97bb0d4") : secret "maas-api-e2e-inf-b-234840-serving-cert" not found
Sep 15 10:30:40.504216 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:40.504180    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:40.828208 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:40.828176    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:41.342189 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:41.342152    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:41.347468 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:41.347416    2583 patch_prober.go:28] interesting pod/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 15 10:30:41.347650 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:41.347498    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx" podUID="3cead169-6453-4e87-8ab8-d1d47e6502d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 15 10:30:41.417606 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:41.417575    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2db6e97f-345b-4fa3-9b48-127cd97bb0d4-maas-api-tls\") pod \"maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg\" (UID: \"2db6e97f-345b-4fa3-9b48-127cd97bb0d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg"
Sep 15 10:30:41.420241 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:41.420218    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2db6e97f-345b-4fa3-9b48-127cd97bb0d4-maas-api-tls\") pod \"maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg\" (UID: \"2db6e97f-345b-4fa3-9b48-127cd97bb0d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg"
Sep 15 10:30:41.475637 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:41.475604    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg"
Sep 15 10:30:41.752783 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:41.750876    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg"]
Sep 15 10:30:42.079690 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:42.079593    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg" event={"ID":"2db6e97f-345b-4fa3-9b48-127cd97bb0d4","Type":"ContainerStarted","Data":"3f08879986c6934ea3eeaa28f1f258b27228e277736bb40254a76922eea37145"}
Sep 15 10:30:42.336750 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:42.336659    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:42.347864 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:42.347832    2583 patch_prober.go:28] interesting pod/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 15 10:30:42.348381 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:42.348352    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx" podUID="3cead169-6453-4e87-8ab8-d1d47e6502d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 15 10:30:43.043683 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:43.043655    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp"
Sep 15 10:30:43.351872 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:43.351784    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:43.352269 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:43.352245    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:43.352849 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:43.352833    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:30:43.535270 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:43.535238    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:44.096242 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:44.096213    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg" event={"ID":"2db6e97f-345b-4fa3-9b48-127cd97bb0d4","Type":"ContainerStarted","Data":"57a4e042eee80c63717957eb7b9c48d4d7c74b80fd6f5d90de6861ee0d4e03c1"}
Sep 15 10:30:44.096431 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:44.096371    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg"
Sep 15 10:30:44.112499 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:44.112438    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg" podStartSLOduration=3.265873234 podStartE2EDuration="5.112421906s" totalImagesPullingTime="1.846548672s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:30:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:30:44.110195287 +0000 UTC m=+759.357162270" watchObservedRunningTime="2026-09-15 10:30:44.112421906 +0000 UTC m=+759.359388918"
Sep 15 10:30:44.277750 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:44.277718    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:44.379194 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:44.379114    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:44.506583 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:44.506554    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:46.079551 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:46.079511    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:46.143927 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:46.143897    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:46.270928 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:46.270893    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:47.115797 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:47.115764    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:47.780757 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:47.780708    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:48.407197 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:48.407171    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:50.107576 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:50.107549    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg"
Sep 15 10:30:50.423404 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:50.423353    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:50.519194 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:50.519152    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:51.006256 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:51.006223    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:51.933250 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:51.933216    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:52.187296 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:52.187216    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:53.179393 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:53.179357    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:53.940361 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:53.940266    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:54.726971 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:54.726926    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:54.894082 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:54.894036    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:55.543839 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:55.543800    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:56.181686 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:56.181648    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:56.617377 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:56.617275    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:56.720735 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:56.720701    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:56.819862 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:56.819826    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:56.920247 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:56.920157    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:57.181840 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:57.181751    2583 generic.go:356] "Generic (PLEG): container finished" podID="cffc93c4-c0e4-480c-b91f-5fd9cfab8234" containerID="1eb8eec838a3efcfaae5e42127981f9c5090db9c998a413841c514fa6c96c2b5" exitCode=0
Sep 15 10:30:57.181840 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:57.181788    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp" event={"ID":"cffc93c4-c0e4-480c-b91f-5fd9cfab8234","Type":"ContainerDied","Data":"1eb8eec838a3efcfaae5e42127981f9c5090db9c998a413841c514fa6c96c2b5"}
Sep 15 10:30:57.182275 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:57.182259    2583 scope.go:117] "RemoveContainer" containerID="1eb8eec838a3efcfaae5e42127981f9c5090db9c998a413841c514fa6c96c2b5"
Sep 15 10:30:57.183506 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:57.183475    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:30:57.183620 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:57.183547    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:30:57.183620 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:57.183577    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:30:57.517037 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:57.516933    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:58.189278 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:58.189244    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp" event={"ID":"cffc93c4-c0e4-480c-b91f-5fd9cfab8234","Type":"ContainerStarted","Data":"9dc540cf29fff462c93d48670a692214e2341f66ad7943cb8d0f8ae44bb8f4d5"}
Sep 15 10:30:58.278491 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:58.278428    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:58.669493 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:58.669437    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:58.670740 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:58.670715    2583 patch_prober.go:28] interesting pod/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 15 10:30:58.670858 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:58.670762    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp" podUID="cffc93c4-c0e4-480c-b91f-5fd9cfab8234" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 15 10:30:58.821924 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:58.821889    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:58.923367 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:58.923276    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:58.963584 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:58.963547    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:30:59.003150 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:59.003116    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:59.063356 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:59.063300    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:59.186007 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:59.185903    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:59.514796 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:59.514686    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:59.614089 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:59.614054    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:59.669052 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:59.669010    2583 patch_prober.go:28] interesting pod/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 15 10:30:59.669260 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:59.669103    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp" podUID="cffc93c4-c0e4-480c-b91f-5fd9cfab8234" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 15 10:30:59.709833 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:59.709795    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:30:59.818519 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:30:59.818403    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:00.121981 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:00.121898    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:00.510286 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:00.510251    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:00.613276 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:00.613241    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:00.669109 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:00.669071    2583 patch_prober.go:28] interesting pod/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 15 10:31:00.669285 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:00.669149    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp" podUID="cffc93c4-c0e4-480c-b91f-5fd9cfab8234" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 15 10:31:01.669773 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:01.669715    2583 patch_prober.go:28] interesting pod/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 15 10:31:01.670763 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:01.670336    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp" podUID="cffc93c4-c0e4-480c-b91f-5fd9cfab8234" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 15 10:31:01.686943 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:01.686888    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:02.398296 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:02.398265    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:02.669824 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:02.669718    2583 patch_prober.go:28] interesting pod/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 15 10:31:02.669824 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:02.669783    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp" podUID="cffc93c4-c0e4-480c-b91f-5fd9cfab8234" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 15 10:31:02.794985 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:02.794954    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:02.896850 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:02.896812    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:03.237177 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:03.237144    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:03.669169 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:03.669130    2583 patch_prober.go:28] interesting pod/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 15 10:31:03.669357 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:03.669186    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp" podUID="cffc93c4-c0e4-480c-b91f-5fd9cfab8234" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 15 10:31:04.000386 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:04.000291    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:04.673358 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:04.673329    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:31:04.674325 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:04.674309    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:31:04.833753 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:04.833720    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:05.581670 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:05.581631    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc"]
Sep 15 10:31:05.585560 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:05.585537    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc"
Sep 15 10:31:05.587824 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:05.587781    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-b797d492-serving-cert\""
Sep 15 10:31:05.599001 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:05.598671    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc"]
Sep 15 10:31:05.613665 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:05.613638    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:05.662306 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:05.662269    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a026a14d-873f-4ae1-b00c-2e59443413bf-maas-api-tls\") pod \"maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc\" (UID: \"a026a14d-873f-4ae1-b00c-2e59443413bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc"
Sep 15 10:31:05.662306 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:05.662308    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5gzl\" (UniqueName: \"kubernetes.io/projected/a026a14d-873f-4ae1-b00c-2e59443413bf-kube-api-access-v5gzl\") pod \"maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc\" (UID: \"a026a14d-873f-4ae1-b00c-2e59443413bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc"
Sep 15 10:31:05.662609 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:05.662352    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a026a14d-873f-4ae1-b00c-2e59443413bf-openshift-service-ca\") pod \"maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc\" (UID: \"a026a14d-873f-4ae1-b00c-2e59443413bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc"
Sep 15 10:31:05.662609 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:05.662480    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a026a14d-873f-4ae1-b00c-2e59443413bf-metrics-certs\") pod \"maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc\" (UID: \"a026a14d-873f-4ae1-b00c-2e59443413bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc"
Sep 15 10:31:05.673378 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:05.673353    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:05.763250 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:05.763214    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a026a14d-873f-4ae1-b00c-2e59443413bf-maas-api-tls\") pod \"maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc\" (UID: \"a026a14d-873f-4ae1-b00c-2e59443413bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc"
Sep 15 10:31:05.763250 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:05.763256    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5gzl\" (UniqueName: \"kubernetes.io/projected/a026a14d-873f-4ae1-b00c-2e59443413bf-kube-api-access-v5gzl\") pod \"maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc\" (UID: \"a026a14d-873f-4ae1-b00c-2e59443413bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc"
Sep 15 10:31:05.763526 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:05.763304    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a026a14d-873f-4ae1-b00c-2e59443413bf-openshift-service-ca\") pod \"maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc\" (UID: \"a026a14d-873f-4ae1-b00c-2e59443413bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc"
Sep 15 10:31:05.763526 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:05.763349    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a026a14d-873f-4ae1-b00c-2e59443413bf-metrics-certs\") pod \"maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc\" (UID: \"a026a14d-873f-4ae1-b00c-2e59443413bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc"
Sep 15 10:31:05.764044 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:05.764018    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a026a14d-873f-4ae1-b00c-2e59443413bf-openshift-service-ca\") pod \"maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc\" (UID: \"a026a14d-873f-4ae1-b00c-2e59443413bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc"
Sep 15 10:31:05.765758 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:05.765739    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a026a14d-873f-4ae1-b00c-2e59443413bf-maas-api-tls\") pod \"maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc\" (UID: \"a026a14d-873f-4ae1-b00c-2e59443413bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc"
Sep 15 10:31:05.765868 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:05.765765    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a026a14d-873f-4ae1-b00c-2e59443413bf-metrics-certs\") pod \"maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc\" (UID: \"a026a14d-873f-4ae1-b00c-2e59443413bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc"
Sep 15 10:31:05.771513 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:05.771492    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5gzl\" (UniqueName: \"kubernetes.io/projected/a026a14d-873f-4ae1-b00c-2e59443413bf-kube-api-access-v5gzl\") pod \"maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc\" (UID: \"a026a14d-873f-4ae1-b00c-2e59443413bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc"
Sep 15 10:31:05.774596 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:05.774575    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:05.903027 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:05.902993    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc"
Sep 15 10:31:06.060015 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:06.059985    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:06.157714 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:06.157621    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:06.186426 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:06.186402    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc"]
Sep 15 10:31:06.188027 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:31:06.188002    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda026a14d_873f_4ae1_b00c_2e59443413bf.slice/crio-d8f223b4b6f0e7763f73aaa2f944f25dbfb5ae940cf21f58bae4f150c7bd0a15 WatchSource:0}: Error finding container d8f223b4b6f0e7763f73aaa2f944f25dbfb5ae940cf21f58bae4f150c7bd0a15: Status 404 returned error can't find the container with id d8f223b4b6f0e7763f73aaa2f944f25dbfb5ae940cf21f58bae4f150c7bd0a15
Sep 15 10:31:06.241296 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:06.239543    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc" event={"ID":"a026a14d-873f-4ae1-b00c-2e59443413bf","Type":"ContainerStarted","Data":"d8f223b4b6f0e7763f73aaa2f944f25dbfb5ae940cf21f58bae4f150c7bd0a15"}
Sep 15 10:31:06.476652 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:06.476514    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:07.179664 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:07.179632    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:07.246999 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:07.246961    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc" event={"ID":"a026a14d-873f-4ae1-b00c-2e59443413bf","Type":"ContainerStarted","Data":"f9d3ac1b87274319611ab4d81d66070efa274a4ea04b94a8c2820fa3ba68b326"}
Sep 15 10:31:07.247194 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:07.247033    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc"
Sep 15 10:31:07.266350 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:07.266289    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc" podStartSLOduration=1.8971315199999998 podStartE2EDuration="2.266271157s" totalImagesPullingTime="369.139637ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:31:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:31:07.26328262 +0000 UTC m=+782.510249590" watchObservedRunningTime="2026-09-15 10:31:07.266271157 +0000 UTC m=+782.513238127"
Sep 15 10:31:07.974495 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:07.974459    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:13.258379 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:13.258348    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc"
Sep 15 10:31:20.943164 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:20.943131    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:27.615552 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:27.615515    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:28.615340 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:28.615306    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:29.296229 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:29.296187    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:29.988311 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:29.988278    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:30.685139 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:30.685105    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:31.283080 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:31.283032    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:31.970327 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:31.970288    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:32.587543 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:32.587513    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:33.903795 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:33.903758    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:34.491978 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:34.491934    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc"]
Sep 15 10:31:34.492194 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:34.492166    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc" podUID="a026a14d-873f-4ae1-b00c-2e59443413bf" containerName="maas-api" containerID="cri-o://f9d3ac1b87274319611ab4d81d66070efa274a4ea04b94a8c2820fa3ba68b326" gracePeriod=30
Sep 15 10:31:34.523804 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:34.523778    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:34.623760 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:34.623723    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:34.709280 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:34.709249    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:34.744364 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:34.744297    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc"
Sep 15 10:31:34.769054 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:34.769016    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:34.851312 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:34.851282    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a026a14d-873f-4ae1-b00c-2e59443413bf-maas-api-tls\") pod \"a026a14d-873f-4ae1-b00c-2e59443413bf\" (UID: \"a026a14d-873f-4ae1-b00c-2e59443413bf\") "
Sep 15 10:31:34.851505 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:34.851325    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a026a14d-873f-4ae1-b00c-2e59443413bf-openshift-service-ca\") pod \"a026a14d-873f-4ae1-b00c-2e59443413bf\" (UID: \"a026a14d-873f-4ae1-b00c-2e59443413bf\") "
Sep 15 10:31:34.851505 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:34.851346    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v5gzl\" (UniqueName: \"kubernetes.io/projected/a026a14d-873f-4ae1-b00c-2e59443413bf-kube-api-access-v5gzl\") pod \"a026a14d-873f-4ae1-b00c-2e59443413bf\" (UID: \"a026a14d-873f-4ae1-b00c-2e59443413bf\") "
Sep 15 10:31:34.851505 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:34.851373    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a026a14d-873f-4ae1-b00c-2e59443413bf-metrics-certs\") pod \"a026a14d-873f-4ae1-b00c-2e59443413bf\" (UID: \"a026a14d-873f-4ae1-b00c-2e59443413bf\") "
Sep 15 10:31:34.851918 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:34.851891    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a026a14d-873f-4ae1-b00c-2e59443413bf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a026a14d-873f-4ae1-b00c-2e59443413bf" (UID: "a026a14d-873f-4ae1-b00c-2e59443413bf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:31:34.853260 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:34.853227    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:34.854141 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:34.854108    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a026a14d-873f-4ae1-b00c-2e59443413bf-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a026a14d-873f-4ae1-b00c-2e59443413bf" (UID: "a026a14d-873f-4ae1-b00c-2e59443413bf"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:31:34.854243 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:34.854167    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a026a14d-873f-4ae1-b00c-2e59443413bf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a026a14d-873f-4ae1-b00c-2e59443413bf" (UID: "a026a14d-873f-4ae1-b00c-2e59443413bf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:31:34.854243 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:34.854187    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a026a14d-873f-4ae1-b00c-2e59443413bf-kube-api-access-v5gzl" (OuterVolumeSpecName: "kube-api-access-v5gzl") pod "a026a14d-873f-4ae1-b00c-2e59443413bf" (UID: "a026a14d-873f-4ae1-b00c-2e59443413bf"). InnerVolumeSpecName "kube-api-access-v5gzl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:31:34.953386 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:34.953339    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a026a14d-873f-4ae1-b00c-2e59443413bf-maas-api-tls\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:31:34.953386 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:34.953381    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a026a14d-873f-4ae1-b00c-2e59443413bf-openshift-service-ca\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:31:34.953872 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:34.953396    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5gzl\" (UniqueName: \"kubernetes.io/projected/a026a14d-873f-4ae1-b00c-2e59443413bf-kube-api-access-v5gzl\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:31:34.953872 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:34.953412    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a026a14d-873f-4ae1-b00c-2e59443413bf-metrics-certs\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:31:35.104988 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:35.104901    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:35.274645 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:35.274612    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:35.415533 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:35.415501    2583 generic.go:356] "Generic (PLEG): container finished" podID="a026a14d-873f-4ae1-b00c-2e59443413bf" containerID="f9d3ac1b87274319611ab4d81d66070efa274a4ea04b94a8c2820fa3ba68b326" exitCode=0
Sep 15 10:31:35.415712 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:35.415548    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc"
Sep 15 10:31:35.415712 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:35.415587    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc" event={"ID":"a026a14d-873f-4ae1-b00c-2e59443413bf","Type":"ContainerDied","Data":"f9d3ac1b87274319611ab4d81d66070efa274a4ea04b94a8c2820fa3ba68b326"}
Sep 15 10:31:35.415712 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:35.415642    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc" event={"ID":"a026a14d-873f-4ae1-b00c-2e59443413bf","Type":"ContainerDied","Data":"d8f223b4b6f0e7763f73aaa2f944f25dbfb5ae940cf21f58bae4f150c7bd0a15"}
Sep 15 10:31:35.415712 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:35.415666    2583 scope.go:117] "RemoveContainer" containerID="f9d3ac1b87274319611ab4d81d66070efa274a4ea04b94a8c2820fa3ba68b326"
Sep 15 10:31:35.424985 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:35.424968    2583 scope.go:117] "RemoveContainer" containerID="f9d3ac1b87274319611ab4d81d66070efa274a4ea04b94a8c2820fa3ba68b326"
Sep 15 10:31:35.425252 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:31:35.425236    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9d3ac1b87274319611ab4d81d66070efa274a4ea04b94a8c2820fa3ba68b326\": container with ID starting with f9d3ac1b87274319611ab4d81d66070efa274a4ea04b94a8c2820fa3ba68b326 not found: ID does not exist" containerID="f9d3ac1b87274319611ab4d81d66070efa274a4ea04b94a8c2820fa3ba68b326"
Sep 15 10:31:35.425308 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:35.425260    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9d3ac1b87274319611ab4d81d66070efa274a4ea04b94a8c2820fa3ba68b326"} err="failed to get container status \"f9d3ac1b87274319611ab4d81d66070efa274a4ea04b94a8c2820fa3ba68b326\": rpc error: code = NotFound desc = could not find container \"f9d3ac1b87274319611ab4d81d66070efa274a4ea04b94a8c2820fa3ba68b326\": container with ID starting with f9d3ac1b87274319611ab4d81d66070efa274a4ea04b94a8c2820fa3ba68b326 not found: ID does not exist"
Sep 15 10:31:35.431601 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:35.431576    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc"]
Sep 15 10:31:35.434025 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:35.433998    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b797d492-6f574d9548-jvwlc"]
Sep 15 10:31:35.561752 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:35.561723    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:35.804889 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:35.804809    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:35.899297 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:35.899258    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:36.589524 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:36.589489    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:37.270823 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:37.270789    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:37.301163 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:37.301132    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a026a14d-873f-4ae1-b00c-2e59443413bf" path="/var/lib/kubelet/pods/a026a14d-873f-4ae1-b00c-2e59443413bf/volumes"
Sep 15 10:31:37.872387 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:37.872348    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:37.972963 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:37.972922    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:38.911829 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:38.911798    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:39.593848 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:39.593819    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:39.696000 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:39.695964    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:39.796345 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:39.796315    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:40.148748 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:40.148715    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:40.252311 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:40.252279    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:40.674834 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:40.674798    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:41.357223 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:41.357192    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:41.973588 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:41.973556    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:31:42.684894 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:31:42.684859    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:01.510734 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:01.510694    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:02.149816 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:02.149780    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:03.044231 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:03.044197    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:03.600908 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:03.600877    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:04.399492 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:04.399438    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:05.089256 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:05.089225    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:05.990967 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:05.990931    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:06.681871 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:06.681840    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:07.401720 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:07.401687    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:08.114259 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:08.114224    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:08.776956 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:08.776924    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:09.297829 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:09.297794    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:09.398468 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:09.398427    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:09.480732 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:09.480701    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:09.542485 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:09.542434    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:10.381120 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:10.381089    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:10.608193 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:10.608161    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:11.033284 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:11.033249    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:11.130659 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:11.130625    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:11.301250 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:11.301170    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:11.508039 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:11.508005    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:11.806014 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:11.805982    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:12.449879 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:12.449851    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:13.283471 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:13.283419    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:13.899505 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:13.899468    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:14.182306 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:14.182225    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:14.273681 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:14.273648    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:14.812305 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:14.812263    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:15.496109 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:15.496079    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:16.178062 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:16.178027    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:16.793642 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:16.793610    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:17.502217 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:17.502185    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:18.122119 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:18.122088    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:18.223735 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:18.223698    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:18.323413 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:18.323379    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:18.549247 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:18.549217    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:18.637939 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:18.637906    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:22.183718 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:22.183674    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:40.996796 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:40.996756    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:41.719263 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:41.719232    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:42.425336 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:42.425298    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:43.189145 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:43.189110    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:43.885220 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:43.885189    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:44.603459 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:44.603425    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:45.388120 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:45.388079    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:46.054167 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:46.054123    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:46.756168 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:46.756138    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:47.478338 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:47.478307    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:48.093390 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:48.093346    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:48.841189 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:48.841155    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:49.470411 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:49.470374    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:54.617278 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:54.617244    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:54.685287 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:54.685254    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:55.593034 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:55.593002    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:56.249748 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:56.249720    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:56.349912 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:56.349872    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:56.417376 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:56.417345    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:56.479659 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:56.479627    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:56.579697 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:56.579618    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:56.901572 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:56.901540    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:57.238693 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:57.238622    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:57.388302 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:57.388268    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:57.488610 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:57.488578    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:57.727433 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:57.727359    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:58.394299 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:58.394267    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:58.751639 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:58.751563    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:59.097510 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:59.097405    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:32:59.769351 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:32:59.769316    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:33:00.385984 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:00.385950    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:33:00.540999 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:00.540961    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:33:04.580052 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:04.580016    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:33:04.679247 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:04.679216    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:33:04.777512 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:04.777474    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:33:04.880115 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:04.880024    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:33:05.268881 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:05.268839    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:33:25.693955 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:25.693901    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:33:31.132341 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:31.132300    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:33:34.357815 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:34.357778    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:33:35.190393 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:35.190331    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:33:35.803584 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:35.803547    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:33:36.470358 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:36.470323    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:33:37.350417 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:37.350379    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:33:37.482229 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:37.482195    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:33:37.592983 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:37.592953    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:33:38.015231 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:38.015194    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:33:38.113554 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:38.113526    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:33:38.784918 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:38.784887    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:33:44.422867 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:44.422833    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp"]
Sep 15 10:33:44.423335 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:44.423122    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp" podUID="985b3d6b-3519-4a9c-b3c4-515b31bf6b75" containerName="maas-api" containerID="cri-o://053ecd05bbdfb90b7d4b67ef54d6347781fa5c026e30697e7fa3caa0ee0e4012" gracePeriod=30
Sep 15 10:33:44.451086 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:44.451056    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:33:44.523289 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:44.523233    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:33:44.667828 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:44.667802    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp"
Sep 15 10:33:44.697511 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:44.697422    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:33:44.715713 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:44.715677    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/985b3d6b-3519-4a9c-b3c4-515b31bf6b75-openshift-service-ca\") pod \"985b3d6b-3519-4a9c-b3c4-515b31bf6b75\" (UID: \"985b3d6b-3519-4a9c-b3c4-515b31bf6b75\") "
Sep 15 10:33:44.715875 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:44.715731    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/985b3d6b-3519-4a9c-b3c4-515b31bf6b75-metrics-certs\") pod \"985b3d6b-3519-4a9c-b3c4-515b31bf6b75\" (UID: \"985b3d6b-3519-4a9c-b3c4-515b31bf6b75\") "
Sep 15 10:33:44.715875 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:44.715812    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/985b3d6b-3519-4a9c-b3c4-515b31bf6b75-maas-api-tls\") pod \"985b3d6b-3519-4a9c-b3c4-515b31bf6b75\" (UID: \"985b3d6b-3519-4a9c-b3c4-515b31bf6b75\") "
Sep 15 10:33:44.715970 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:44.715925    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-86mkp\" (UniqueName: \"kubernetes.io/projected/985b3d6b-3519-4a9c-b3c4-515b31bf6b75-kube-api-access-86mkp\") pod \"985b3d6b-3519-4a9c-b3c4-515b31bf6b75\" (UID: \"985b3d6b-3519-4a9c-b3c4-515b31bf6b75\") "
Sep 15 10:33:44.716072 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:44.716046    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/985b3d6b-3519-4a9c-b3c4-515b31bf6b75-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "985b3d6b-3519-4a9c-b3c4-515b31bf6b75" (UID: "985b3d6b-3519-4a9c-b3c4-515b31bf6b75"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:33:44.716214 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:44.716200    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/985b3d6b-3519-4a9c-b3c4-515b31bf6b75-openshift-service-ca\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:33:44.718009 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:44.717977    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/985b3d6b-3519-4a9c-b3c4-515b31bf6b75-kube-api-access-86mkp" (OuterVolumeSpecName: "kube-api-access-86mkp") pod "985b3d6b-3519-4a9c-b3c4-515b31bf6b75" (UID: "985b3d6b-3519-4a9c-b3c4-515b31bf6b75"). InnerVolumeSpecName "kube-api-access-86mkp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:33:44.718137 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:44.718078    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/985b3d6b-3519-4a9c-b3c4-515b31bf6b75-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "985b3d6b-3519-4a9c-b3c4-515b31bf6b75" (UID: "985b3d6b-3519-4a9c-b3c4-515b31bf6b75"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:33:44.718137 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:44.718092    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/985b3d6b-3519-4a9c-b3c4-515b31bf6b75-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "985b3d6b-3519-4a9c-b3c4-515b31bf6b75" (UID: "985b3d6b-3519-4a9c-b3c4-515b31bf6b75"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:33:44.768234 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:44.768206    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:33:44.817936 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:44.817899    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/985b3d6b-3519-4a9c-b3c4-515b31bf6b75-metrics-certs\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:33:44.817936 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:44.817936    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/985b3d6b-3519-4a9c-b3c4-515b31bf6b75-maas-api-tls\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:33:44.818188 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:44.817951    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86mkp\" (UniqueName: \"kubernetes.io/projected/985b3d6b-3519-4a9c-b3c4-515b31bf6b75-kube-api-access-86mkp\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:33:44.959494 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:44.959358    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:33:45.085019 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:45.084985    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:33:45.168765 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:45.168735    2583 generic.go:356] "Generic (PLEG): container finished" podID="985b3d6b-3519-4a9c-b3c4-515b31bf6b75" containerID="053ecd05bbdfb90b7d4b67ef54d6347781fa5c026e30697e7fa3caa0ee0e4012" exitCode=0
Sep 15 10:33:45.168966 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:45.168785    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp"
Sep 15 10:33:45.168966 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:45.168819    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp" event={"ID":"985b3d6b-3519-4a9c-b3c4-515b31bf6b75","Type":"ContainerDied","Data":"053ecd05bbdfb90b7d4b67ef54d6347781fa5c026e30697e7fa3caa0ee0e4012"}
Sep 15 10:33:45.168966 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:45.168890    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp" event={"ID":"985b3d6b-3519-4a9c-b3c4-515b31bf6b75","Type":"ContainerDied","Data":"b2399138a1a1fe63bdf802d137aab6938f0c3e20ce3861bb5cf850a39549db70"}
Sep 15 10:33:45.168966 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:45.168922    2583 scope.go:117] "RemoveContainer" containerID="053ecd05bbdfb90b7d4b67ef54d6347781fa5c026e30697e7fa3caa0ee0e4012"
Sep 15 10:33:45.178827 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:45.178810    2583 scope.go:117] "RemoveContainer" containerID="053ecd05bbdfb90b7d4b67ef54d6347781fa5c026e30697e7fa3caa0ee0e4012"
Sep 15 10:33:45.179119 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:33:45.179098    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"053ecd05bbdfb90b7d4b67ef54d6347781fa5c026e30697e7fa3caa0ee0e4012\": container with ID starting with 053ecd05bbdfb90b7d4b67ef54d6347781fa5c026e30697e7fa3caa0ee0e4012 not found: ID does not exist" containerID="053ecd05bbdfb90b7d4b67ef54d6347781fa5c026e30697e7fa3caa0ee0e4012"
Sep 15 10:33:45.179190 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:45.179129    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"053ecd05bbdfb90b7d4b67ef54d6347781fa5c026e30697e7fa3caa0ee0e4012"} err="failed to get container status \"053ecd05bbdfb90b7d4b67ef54d6347781fa5c026e30697e7fa3caa0ee0e4012\": rpc error: code = NotFound desc = could not find container \"053ecd05bbdfb90b7d4b67ef54d6347781fa5c026e30697e7fa3caa0ee0e4012\": container with ID starting with 053ecd05bbdfb90b7d4b67ef54d6347781fa5c026e30697e7fa3caa0ee0e4012 not found: ID does not exist"
Sep 15 10:33:45.190883 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:45.190861    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp"]
Sep 15 10:33:45.193481 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:45.193459    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-8110bc-6d74bcfbb5-v9grp"]
Sep 15 10:33:45.301267 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:45.301187    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="985b3d6b-3519-4a9c-b3c4-515b31bf6b75" path="/var/lib/kubelet/pods/985b3d6b-3519-4a9c-b3c4-515b31bf6b75/volumes"
Sep 15 10:33:45.699341 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:45.699307    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:33:46.311743 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:46.311700    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:33:47.047049 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:47.047020    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:33:47.671332 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:47.671296    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:33:50.039743 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:50.039711    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:33:53.459790 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:53.459755    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:33:59.129251 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:59.129221    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:33:59.234945 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:59.234909    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:33:59.329522 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:33:59.329491    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:02.188868 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:02.188834    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:04.683155 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:04.683127    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:09.782745 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:09.782713    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:10.544165 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:10.544098    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:16.225560 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:16.225518    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:16.326994 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:16.326960    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:16.427114 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:16.427082    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:16.528081 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:16.527986    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:16.827049 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:16.826945    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:16.966987 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:16.966939    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:17.126968 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:17.126869    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:17.226070 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:17.226042    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:17.424584 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:17.424558    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:17.526000 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:17.525974    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:18.658182 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:18.658117    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:22.599880 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:22.599841    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:24.485275 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:24.485214    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:24.545712 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:24.545674    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:25.138671 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:25.138632    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:26.692340 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:26.692309    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:26.879679 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:26.879650    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"]
Sep 15 10:34:26.879955 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:26.879927    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp" podUID="cffc93c4-c0e4-480c-b91f-5fd9cfab8234" containerName="istio-proxy" containerID="cri-o://9dc540cf29fff462c93d48670a692214e2341f66ad7943cb8d0f8ae44bb8f4d5" gracePeriod=30
Sep 15 10:34:26.903416 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:26.903389    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:27.321007 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:27.320950    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:27.599817 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:27.599739    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:27.848100 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:27.848022    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:29.703320 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:29.703284    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:30.314376 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:30.314344    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:30.436062 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:30.436022    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:30.524953 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:30.524917    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:31.181897 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:31.181860    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:31.448593 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:31.448508    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:31.515633 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:31.515584    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:31.674883 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:31.674855    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:31.971071 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:31.971024    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:32.071732 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.071697    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:32.141779 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.141725    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:34:32.211769 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.211744    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:32.299468 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.299416    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-istio-data\") pod \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") "
Sep 15 10:34:32.299468 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.299473    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-istio-podinfo\") pod \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") "
Sep 15 10:34:32.299708 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.299529    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-istio-envoy\") pod \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") "
Sep 15 10:34:32.299708 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.299564    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-workload-certs\") pod \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") "
Sep 15 10:34:32.299708 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.299591    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-istio-token\") pod \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") "
Sep 15 10:34:32.299708 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.299629    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-workload-socket\") pod \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") "
Sep 15 10:34:32.299708 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.299663    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-credential-socket\") pod \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") "
Sep 15 10:34:32.299708 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.299685    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d752d\" (UniqueName: \"kubernetes.io/projected/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-kube-api-access-d752d\") pod \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") "
Sep 15 10:34:32.299708 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.299692    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-istio-data" (OuterVolumeSpecName: "istio-data") pod "cffc93c4-c0e4-480c-b91f-5fd9cfab8234" (UID: "cffc93c4-c0e4-480c-b91f-5fd9cfab8234"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:34:32.300070 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.299715    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-istiod-ca-cert\") pod \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\" (UID: \"cffc93c4-c0e4-480c-b91f-5fd9cfab8234\") "
Sep 15 10:34:32.300070 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.299951    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cffc93c4-c0e4-480c-b91f-5fd9cfab8234" (UID: "cffc93c4-c0e4-480c-b91f-5fd9cfab8234"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:34:32.300070 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.299989    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-istio-data\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:34:32.300070 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.299964    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cffc93c4-c0e4-480c-b91f-5fd9cfab8234" (UID: "cffc93c4-c0e4-480c-b91f-5fd9cfab8234"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:34:32.300070 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.300039    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cffc93c4-c0e4-480c-b91f-5fd9cfab8234" (UID: "cffc93c4-c0e4-480c-b91f-5fd9cfab8234"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:34:32.300333 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.300230    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cffc93c4-c0e4-480c-b91f-5fd9cfab8234" (UID: "cffc93c4-c0e4-480c-b91f-5fd9cfab8234"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:34:32.302104 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.302073    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-kube-api-access-d752d" (OuterVolumeSpecName: "kube-api-access-d752d") pod "cffc93c4-c0e4-480c-b91f-5fd9cfab8234" (UID: "cffc93c4-c0e4-480c-b91f-5fd9cfab8234"). InnerVolumeSpecName "kube-api-access-d752d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:34:32.302104 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.302076    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cffc93c4-c0e4-480c-b91f-5fd9cfab8234" (UID: "cffc93c4-c0e4-480c-b91f-5fd9cfab8234"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 10:34:32.302273 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.302222    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cffc93c4-c0e4-480c-b91f-5fd9cfab8234" (UID: "cffc93c4-c0e4-480c-b91f-5fd9cfab8234"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:34:32.302322 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.302304    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-istio-token" (OuterVolumeSpecName: "istio-token") pod "cffc93c4-c0e4-480c-b91f-5fd9cfab8234" (UID: "cffc93c4-c0e4-480c-b91f-5fd9cfab8234"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:34:32.401282 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.401200    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-workload-certs\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:34:32.401282 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.401235    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-istio-token\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:34:32.401282 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.401248    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-workload-socket\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:34:32.401282 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.401261    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-credential-socket\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:34:32.401282 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.401274    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d752d\" (UniqueName: \"kubernetes.io/projected/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-kube-api-access-d752d\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:34:32.401580 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.401287    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-istiod-ca-cert\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:34:32.401580 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.401299    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-istio-podinfo\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:34:32.401580 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.401316    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cffc93c4-c0e4-480c-b91f-5fd9cfab8234-istio-envoy\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:34:32.447501 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.447470    2583 generic.go:356] "Generic (PLEG): container finished" podID="cffc93c4-c0e4-480c-b91f-5fd9cfab8234" containerID="9dc540cf29fff462c93d48670a692214e2341f66ad7943cb8d0f8ae44bb8f4d5" exitCode=0
Sep 15 10:34:32.447665 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.447509    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp" event={"ID":"cffc93c4-c0e4-480c-b91f-5fd9cfab8234","Type":"ContainerDied","Data":"9dc540cf29fff462c93d48670a692214e2341f66ad7943cb8d0f8ae44bb8f4d5"}
Sep 15 10:34:32.447665 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.447516    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"
Sep 15 10:34:32.447665 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.447546    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp" event={"ID":"cffc93c4-c0e4-480c-b91f-5fd9cfab8234","Type":"ContainerDied","Data":"20e22210ff09a622530578ae5a687afe901937add37a3371e1069fbdcf9b933d"}
Sep 15 10:34:32.447665 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.447566    2583 scope.go:117] "RemoveContainer" containerID="9dc540cf29fff462c93d48670a692214e2341f66ad7943cb8d0f8ae44bb8f4d5"
Sep 15 10:34:32.457724 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.457708    2583 scope.go:117] "RemoveContainer" containerID="1eb8eec838a3efcfaae5e42127981f9c5090db9c998a413841c514fa6c96c2b5"
Sep 15 10:34:32.466934 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.466917    2583 scope.go:117] "RemoveContainer" containerID="9dc540cf29fff462c93d48670a692214e2341f66ad7943cb8d0f8ae44bb8f4d5"
Sep 15 10:34:32.467193 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:34:32.467175    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9dc540cf29fff462c93d48670a692214e2341f66ad7943cb8d0f8ae44bb8f4d5\": container with ID starting with 9dc540cf29fff462c93d48670a692214e2341f66ad7943cb8d0f8ae44bb8f4d5 not found: ID does not exist" containerID="9dc540cf29fff462c93d48670a692214e2341f66ad7943cb8d0f8ae44bb8f4d5"
Sep 15 10:34:32.467254 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.467201    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9dc540cf29fff462c93d48670a692214e2341f66ad7943cb8d0f8ae44bb8f4d5"} err="failed to get container status \"9dc540cf29fff462c93d48670a692214e2341f66ad7943cb8d0f8ae44bb8f4d5\": rpc error: code = NotFound desc = could not find container \"9dc540cf29fff462c93d48670a692214e2341f66ad7943cb8d0f8ae44bb8f4d5\": container with ID starting with 9dc540cf29fff462c93d48670a692214e2341f66ad7943cb8d0f8ae44bb8f4d5 not found: ID does not exist"
Sep 15 10:34:32.467254 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.467217    2583 scope.go:117] "RemoveContainer" containerID="1eb8eec838a3efcfaae5e42127981f9c5090db9c998a413841c514fa6c96c2b5"
Sep 15 10:34:32.467477 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:34:32.467456    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1eb8eec838a3efcfaae5e42127981f9c5090db9c998a413841c514fa6c96c2b5\": container with ID starting with 1eb8eec838a3efcfaae5e42127981f9c5090db9c998a413841c514fa6c96c2b5 not found: ID does not exist" containerID="1eb8eec838a3efcfaae5e42127981f9c5090db9c998a413841c514fa6c96c2b5"
Sep 15 10:34:32.467530 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.467482    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1eb8eec838a3efcfaae5e42127981f9c5090db9c998a413841c514fa6c96c2b5"} err="failed to get container status \"1eb8eec838a3efcfaae5e42127981f9c5090db9c998a413841c514fa6c96c2b5\": rpc error: code = NotFound desc = could not find container \"1eb8eec838a3efcfaae5e42127981f9c5090db9c998a413841c514fa6c96c2b5\": container with ID starting with 1eb8eec838a3efcfaae5e42127981f9c5090db9c998a413841c514fa6c96c2b5 not found: ID does not exist"
Sep 15 10:34:32.529675 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.529639    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"]
Sep 15 10:34:32.560883 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:32.560851    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-8110bc-openshift-default-6b454b884d-8kfrp"]
Sep 15 10:34:33.098087 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:33.098056    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:33.301339 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:33.301298    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cffc93c4-c0e4-480c-b91f-5fd9cfab8234" path="/var/lib/kubelet/pods/cffc93c4-c0e4-480c-b91f-5fd9cfab8234/volumes"
Sep 15 10:34:33.759429 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:33.759398    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6"]
Sep 15 10:34:33.759986 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:33.759964    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="985b3d6b-3519-4a9c-b3c4-515b31bf6b75" containerName="maas-api"
Sep 15 10:34:33.759986 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:33.759986    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="985b3d6b-3519-4a9c-b3c4-515b31bf6b75" containerName="maas-api"
Sep 15 10:34:33.760475 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:33.760022    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a026a14d-873f-4ae1-b00c-2e59443413bf" containerName="maas-api"
Sep 15 10:34:33.760475 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:33.760030    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a026a14d-873f-4ae1-b00c-2e59443413bf" containerName="maas-api"
Sep 15 10:34:33.760475 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:33.760043    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cffc93c4-c0e4-480c-b91f-5fd9cfab8234" containerName="istio-proxy"
Sep 15 10:34:33.760475 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:33.760052    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="cffc93c4-c0e4-480c-b91f-5fd9cfab8234" containerName="istio-proxy"
Sep 15 10:34:33.760475 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:33.760062    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cffc93c4-c0e4-480c-b91f-5fd9cfab8234" containerName="istio-proxy"
Sep 15 10:34:33.760475 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:33.760070    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="cffc93c4-c0e4-480c-b91f-5fd9cfab8234" containerName="istio-proxy"
Sep 15 10:34:33.760475 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:33.760196    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="cffc93c4-c0e4-480c-b91f-5fd9cfab8234" containerName="istio-proxy"
Sep 15 10:34:33.760475 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:33.760212    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="985b3d6b-3519-4a9c-b3c4-515b31bf6b75" containerName="maas-api"
Sep 15 10:34:33.760475 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:33.760231    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="cffc93c4-c0e4-480c-b91f-5fd9cfab8234" containerName="istio-proxy"
Sep 15 10:34:33.760475 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:33.760240    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a026a14d-873f-4ae1-b00c-2e59443413bf" containerName="maas-api"
Sep 15 10:34:33.763753 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:33.763738    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6"
Sep 15 10:34:33.765801 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:33.765783    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-aeb4e3-serving-cert\""
Sep 15 10:34:33.774736 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:33.774715    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6"]
Sep 15 10:34:33.789479 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:33.789430    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:33.853397 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:33.853367    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:33.915984 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:33.915949    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1-metrics-certs\") pod \"maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6\" (UID: \"c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6"
Sep 15 10:34:33.916169 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:33.915987    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1-openshift-service-ca\") pod \"maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6\" (UID: \"c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6"
Sep 15 10:34:33.916169 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:33.916071    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d77ck\" (UniqueName: \"kubernetes.io/projected/c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1-kube-api-access-d77ck\") pod \"maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6\" (UID: \"c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6"
Sep 15 10:34:33.916287 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:33.916187    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1-maas-api-tls\") pod \"maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6\" (UID: \"c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6"
Sep 15 10:34:34.017096 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:34.017023    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1-maas-api-tls\") pod \"maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6\" (UID: \"c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6"
Sep 15 10:34:34.017096 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:34.017070    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1-metrics-certs\") pod \"maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6\" (UID: \"c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6"
Sep 15 10:34:34.017096 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:34.017087    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1-openshift-service-ca\") pod \"maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6\" (UID: \"c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6"
Sep 15 10:34:34.017386 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:34.017131    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d77ck\" (UniqueName: \"kubernetes.io/projected/c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1-kube-api-access-d77ck\") pod \"maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6\" (UID: \"c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6"
Sep 15 10:34:34.017386 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:34:34.017181    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-aeb4e3-serving-cert: secret "maas-api-e2e-explicit-aeb4e3-serving-cert" not found
Sep 15 10:34:34.017386 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:34:34.017249    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1-maas-api-tls podName:c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1 nodeName:}" failed. No retries permitted until 2026-09-15 10:34:34.51722837 +0000 UTC m=+989.764195335 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1-maas-api-tls") pod "maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6" (UID: "c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1") : secret "maas-api-e2e-explicit-aeb4e3-serving-cert" not found
Sep 15 10:34:34.017832 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:34.017815    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1-openshift-service-ca\") pod \"maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6\" (UID: \"c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6"
Sep 15 10:34:34.019395 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:34.019370    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1-metrics-certs\") pod \"maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6\" (UID: \"c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6"
Sep 15 10:34:34.028516 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:34.028490    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d77ck\" (UniqueName: \"kubernetes.io/projected/c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1-kube-api-access-d77ck\") pod \"maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6\" (UID: \"c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6"
Sep 15 10:34:34.306341 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:34.306261    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:34.486406 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:34.486366    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:34.522807 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:34.522776    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1-maas-api-tls\") pod \"maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6\" (UID: \"c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6"
Sep 15 10:34:34.525028 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:34.525007    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1-maas-api-tls\") pod \"maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6\" (UID: \"c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6"
Sep 15 10:34:34.586126 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:34.586060    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:34.675249 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:34.675214    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6"
Sep 15 10:34:34.951945 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:34.951920    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6"]
Sep 15 10:34:34.953285 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:34:34.953263    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6e0d2bc_a719_482b_98c2_cc7c1c9b41d1.slice/crio-2be5aab308b5f9c6cf152a5860618c9c41557b55e43b9773d2eb836c80f433f7 WatchSource:0}: Error finding container 2be5aab308b5f9c6cf152a5860618c9c41557b55e43b9773d2eb836c80f433f7: Status 404 returned error can't find the container with id 2be5aab308b5f9c6cf152a5860618c9c41557b55e43b9773d2eb836c80f433f7
Sep 15 10:34:34.954697 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:34.954679    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 10:34:35.474531 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:35.474501    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6" event={"ID":"c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1","Type":"ContainerStarted","Data":"2be5aab308b5f9c6cf152a5860618c9c41557b55e43b9773d2eb836c80f433f7"}
Sep 15 10:34:36.481998 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:36.481961    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6" event={"ID":"c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1","Type":"ContainerStarted","Data":"06eb528907cb8dbc785a5c39a8c67cbc49c23eb9c2032f9d7578a4bf826b5406"}
Sep 15 10:34:36.482388 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:36.482026    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6"
Sep 15 10:34:36.504632 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:36.504586    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6" podStartSLOduration=2.503493477 podStartE2EDuration="3.504568188s" totalImagesPullingTime="1.001074711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:34:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:34:36.501082483 +0000 UTC m=+991.748049453" watchObservedRunningTime="2026-09-15 10:34:36.504568188 +0000 UTC m=+991.751535158"
Sep 15 10:34:37.149630 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:37.149598    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:37.792419 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:37.792386    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:38.321997 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:38.321963    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:38.763149 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:38.763111    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:39.478261 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:39.478218    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:40.104581 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:40.104549    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:40.867292 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:40.867259    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:41.455475 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:41.455429    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8f3975b3-b679f575-8tpqj"]
Sep 15 10:34:41.459485 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:41.459465    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f3975b3-b679f575-8tpqj"
Sep 15 10:34:41.461798 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:41.461775    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-8f3975b3-serving-cert\""
Sep 15 10:34:41.471154 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:41.471134    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8f3975b3-b679f575-8tpqj"]
Sep 15 10:34:41.491781 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:41.491755    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c5b75f9-d22a-45fe-bedb-69d27b3e4e94-metrics-certs\") pod \"maas-api-e2e-mt-8f3975b3-b679f575-8tpqj\" (UID: \"3c5b75f9-d22a-45fe-bedb-69d27b3e4e94\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f3975b3-b679f575-8tpqj"
Sep 15 10:34:41.491913 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:41.491815    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:41.491913 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:41.491848    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c5b75f9-d22a-45fe-bedb-69d27b3e4e94-openshift-service-ca\") pod \"maas-api-e2e-mt-8f3975b3-b679f575-8tpqj\" (UID: \"3c5b75f9-d22a-45fe-bedb-69d27b3e4e94\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f3975b3-b679f575-8tpqj"
Sep 15 10:34:41.491913 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:41.491886    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vb6dq\" (UniqueName: \"kubernetes.io/projected/3c5b75f9-d22a-45fe-bedb-69d27b3e4e94-kube-api-access-vb6dq\") pod \"maas-api-e2e-mt-8f3975b3-b679f575-8tpqj\" (UID: \"3c5b75f9-d22a-45fe-bedb-69d27b3e4e94\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f3975b3-b679f575-8tpqj"
Sep 15 10:34:41.492059 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:41.491914    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c5b75f9-d22a-45fe-bedb-69d27b3e4e94-maas-api-tls\") pod \"maas-api-e2e-mt-8f3975b3-b679f575-8tpqj\" (UID: \"3c5b75f9-d22a-45fe-bedb-69d27b3e4e94\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f3975b3-b679f575-8tpqj"
Sep 15 10:34:41.555807 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:41.555779    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:41.592771 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:41.592734    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c5b75f9-d22a-45fe-bedb-69d27b3e4e94-metrics-certs\") pod \"maas-api-e2e-mt-8f3975b3-b679f575-8tpqj\" (UID: \"3c5b75f9-d22a-45fe-bedb-69d27b3e4e94\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f3975b3-b679f575-8tpqj"
Sep 15 10:34:41.592950 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:41.592797    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c5b75f9-d22a-45fe-bedb-69d27b3e4e94-openshift-service-ca\") pod \"maas-api-e2e-mt-8f3975b3-b679f575-8tpqj\" (UID: \"3c5b75f9-d22a-45fe-bedb-69d27b3e4e94\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f3975b3-b679f575-8tpqj"
Sep 15 10:34:41.592950 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:41.592825    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vb6dq\" (UniqueName: \"kubernetes.io/projected/3c5b75f9-d22a-45fe-bedb-69d27b3e4e94-kube-api-access-vb6dq\") pod \"maas-api-e2e-mt-8f3975b3-b679f575-8tpqj\" (UID: \"3c5b75f9-d22a-45fe-bedb-69d27b3e4e94\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f3975b3-b679f575-8tpqj"
Sep 15 10:34:41.592950 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:41.592847    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c5b75f9-d22a-45fe-bedb-69d27b3e4e94-maas-api-tls\") pod \"maas-api-e2e-mt-8f3975b3-b679f575-8tpqj\" (UID: \"3c5b75f9-d22a-45fe-bedb-69d27b3e4e94\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f3975b3-b679f575-8tpqj"
Sep 15 10:34:41.593122 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:34:41.593014    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-8f3975b3-serving-cert: secret "maas-api-e2e-mt-8f3975b3-serving-cert" not found
Sep 15 10:34:41.593122 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:34:41.593094    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c5b75f9-d22a-45fe-bedb-69d27b3e4e94-maas-api-tls podName:3c5b75f9-d22a-45fe-bedb-69d27b3e4e94 nodeName:}" failed. No retries permitted until 2026-09-15 10:34:42.093072856 +0000 UTC m=+997.340039827 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3c5b75f9-d22a-45fe-bedb-69d27b3e4e94-maas-api-tls") pod "maas-api-e2e-mt-8f3975b3-b679f575-8tpqj" (UID: "3c5b75f9-d22a-45fe-bedb-69d27b3e4e94") : secret "maas-api-e2e-mt-8f3975b3-serving-cert" not found
Sep 15 10:34:41.593492 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:41.593473    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c5b75f9-d22a-45fe-bedb-69d27b3e4e94-openshift-service-ca\") pod \"maas-api-e2e-mt-8f3975b3-b679f575-8tpqj\" (UID: \"3c5b75f9-d22a-45fe-bedb-69d27b3e4e94\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f3975b3-b679f575-8tpqj"
Sep 15 10:34:41.595247 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:41.595226    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c5b75f9-d22a-45fe-bedb-69d27b3e4e94-metrics-certs\") pod \"maas-api-e2e-mt-8f3975b3-b679f575-8tpqj\" (UID: \"3c5b75f9-d22a-45fe-bedb-69d27b3e4e94\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f3975b3-b679f575-8tpqj"
Sep 15 10:34:41.601136 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:41.601116    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb6dq\" (UniqueName: \"kubernetes.io/projected/3c5b75f9-d22a-45fe-bedb-69d27b3e4e94-kube-api-access-vb6dq\") pod \"maas-api-e2e-mt-8f3975b3-b679f575-8tpqj\" (UID: \"3c5b75f9-d22a-45fe-bedb-69d27b3e4e94\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f3975b3-b679f575-8tpqj"
Sep 15 10:34:41.739306 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:41.739237    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:42.096751 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:42.096658    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c5b75f9-d22a-45fe-bedb-69d27b3e4e94-maas-api-tls\") pod \"maas-api-e2e-mt-8f3975b3-b679f575-8tpqj\" (UID: \"3c5b75f9-d22a-45fe-bedb-69d27b3e4e94\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f3975b3-b679f575-8tpqj"
Sep 15 10:34:42.099076 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:42.099053    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c5b75f9-d22a-45fe-bedb-69d27b3e4e94-maas-api-tls\") pod \"maas-api-e2e-mt-8f3975b3-b679f575-8tpqj\" (UID: \"3c5b75f9-d22a-45fe-bedb-69d27b3e4e94\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f3975b3-b679f575-8tpqj"
Sep 15 10:34:42.337356 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:42.337317    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:42.371495 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:42.371399    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f3975b3-b679f575-8tpqj"
Sep 15 10:34:42.438434 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:42.438386    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:42.492654 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:42.492634    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6"
Sep 15 10:34:42.641850 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:42.641823    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8f3975b3-b679f575-8tpqj"]
Sep 15 10:34:42.642682 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:34:42.642657    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c5b75f9_d22a_45fe_bedb_69d27b3e4e94.slice/crio-91884b9007f63dc69af01bc4fedbfe5e601975a0a62741ca63c63bb8bcfa6ddf WatchSource:0}: Error finding container 91884b9007f63dc69af01bc4fedbfe5e601975a0a62741ca63c63bb8bcfa6ddf: Status 404 returned error can't find the container with id 91884b9007f63dc69af01bc4fedbfe5e601975a0a62741ca63c63bb8bcfa6ddf
Sep 15 10:34:43.526498 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:43.526435    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f3975b3-b679f575-8tpqj" event={"ID":"3c5b75f9-d22a-45fe-bedb-69d27b3e4e94","Type":"ContainerStarted","Data":"c1f0a328078303a3d6d6f91262e40d767b14261d2e15c91bd24d8c1fa93f9b61"}
Sep 15 10:34:43.526904 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:43.526521    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f3975b3-b679f575-8tpqj" event={"ID":"3c5b75f9-d22a-45fe-bedb-69d27b3e4e94","Type":"ContainerStarted","Data":"91884b9007f63dc69af01bc4fedbfe5e601975a0a62741ca63c63bb8bcfa6ddf"}
Sep 15 10:34:43.526904 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:43.526571    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f3975b3-b679f575-8tpqj"
Sep 15 10:34:43.543763 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:43.543717    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f3975b3-b679f575-8tpqj" podStartSLOduration=2.111538801 podStartE2EDuration="2.543703326s" totalImagesPullingTime="432.164525ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:34:43.542701942 +0000 UTC m=+998.789668911" watchObservedRunningTime="2026-09-15 10:34:43.543703326 +0000 UTC m=+998.790670327"
Sep 15 10:34:49.537628 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:49.537600    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f3975b3-b679f575-8tpqj"
Sep 15 10:34:54.907905 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:54.907874    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:56.519073 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:56.519040    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:57.411138 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:57.411105    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:58.019558 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:58.019500    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:58.806214 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:58.806180    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:34:59.405830 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:34:59.405757    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:00.340991 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:00.340962    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:00.441400 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:00.441374    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:01.200120 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:01.200093    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:01.908888 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:01.908858    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:02.622116 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:02.622087    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:02.957770 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:02.957742    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:03.400792 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:03.400757    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:03.491682 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:03.491651    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:03.602814 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:03.602782    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:03.810247 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:03.810165    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:04.541363 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:04.541327    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:05.409122 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:05.409091    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:06.235327 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:06.235295    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:07.185412 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:07.185375    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:09.540465 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:09.540403    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:09.597819 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:09.597786    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:10.750467 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:10.750408    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:11.383461 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:11.383409    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:11.996118 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:11.996087    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:12.686724 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:12.686679    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:13.316739 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:13.316705    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:13.969528 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:13.969496    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:14.611468 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:14.611416    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:20.365874 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:20.365842    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8f3975b3-b679f575-8tpqj"]
Sep 15 10:35:20.366291 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:20.366106    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f3975b3-b679f575-8tpqj" podUID="3c5b75f9-d22a-45fe-bedb-69d27b3e4e94" containerName="maas-api" containerID="cri-o://c1f0a328078303a3d6d6f91262e40d767b14261d2e15c91bd24d8c1fa93f9b61" gracePeriod=30
Sep 15 10:35:20.392368 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:20.392342    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:20.461114 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:20.461076    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:20.566180 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:20.566146    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:20.615157 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:20.615137    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f3975b3-b679f575-8tpqj"
Sep 15 10:35:20.636461 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:20.636422    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:20.743778 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:20.743739    2583 generic.go:356] "Generic (PLEG): container finished" podID="3c5b75f9-d22a-45fe-bedb-69d27b3e4e94" containerID="c1f0a328078303a3d6d6f91262e40d767b14261d2e15c91bd24d8c1fa93f9b61" exitCode=0
Sep 15 10:35:20.744176 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:20.744153    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f3975b3-b679f575-8tpqj" event={"ID":"3c5b75f9-d22a-45fe-bedb-69d27b3e4e94","Type":"ContainerDied","Data":"c1f0a328078303a3d6d6f91262e40d767b14261d2e15c91bd24d8c1fa93f9b61"}
Sep 15 10:35:20.744361 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:20.744345    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f3975b3-b679f575-8tpqj" event={"ID":"3c5b75f9-d22a-45fe-bedb-69d27b3e4e94","Type":"ContainerDied","Data":"91884b9007f63dc69af01bc4fedbfe5e601975a0a62741ca63c63bb8bcfa6ddf"}
Sep 15 10:35:20.744497 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:20.744484    2583 scope.go:117] "RemoveContainer" containerID="c1f0a328078303a3d6d6f91262e40d767b14261d2e15c91bd24d8c1fa93f9b61"
Sep 15 10:35:20.744779 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:20.744761    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f3975b3-b679f575-8tpqj"
Sep 15 10:35:20.750204 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:20.750181    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c5b75f9-d22a-45fe-bedb-69d27b3e4e94-metrics-certs\") pod \"3c5b75f9-d22a-45fe-bedb-69d27b3e4e94\" (UID: \"3c5b75f9-d22a-45fe-bedb-69d27b3e4e94\") "
Sep 15 10:35:20.750323 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:20.750258    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c5b75f9-d22a-45fe-bedb-69d27b3e4e94-openshift-service-ca\") pod \"3c5b75f9-d22a-45fe-bedb-69d27b3e4e94\" (UID: \"3c5b75f9-d22a-45fe-bedb-69d27b3e4e94\") "
Sep 15 10:35:20.750388 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:20.750365    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vb6dq\" (UniqueName: \"kubernetes.io/projected/3c5b75f9-d22a-45fe-bedb-69d27b3e4e94-kube-api-access-vb6dq\") pod \"3c5b75f9-d22a-45fe-bedb-69d27b3e4e94\" (UID: \"3c5b75f9-d22a-45fe-bedb-69d27b3e4e94\") "
Sep 15 10:35:20.750672 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:20.750644    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c5b75f9-d22a-45fe-bedb-69d27b3e4e94-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3c5b75f9-d22a-45fe-bedb-69d27b3e4e94" (UID: "3c5b75f9-d22a-45fe-bedb-69d27b3e4e94"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:35:20.750793 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:20.750726    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c5b75f9-d22a-45fe-bedb-69d27b3e4e94-maas-api-tls\") pod \"3c5b75f9-d22a-45fe-bedb-69d27b3e4e94\" (UID: \"3c5b75f9-d22a-45fe-bedb-69d27b3e4e94\") "
Sep 15 10:35:20.751115 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:20.751093    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c5b75f9-d22a-45fe-bedb-69d27b3e4e94-openshift-service-ca\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:35:20.752398 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:20.752376    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c5b75f9-d22a-45fe-bedb-69d27b3e4e94-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3c5b75f9-d22a-45fe-bedb-69d27b3e4e94" (UID: "3c5b75f9-d22a-45fe-bedb-69d27b3e4e94"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:35:20.752635 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:20.752611    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c5b75f9-d22a-45fe-bedb-69d27b3e4e94-kube-api-access-vb6dq" (OuterVolumeSpecName: "kube-api-access-vb6dq") pod "3c5b75f9-d22a-45fe-bedb-69d27b3e4e94" (UID: "3c5b75f9-d22a-45fe-bedb-69d27b3e4e94"). InnerVolumeSpecName "kube-api-access-vb6dq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:35:20.752957 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:20.752939    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c5b75f9-d22a-45fe-bedb-69d27b3e4e94-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3c5b75f9-d22a-45fe-bedb-69d27b3e4e94" (UID: "3c5b75f9-d22a-45fe-bedb-69d27b3e4e94"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:35:20.757089 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:20.757060    2583 scope.go:117] "RemoveContainer" containerID="c1f0a328078303a3d6d6f91262e40d767b14261d2e15c91bd24d8c1fa93f9b61"
Sep 15 10:35:20.757397 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:35:20.757371    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1f0a328078303a3d6d6f91262e40d767b14261d2e15c91bd24d8c1fa93f9b61\": container with ID starting with c1f0a328078303a3d6d6f91262e40d767b14261d2e15c91bd24d8c1fa93f9b61 not found: ID does not exist" containerID="c1f0a328078303a3d6d6f91262e40d767b14261d2e15c91bd24d8c1fa93f9b61"
Sep 15 10:35:20.757481 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:20.757408    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1f0a328078303a3d6d6f91262e40d767b14261d2e15c91bd24d8c1fa93f9b61"} err="failed to get container status \"c1f0a328078303a3d6d6f91262e40d767b14261d2e15c91bd24d8c1fa93f9b61\": rpc error: code = NotFound desc = could not find container \"c1f0a328078303a3d6d6f91262e40d767b14261d2e15c91bd24d8c1fa93f9b61\": container with ID starting with c1f0a328078303a3d6d6f91262e40d767b14261d2e15c91bd24d8c1fa93f9b61 not found: ID does not exist"
Sep 15 10:35:20.852138 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:20.852105    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c5b75f9-d22a-45fe-bedb-69d27b3e4e94-metrics-certs\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:35:20.852138 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:20.852134    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vb6dq\" (UniqueName: \"kubernetes.io/projected/3c5b75f9-d22a-45fe-bedb-69d27b3e4e94-kube-api-access-vb6dq\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:35:20.852325 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:20.852147    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c5b75f9-d22a-45fe-bedb-69d27b3e4e94-maas-api-tls\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:35:20.979507 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:20.979387    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:21.067987 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:21.067948    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8f3975b3-b679f575-8tpqj"]
Sep 15 10:35:21.072787 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:21.072725    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:21.074722 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:21.074702    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8f3975b3-b679f575-8tpqj"]
Sep 15 10:35:21.300997 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:21.300917    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c5b75f9-d22a-45fe-bedb-69d27b3e4e94" path="/var/lib/kubelet/pods/3c5b75f9-d22a-45fe-bedb-69d27b3e4e94/volumes"
Sep 15 10:35:21.694613 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:21.694573    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:22.393012 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:22.392974    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:23.194620 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:23.194579    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:23.839283 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:23.839249    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:24.477208 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:24.477160    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:24.944243 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:24.944200    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6"]
Sep 15 10:35:24.944680 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:24.944663    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c5b75f9-d22a-45fe-bedb-69d27b3e4e94" containerName="maas-api"
Sep 15 10:35:24.944680 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:24.944680    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c5b75f9-d22a-45fe-bedb-69d27b3e4e94" containerName="maas-api"
Sep 15 10:35:24.944842 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:24.944778    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c5b75f9-d22a-45fe-bedb-69d27b3e4e94" containerName="maas-api"
Sep 15 10:35:24.950174 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:24.950158    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6"
Sep 15 10:35:24.953616 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:24.953595    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-234840\"/\"kube-root-ca.crt\""
Sep 15 10:35:24.954143 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:24.953758    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-234840\"/\"test-model-234840-kserve-self-signed-certs\""
Sep 15 10:35:24.954143 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:24.953995    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-234840\"/\"default-dockercfg-nbbgz\""
Sep 15 10:35:24.954369 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:24.954348    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-234840\"/\"openshift-service-ca.crt\""
Sep 15 10:35:24.957106 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:24.957084    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6"]
Sep 15 10:35:25.089657 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:25.089584    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b44d7\" (UniqueName: \"kubernetes.io/projected/fcedb166-03b2-448c-b397-2d8f44f799ba-kube-api-access-b44d7\") pod \"test-model-234840-kserve-74c775dc8c-xfvx6\" (UID: \"fcedb166-03b2-448c-b397-2d8f44f799ba\") " pod="ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6"
Sep 15 10:35:25.089863 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:25.089679    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcedb166-03b2-448c-b397-2d8f44f799ba-tls-certs\") pod \"test-model-234840-kserve-74c775dc8c-xfvx6\" (UID: \"fcedb166-03b2-448c-b397-2d8f44f799ba\") " pod="ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6"
Sep 15 10:35:25.089863 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:25.089725    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcedb166-03b2-448c-b397-2d8f44f799ba-dshm\") pod \"test-model-234840-kserve-74c775dc8c-xfvx6\" (UID: \"fcedb166-03b2-448c-b397-2d8f44f799ba\") " pod="ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6"
Sep 15 10:35:25.089863 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:25.089765    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcedb166-03b2-448c-b397-2d8f44f799ba-home\") pod \"test-model-234840-kserve-74c775dc8c-xfvx6\" (UID: \"fcedb166-03b2-448c-b397-2d8f44f799ba\") " pod="ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6"
Sep 15 10:35:25.089863 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:25.089787    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcedb166-03b2-448c-b397-2d8f44f799ba-tmp-dir\") pod \"test-model-234840-kserve-74c775dc8c-xfvx6\" (UID: \"fcedb166-03b2-448c-b397-2d8f44f799ba\") " pod="ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6"
Sep 15 10:35:25.090041 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:25.089868    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcedb166-03b2-448c-b397-2d8f44f799ba-model-cache\") pod \"test-model-234840-kserve-74c775dc8c-xfvx6\" (UID: \"fcedb166-03b2-448c-b397-2d8f44f799ba\") " pod="ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6"
Sep 15 10:35:25.120176 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:25.120140    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:25.190760 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:25.190721    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcedb166-03b2-448c-b397-2d8f44f799ba-dshm\") pod \"test-model-234840-kserve-74c775dc8c-xfvx6\" (UID: \"fcedb166-03b2-448c-b397-2d8f44f799ba\") " pod="ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6"
Sep 15 10:35:25.190760 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:25.190762    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcedb166-03b2-448c-b397-2d8f44f799ba-home\") pod \"test-model-234840-kserve-74c775dc8c-xfvx6\" (UID: \"fcedb166-03b2-448c-b397-2d8f44f799ba\") " pod="ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6"
Sep 15 10:35:25.191007 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:25.190787    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcedb166-03b2-448c-b397-2d8f44f799ba-tmp-dir\") pod \"test-model-234840-kserve-74c775dc8c-xfvx6\" (UID: \"fcedb166-03b2-448c-b397-2d8f44f799ba\") " pod="ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6"
Sep 15 10:35:25.191007 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:25.190880    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcedb166-03b2-448c-b397-2d8f44f799ba-model-cache\") pod \"test-model-234840-kserve-74c775dc8c-xfvx6\" (UID: \"fcedb166-03b2-448c-b397-2d8f44f799ba\") " pod="ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6"
Sep 15 10:35:25.191007 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:25.190992    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b44d7\" (UniqueName: \"kubernetes.io/projected/fcedb166-03b2-448c-b397-2d8f44f799ba-kube-api-access-b44d7\") pod \"test-model-234840-kserve-74c775dc8c-xfvx6\" (UID: \"fcedb166-03b2-448c-b397-2d8f44f799ba\") " pod="ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6"
Sep 15 10:35:25.191243 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:25.191029    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcedb166-03b2-448c-b397-2d8f44f799ba-tls-certs\") pod \"test-model-234840-kserve-74c775dc8c-xfvx6\" (UID: \"fcedb166-03b2-448c-b397-2d8f44f799ba\") " pod="ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6"
Sep 15 10:35:25.191243 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:25.191171    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcedb166-03b2-448c-b397-2d8f44f799ba-home\") pod \"test-model-234840-kserve-74c775dc8c-xfvx6\" (UID: \"fcedb166-03b2-448c-b397-2d8f44f799ba\") " pod="ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6"
Sep 15 10:35:25.191362 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:25.191250    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcedb166-03b2-448c-b397-2d8f44f799ba-tmp-dir\") pod \"test-model-234840-kserve-74c775dc8c-xfvx6\" (UID: \"fcedb166-03b2-448c-b397-2d8f44f799ba\") " pod="ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6"
Sep 15 10:35:25.191468 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:25.191425    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcedb166-03b2-448c-b397-2d8f44f799ba-model-cache\") pod \"test-model-234840-kserve-74c775dc8c-xfvx6\" (UID: \"fcedb166-03b2-448c-b397-2d8f44f799ba\") " pod="ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6"
Sep 15 10:35:25.193992 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:25.193966    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcedb166-03b2-448c-b397-2d8f44f799ba-tls-certs\") pod \"test-model-234840-kserve-74c775dc8c-xfvx6\" (UID: \"fcedb166-03b2-448c-b397-2d8f44f799ba\") " pod="ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6"
Sep 15 10:35:25.195124 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:25.195063    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcedb166-03b2-448c-b397-2d8f44f799ba-dshm\") pod \"test-model-234840-kserve-74c775dc8c-xfvx6\" (UID: \"fcedb166-03b2-448c-b397-2d8f44f799ba\") " pod="ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6"
Sep 15 10:35:25.199298 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:25.199274    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b44d7\" (UniqueName: \"kubernetes.io/projected/fcedb166-03b2-448c-b397-2d8f44f799ba-kube-api-access-b44d7\") pod \"test-model-234840-kserve-74c775dc8c-xfvx6\" (UID: \"fcedb166-03b2-448c-b397-2d8f44f799ba\") " pod="ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6"
Sep 15 10:35:25.260644 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:25.260610    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6"
Sep 15 10:35:25.534099 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:25.534075    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6"]
Sep 15 10:35:25.535747 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:35:25.535723    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfcedb166_03b2_448c_b397_2d8f44f799ba.slice/crio-840a675c60e6db962bbc6550bd0285c86cd96e7beec7549e45338b27a1784cca WatchSource:0}: Error finding container 840a675c60e6db962bbc6550bd0285c86cd96e7beec7549e45338b27a1784cca: Status 404 returned error can't find the container with id 840a675c60e6db962bbc6550bd0285c86cd96e7beec7549e45338b27a1784cca
Sep 15 10:35:25.782942 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:25.782842    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6" event={"ID":"fcedb166-03b2-448c-b397-2d8f44f799ba","Type":"ContainerStarted","Data":"f4b8d9f135d230db4995920620855b58090c46c4ec70dda6463527d7d069b00b"}
Sep 15 10:35:25.782942 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:25.782913    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6" event={"ID":"fcedb166-03b2-448c-b397-2d8f44f799ba","Type":"ContainerStarted","Data":"840a675c60e6db962bbc6550bd0285c86cd96e7beec7549e45338b27a1784cca"}
Sep 15 10:35:25.800795 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:25.800749    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6" podStartSLOduration=1.800733028 podStartE2EDuration="1.800733028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:35:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:35:25.798885077 +0000 UTC m=+1041.045852046" watchObservedRunningTime="2026-09-15 10:35:25.800733028 +0000 UTC m=+1041.047699998"
Sep 15 10:35:26.552347 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:26.552313    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:26.654098 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:26.654068    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:26.816067 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:26.815978    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:27.159983 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:27.159952    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:27.257337 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:27.257299    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:27.367355 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:27.367320    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:33.645620 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:33.645573    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:33.709388 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:33.709357    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:33.788137 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:33.788104    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:33.979607 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:33.979522    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:35.261250 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:35.261217    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6"
Sep 15 10:35:35.261656 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:35.261268    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6"
Sep 15 10:35:35.273615 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:35.273592    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6"
Sep 15 10:35:35.853595 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:35.853567    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6"
Sep 15 10:35:39.856350 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:39.856321    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:39.957658 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:39.957628    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:40.771539 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:40.771506    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:41.047048 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.046966    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"]
Sep 15 10:35:41.050986 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.050956    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:35:41.053219 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.053195    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-30651f-openshift-default-dockercfg-r2bsj\""
Sep 15 10:35:41.060660 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.060633    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"]
Sep 15 10:35:41.080306 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.080286    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:41.145922 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.145899    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e854721-2229-44c8-b91e-2ab76dd1a162-istiod-ca-cert\") pod \"e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") " pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:35:41.146031 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.145929    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e854721-2229-44c8-b91e-2ab76dd1a162-istio-data\") pod \"e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") " pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:35:41.146031 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.145949    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc99w\" (UniqueName: \"kubernetes.io/projected/5e854721-2229-44c8-b91e-2ab76dd1a162-kube-api-access-cc99w\") pod \"e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") " pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:35:41.146031 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.145976    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e854721-2229-44c8-b91e-2ab76dd1a162-credential-socket\") pod \"e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") " pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:35:41.146148 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.146031    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e854721-2229-44c8-b91e-2ab76dd1a162-istio-token\") pod \"e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") " pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:35:41.146148 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.146065    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e854721-2229-44c8-b91e-2ab76dd1a162-istio-envoy\") pod \"e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") " pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:35:41.146148 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.146108    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e854721-2229-44c8-b91e-2ab76dd1a162-workload-certs\") pod \"e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") " pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:35:41.146241 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.146198    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e854721-2229-44c8-b91e-2ab76dd1a162-workload-socket\") pod \"e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") " pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:35:41.146241 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.146214    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e854721-2229-44c8-b91e-2ab76dd1a162-istio-podinfo\") pod \"e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") " pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:35:41.247018 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.246988    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e854721-2229-44c8-b91e-2ab76dd1a162-istio-envoy\") pod \"e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") " pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:35:41.247200 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.247050    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e854721-2229-44c8-b91e-2ab76dd1a162-workload-certs\") pod \"e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") " pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:35:41.247200 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.247125    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e854721-2229-44c8-b91e-2ab76dd1a162-workload-socket\") pod \"e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") " pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:35:41.247200 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.247151    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e854721-2229-44c8-b91e-2ab76dd1a162-istio-podinfo\") pod \"e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") " pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:35:41.247355 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.247206    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e854721-2229-44c8-b91e-2ab76dd1a162-istiod-ca-cert\") pod \"e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") " pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:35:41.247355 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.247237    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e854721-2229-44c8-b91e-2ab76dd1a162-istio-data\") pod \"e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") " pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:35:41.247355 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.247266    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc99w\" (UniqueName: \"kubernetes.io/projected/5e854721-2229-44c8-b91e-2ab76dd1a162-kube-api-access-cc99w\") pod \"e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") " pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:35:41.247355 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.247304    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e854721-2229-44c8-b91e-2ab76dd1a162-credential-socket\") pod \"e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") " pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:35:41.247355 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.247333    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e854721-2229-44c8-b91e-2ab76dd1a162-istio-token\") pod \"e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") " pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:35:41.247649 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.247586    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e854721-2229-44c8-b91e-2ab76dd1a162-workload-socket\") pod \"e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") " pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:35:41.247710 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.247686    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e854721-2229-44c8-b91e-2ab76dd1a162-istio-data\") pod \"e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") " pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:35:41.247937 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.247915    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e854721-2229-44c8-b91e-2ab76dd1a162-credential-socket\") pod \"e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") " pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:35:41.248019 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.247976    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e854721-2229-44c8-b91e-2ab76dd1a162-workload-certs\") pod \"e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") " pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:35:41.248075 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.248053    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e854721-2229-44c8-b91e-2ab76dd1a162-istiod-ca-cert\") pod \"e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") " pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:35:41.249866 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.249842    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e854721-2229-44c8-b91e-2ab76dd1a162-istio-envoy\") pod \"e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") " pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:35:41.250093 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.250076    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e854721-2229-44c8-b91e-2ab76dd1a162-istio-podinfo\") pod \"e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") " pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:35:41.255119 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.255098    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e854721-2229-44c8-b91e-2ab76dd1a162-istio-token\") pod \"e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") " pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:35:41.255209 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.255178    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc99w\" (UniqueName: \"kubernetes.io/projected/5e854721-2229-44c8-b91e-2ab76dd1a162-kube-api-access-cc99w\") pod \"e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") " pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:35:41.279200 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.279175    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:41.364813 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.364734    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:35:41.639604 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.639577    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"]
Sep 15 10:35:41.640995 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:35:41.640971    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e854721_2229_44c8_b91e_2ab76dd1a162.slice/crio-7e4f7a347a42e313e37f31c6953102c3d5b09ea6e30abf18a657287a8c7731a3 WatchSource:0}: Error finding container 7e4f7a347a42e313e37f31c6953102c3d5b09ea6e30abf18a657287a8c7731a3: Status 404 returned error can't find the container with id 7e4f7a347a42e313e37f31c6953102c3d5b09ea6e30abf18a657287a8c7731a3
Sep 15 10:35:41.643642 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.643612    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:35:41.643700 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.643674    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:35:41.643738 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.643701    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:35:41.884209 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.884172    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk" event={"ID":"5e854721-2229-44c8-b91e-2ab76dd1a162","Type":"ContainerStarted","Data":"ca4dc2de215d2a79a4371d21d7aab8643722161d4440b8edf1531a715a191dde"}
Sep 15 10:35:41.884419 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.884256    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk" event={"ID":"5e854721-2229-44c8-b91e-2ab76dd1a162","Type":"ContainerStarted","Data":"7e4f7a347a42e313e37f31c6953102c3d5b09ea6e30abf18a657287a8c7731a3"}
Sep 15 10:35:41.906755 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:41.906632    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk" podStartSLOduration=0.906609151 podStartE2EDuration="906.609151ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:35:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:35:41.901926532 +0000 UTC m=+1057.148893503" watchObservedRunningTime="2026-09-15 10:35:41.906609151 +0000 UTC m=+1057.153576123"
Sep 15 10:35:42.294044 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:42.293965    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:42.365857 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:42.365821    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:35:42.370830 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:42.370802    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:35:42.890606 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:42.890576    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:35:42.891523 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:42.891502    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:35:43.095480 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:43.095431    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:43.673595 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:43.673560    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:43.779309 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:43.779275    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:43.874917 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:43.874882    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:44.697869 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:44.697838    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:45.451590 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:45.451558    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:46.475506 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:46.475479    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:47.191785 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:47.191751    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:47.960598 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:47.960564    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:48.061437 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:48.061404    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:48.335310 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:48.335220    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:48.473073 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:48.473046    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:48.579816 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:48.579780    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:35:54.342374 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:35:54.342344    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:05.447870 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:05.447836    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:08.992882 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:08.992847    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:10.145563 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:10.145531    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:10.786849 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:10.786816    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:11.490323 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:11.490285    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:12.090180 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:12.090144    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:12.971871 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:12.971842    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:13.567154 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:13.567116    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:13.808996 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:13.808958    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:14.120067 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:14.120034    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:14.273838 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:14.273808    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:14.419975 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:14.419941    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:14.620817 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:14.620785    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:16.488889 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:16.488852    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:17.162118 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:17.162082    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:17.266372 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:17.266337    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:17.341292 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:17.341257    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:17.960195 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:17.960160    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:18.782250 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:18.782220    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:19.116483 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:19.116382    2583 generic.go:356] "Generic (PLEG): container finished" podID="5e854721-2229-44c8-b91e-2ab76dd1a162" containerID="ca4dc2de215d2a79a4371d21d7aab8643722161d4440b8edf1531a715a191dde" exitCode=0
Sep 15 10:36:19.116483 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:19.116457    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk" event={"ID":"5e854721-2229-44c8-b91e-2ab76dd1a162","Type":"ContainerDied","Data":"ca4dc2de215d2a79a4371d21d7aab8643722161d4440b8edf1531a715a191dde"}
Sep 15 10:36:19.116911 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:19.116895    2583 scope.go:117] "RemoveContainer" containerID="ca4dc2de215d2a79a4371d21d7aab8643722161d4440b8edf1531a715a191dde"
Sep 15 10:36:19.118108 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:19.118078    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:36:19.118198 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:19.118151    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:36:19.118198 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:19.118183    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:36:19.468355 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:19.468317    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:20.124249 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:20.124215    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk" event={"ID":"5e854721-2229-44c8-b91e-2ab76dd1a162","Type":"ContainerStarted","Data":"bd2f8fb841046c2c9419e0f8bbdf022c27aeb4c8fa57399acc851f96d9055baf"}
Sep 15 10:36:20.252200 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:20.252165    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:20.365391 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:20.365356    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:36:20.365601 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:20.365575    2583 patch_prober.go:28] interesting pod/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 15 10:36:20.365681 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:20.365630    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk" podUID="5e854721-2229-44c8-b91e-2ab76dd1a162" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 15 10:36:20.897132 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:20.897098    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:21.366126 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:21.366033    2583 patch_prober.go:28] interesting pod/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 15 10:36:21.366740 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:21.366703    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk" podUID="5e854721-2229-44c8-b91e-2ab76dd1a162" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 15 10:36:21.462662 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:21.462623    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87"]
Sep 15 10:36:21.480518 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:21.480481    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87"]
Sep 15 10:36:21.480674 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:21.480602    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87"
Sep 15 10:36:21.483135 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:21.483110    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-ecf56f-serving-cert\""
Sep 15 10:36:21.486756 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:21.486730    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:21.565974 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:21.565944    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:21.579636 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:21.579600    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6e82451-e1e6-4a82-be75-dde9c4c906b8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87\" (UID: \"d6e82451-e1e6-4a82-be75-dde9c4c906b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87"
Sep 15 10:36:21.579827 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:21.579704    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdpss\" (UniqueName: \"kubernetes.io/projected/d6e82451-e1e6-4a82-be75-dde9c4c906b8-kube-api-access-kdpss\") pod \"maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87\" (UID: \"d6e82451-e1e6-4a82-be75-dde9c4c906b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87"
Sep 15 10:36:21.579827 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:21.579758    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6e82451-e1e6-4a82-be75-dde9c4c906b8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87\" (UID: \"d6e82451-e1e6-4a82-be75-dde9c4c906b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87"
Sep 15 10:36:21.579827 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:21.579782    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6e82451-e1e6-4a82-be75-dde9c4c906b8-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87\" (UID: \"d6e82451-e1e6-4a82-be75-dde9c4c906b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87"
Sep 15 10:36:21.680717 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:21.680687    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6e82451-e1e6-4a82-be75-dde9c4c906b8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87\" (UID: \"d6e82451-e1e6-4a82-be75-dde9c4c906b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87"
Sep 15 10:36:21.680896 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:21.680753    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdpss\" (UniqueName: \"kubernetes.io/projected/d6e82451-e1e6-4a82-be75-dde9c4c906b8-kube-api-access-kdpss\") pod \"maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87\" (UID: \"d6e82451-e1e6-4a82-be75-dde9c4c906b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87"
Sep 15 10:36:21.680896 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:21.680792    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6e82451-e1e6-4a82-be75-dde9c4c906b8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87\" (UID: \"d6e82451-e1e6-4a82-be75-dde9c4c906b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87"
Sep 15 10:36:21.680896 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:21.680807    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6e82451-e1e6-4a82-be75-dde9c4c906b8-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87\" (UID: \"d6e82451-e1e6-4a82-be75-dde9c4c906b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87"
Sep 15 10:36:21.681426 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:21.681386    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6e82451-e1e6-4a82-be75-dde9c4c906b8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87\" (UID: \"d6e82451-e1e6-4a82-be75-dde9c4c906b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87"
Sep 15 10:36:21.683300 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:21.683278    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6e82451-e1e6-4a82-be75-dde9c4c906b8-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87\" (UID: \"d6e82451-e1e6-4a82-be75-dde9c4c906b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87"
Sep 15 10:36:21.683400 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:21.683340    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6e82451-e1e6-4a82-be75-dde9c4c906b8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87\" (UID: \"d6e82451-e1e6-4a82-be75-dde9c4c906b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87"
Sep 15 10:36:21.688416 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:21.688396    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdpss\" (UniqueName: \"kubernetes.io/projected/d6e82451-e1e6-4a82-be75-dde9c4c906b8-kube-api-access-kdpss\") pod \"maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87\" (UID: \"d6e82451-e1e6-4a82-be75-dde9c4c906b8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87"
Sep 15 10:36:21.699090 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:21.699063    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:21.793644 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:21.793608    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87"
Sep 15 10:36:21.904646 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:21.904612    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:22.005144 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:22.005048    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:22.078089 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:22.078059    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87"]
Sep 15 10:36:22.079661 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:36:22.079635    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6e82451_e1e6_4a82_be75_dde9c4c906b8.slice/crio-329c169345c74203f87c17d76bff9e939667d001d80498d9985880cb66dedaf7 WatchSource:0}: Error finding container 329c169345c74203f87c17d76bff9e939667d001d80498d9985880cb66dedaf7: Status 404 returned error can't find the container with id 329c169345c74203f87c17d76bff9e939667d001d80498d9985880cb66dedaf7
Sep 15 10:36:22.139181 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:22.139138    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87" event={"ID":"d6e82451-e1e6-4a82-be75-dde9c4c906b8","Type":"ContainerStarted","Data":"329c169345c74203f87c17d76bff9e939667d001d80498d9985880cb66dedaf7"}
Sep 15 10:36:22.365403 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:22.365272    2583 patch_prober.go:28] interesting pod/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 15 10:36:22.365403 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:22.365350    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk" podUID="5e854721-2229-44c8-b91e-2ab76dd1a162" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 15 10:36:23.150829 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:23.150784    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87" event={"ID":"d6e82451-e1e6-4a82-be75-dde9c4c906b8","Type":"ContainerStarted","Data":"6f4e50fbd0c22d2fa3a9f27cf492904d198b7fa70a40846c4e42c64bf8b38404"}
Sep 15 10:36:23.151690 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:23.151665    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87"
Sep 15 10:36:23.169471 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:23.169403    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87" podStartSLOduration=1.6738588559999998 podStartE2EDuration="2.169364654s" totalImagesPullingTime="495.505798ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:36:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:36:23.168505189 +0000 UTC m=+1098.415472158" watchObservedRunningTime="2026-09-15 10:36:23.169364654 +0000 UTC m=+1098.416331624"
Sep 15 10:36:23.366006 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:23.365961    2583 patch_prober.go:28] interesting pod/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 15 10:36:23.366209 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:23.366029    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk" podUID="5e854721-2229-44c8-b91e-2ab76dd1a162" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 15 10:36:24.365477 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:24.365416    2583 patch_prober.go:28] interesting pod/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 15 10:36:24.366005 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:24.365508    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk" podUID="5e854721-2229-44c8-b91e-2ab76dd1a162" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 15 10:36:25.365536 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:25.365494    2583 patch_prober.go:28] interesting pod/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 15 10:36:25.366080 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:25.365560    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk" podUID="5e854721-2229-44c8-b91e-2ab76dd1a162" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 15 10:36:26.369647 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:26.369615    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:36:26.370112 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:26.369914    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:36:26.370496 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:26.370478    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:36:28.075313 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:28.075282    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:28.764628 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:28.764595    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:29.474535 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:29.474503    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:30.089331 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:30.089290    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:30.189484 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:30.189435    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87"
Sep 15 10:36:30.784224 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:30.784187    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:31.489636 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:31.489603    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:32.150512 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:32.150474    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:32.962736 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:32.962669    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:33.685092 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:33.685061    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:34.385884 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:34.385851    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:35.078819 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:35.078788    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:35.671459 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:35.671417    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:36.382133 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:36.382100    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:36.935845 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:36.935813    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:37.037980 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:37.037942    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:37.573811 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:37.573778    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:38.304622 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:38.304588    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:38.812791 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:38.812754    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6"]
Sep 15 10:36:38.813252 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:38.813035    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6" podUID="c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1" containerName="maas-api" containerID="cri-o://06eb528907cb8dbc785a5c39a8c67cbc49c23eb9c2032f9d7578a4bf826b5406" gracePeriod=30
Sep 15 10:36:38.838686 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:38.838662    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:38.938616 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:38.938564    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:39.038061 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:39.038030    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:39.069816 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:39.069759    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6"
Sep 15 10:36:39.100638 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:39.100608    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:39.159562 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:39.159532    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1-openshift-service-ca\") pod \"c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1\" (UID: \"c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1\") "
Sep 15 10:36:39.159716 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:39.159585    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1-metrics-certs\") pod \"c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1\" (UID: \"c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1\") "
Sep 15 10:36:39.159716 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:39.159646    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1-maas-api-tls\") pod \"c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1\" (UID: \"c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1\") "
Sep 15 10:36:39.159716 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:39.159682    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d77ck\" (UniqueName: \"kubernetes.io/projected/c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1-kube-api-access-d77ck\") pod \"c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1\" (UID: \"c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1\") "
Sep 15 10:36:39.160040 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:39.160006    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1" (UID: "c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:36:39.160187 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:39.160168    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1-openshift-service-ca\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:36:39.161687 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:39.161667    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1" (UID: "c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:36:39.162088 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:39.162066    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1" (UID: "c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:36:39.162138 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:39.162085    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1-kube-api-access-d77ck" (OuterVolumeSpecName: "kube-api-access-d77ck") pod "c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1" (UID: "c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1"). InnerVolumeSpecName "kube-api-access-d77ck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:36:39.261749 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:39.261718    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1-metrics-certs\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:36:39.261749 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:39.261746    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1-maas-api-tls\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:36:39.261995 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:39.261758    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d77ck\" (UniqueName: \"kubernetes.io/projected/c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1-kube-api-access-d77ck\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:36:39.271754 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:39.271727    2583 generic.go:356] "Generic (PLEG): container finished" podID="c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1" containerID="06eb528907cb8dbc785a5c39a8c67cbc49c23eb9c2032f9d7578a4bf826b5406" exitCode=0
Sep 15 10:36:39.271886 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:39.271772    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6"
Sep 15 10:36:39.271886 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:39.271795    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6" event={"ID":"c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1","Type":"ContainerDied","Data":"06eb528907cb8dbc785a5c39a8c67cbc49c23eb9c2032f9d7578a4bf826b5406"}
Sep 15 10:36:39.271886 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:39.271833    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6" event={"ID":"c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1","Type":"ContainerDied","Data":"2be5aab308b5f9c6cf152a5860618c9c41557b55e43b9773d2eb836c80f433f7"}
Sep 15 10:36:39.271886 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:39.271854    2583 scope.go:117] "RemoveContainer" containerID="06eb528907cb8dbc785a5c39a8c67cbc49c23eb9c2032f9d7578a4bf826b5406"
Sep 15 10:36:39.281627 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:39.281610    2583 scope.go:117] "RemoveContainer" containerID="06eb528907cb8dbc785a5c39a8c67cbc49c23eb9c2032f9d7578a4bf826b5406"
Sep 15 10:36:39.281890 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:36:39.281870    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06eb528907cb8dbc785a5c39a8c67cbc49c23eb9c2032f9d7578a4bf826b5406\": container with ID starting with 06eb528907cb8dbc785a5c39a8c67cbc49c23eb9c2032f9d7578a4bf826b5406 not found: ID does not exist" containerID="06eb528907cb8dbc785a5c39a8c67cbc49c23eb9c2032f9d7578a4bf826b5406"
Sep 15 10:36:39.281943 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:39.281902    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06eb528907cb8dbc785a5c39a8c67cbc49c23eb9c2032f9d7578a4bf826b5406"} err="failed to get container status \"06eb528907cb8dbc785a5c39a8c67cbc49c23eb9c2032f9d7578a4bf826b5406\": rpc error: code = NotFound desc = could not find container \"06eb528907cb8dbc785a5c39a8c67cbc49c23eb9c2032f9d7578a4bf826b5406\": container with ID starting with 06eb528907cb8dbc785a5c39a8c67cbc49c23eb9c2032f9d7578a4bf826b5406 not found: ID does not exist"
Sep 15 10:36:39.302866 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:39.302844    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6"]
Sep 15 10:36:39.304421 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:39.304403    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-aeb4e3-84dbd6bf5c-kc7v6"]
Sep 15 10:36:39.355332 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:39.355261    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:39.471322 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:39.471293    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:40.173927 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:40.173891    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:40.925040 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:40.925000    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:41.303675 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:41.302063    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1" path="/var/lib/kubelet/pods/c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1/volumes"
Sep 15 10:36:41.796636 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:41.796603    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:42.700166 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:42.700137    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:43.292350 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:43.292305    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:44.008368 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:44.008337    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:44.440333 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:44.440303    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:44.707126 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:44.707012    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:48.026255 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.026201    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:48.281505 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.281411    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"]
Sep 15 10:36:48.281941 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.281924    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1" containerName="maas-api"
Sep 15 10:36:48.282002 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.281944    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1" containerName="maas-api"
Sep 15 10:36:48.282042 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.282010    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6e0d2bc-a719-482b-98c2-cc7c1c9b41d1" containerName="maas-api"
Sep 15 10:36:48.286474 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.286453    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:36:48.288829 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.288809    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-6111ecc0-openshift-default-dockercfg-qk7l9\""
Sep 15 10:36:48.298158 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.298135    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"]
Sep 15 10:36:48.314037 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.314006    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:48.351090 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.351050    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-istio-data\") pod \"e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") " pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:36:48.351230 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.351112    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-workload-socket\") pod \"e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") " pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:36:48.351230 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.351154    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffhs8\" (UniqueName: \"kubernetes.io/projected/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-kube-api-access-ffhs8\") pod \"e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") " pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:36:48.351326 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.351249    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-workload-certs\") pod \"e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") " pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:36:48.351326 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.351273    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-istio-token\") pod \"e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") " pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:36:48.351326 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.351291    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-istio-podinfo\") pod \"e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") " pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:36:48.351504 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.351339    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-credential-socket\") pod \"e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") " pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:36:48.351504 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.351365    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-istiod-ca-cert\") pod \"e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") " pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:36:48.351504 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.351431    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-istio-envoy\") pod \"e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") " pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:36:48.430452 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.430421    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:48.452740 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.452703    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-workload-certs\") pod \"e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") " pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:36:48.452740 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.452738    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-istio-token\") pod \"e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") " pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:36:48.452935 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.452755    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-istio-podinfo\") pod \"e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") " pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:36:48.452935 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.452786    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-credential-socket\") pod \"e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") " pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:36:48.452935 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.452807    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-istiod-ca-cert\") pod \"e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") " pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:36:48.452935 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.452827    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-istio-envoy\") pod \"e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") " pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:36:48.452935 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.452873    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-istio-data\") pod \"e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") " pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:36:48.452935 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.452910    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-workload-socket\") pod \"e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") " pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:36:48.453225 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.453175    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-workload-certs\") pod \"e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") " pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:36:48.453284 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.453260    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffhs8\" (UniqueName: \"kubernetes.io/projected/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-kube-api-access-ffhs8\") pod \"e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") " pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:36:48.453379 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.453354    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-workload-socket\") pod \"e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") " pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:36:48.453433 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.453412    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-credential-socket\") pod \"e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") " pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:36:48.453768 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.453738    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-istiod-ca-cert\") pod \"e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") " pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:36:48.453849 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.453814    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-istio-data\") pod \"e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") " pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:36:48.455236 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.455215    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-istio-envoy\") pod \"e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") " pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:36:48.455501 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.455485    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-istio-podinfo\") pod \"e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") " pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:36:48.463311 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.463290    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-istio-token\") pod \"e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") " pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:36:48.463418 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.463294    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffhs8\" (UniqueName: \"kubernetes.io/projected/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-kube-api-access-ffhs8\") pod \"e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") " pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:36:48.598813 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.598712    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:36:48.872310 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.872281    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"]
Sep 15 10:36:48.873188 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:36:48.873165    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac3cc4d8_ad26_4bfa_8690_dad507e6e055.slice/crio-99055d120761b8bd884dc758db8890d7525a568dddf08640a28951c2b4000445 WatchSource:0}: Error finding container 99055d120761b8bd884dc758db8890d7525a568dddf08640a28951c2b4000445: Status 404 returned error can't find the container with id 99055d120761b8bd884dc758db8890d7525a568dddf08640a28951c2b4000445
Sep 15 10:36:48.875592 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.875561    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:36:48.875671 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.875625    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:36:48.875671 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:48.875658    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:36:49.332862 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:49.332824    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw" event={"ID":"ac3cc4d8-ad26-4bfa-8690-dad507e6e055","Type":"ContainerStarted","Data":"b11d7ac7ec2b3856c2941bdffb010bbf28295187a85e5c9e1cc021cb63e0c9a1"}
Sep 15 10:36:49.333264 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:49.332882    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw" event={"ID":"ac3cc4d8-ad26-4bfa-8690-dad507e6e055","Type":"ContainerStarted","Data":"99055d120761b8bd884dc758db8890d7525a568dddf08640a28951c2b4000445"}
Sep 15 10:36:49.352424 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:49.352380    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw" podStartSLOduration=1.352363667 podStartE2EDuration="1.352363667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:36:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:36:49.351709607 +0000 UTC m=+1124.598676581" watchObservedRunningTime="2026-09-15 10:36:49.352363667 +0000 UTC m=+1124.599330637"
Sep 15 10:36:49.598976 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:49.598896    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:36:49.603980 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:49.603949    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:36:50.338961 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:50.338926    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:36:50.339846 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:50.339826    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:36:50.380515 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:50.380487    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:51.131925 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:51.131894    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:51.231725 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:51.231694    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:51.338240 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:51.338203    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:52.070027 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:52.069998    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:52.879246 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:52.879214    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:54.026618 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:54.026584    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:54.409102 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:54.409073    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:55.110132 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:55.110103    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:55.778997 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:55.778958    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6111ecc0-5985b5684-lxz65"]
Sep 15 10:36:55.783498 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:55.783431    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6111ecc0-5985b5684-lxz65"
Sep 15 10:36:55.785697 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:55.785674    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-6111ecc0-serving-cert\""
Sep 15 10:36:55.790425 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:55.790403    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6111ecc0-5985b5684-lxz65"]
Sep 15 10:36:55.809325 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:55.809296    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:55.906245 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:55.906213    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:55.932740 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:55.932702    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tql6b\" (UniqueName: \"kubernetes.io/projected/58e0a2c3-78bf-41e6-85ea-9101d6de94e1-kube-api-access-tql6b\") pod \"maas-api-e2e-mt-6111ecc0-5985b5684-lxz65\" (UID: \"58e0a2c3-78bf-41e6-85ea-9101d6de94e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6111ecc0-5985b5684-lxz65"
Sep 15 10:36:55.932939 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:55.932762    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58e0a2c3-78bf-41e6-85ea-9101d6de94e1-openshift-service-ca\") pod \"maas-api-e2e-mt-6111ecc0-5985b5684-lxz65\" (UID: \"58e0a2c3-78bf-41e6-85ea-9101d6de94e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6111ecc0-5985b5684-lxz65"
Sep 15 10:36:55.932939 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:55.932859    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58e0a2c3-78bf-41e6-85ea-9101d6de94e1-metrics-certs\") pod \"maas-api-e2e-mt-6111ecc0-5985b5684-lxz65\" (UID: \"58e0a2c3-78bf-41e6-85ea-9101d6de94e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6111ecc0-5985b5684-lxz65"
Sep 15 10:36:55.933069 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:55.932969    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58e0a2c3-78bf-41e6-85ea-9101d6de94e1-maas-api-tls\") pod \"maas-api-e2e-mt-6111ecc0-5985b5684-lxz65\" (UID: \"58e0a2c3-78bf-41e6-85ea-9101d6de94e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6111ecc0-5985b5684-lxz65"
Sep 15 10:36:56.033676 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:56.033579    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tql6b\" (UniqueName: \"kubernetes.io/projected/58e0a2c3-78bf-41e6-85ea-9101d6de94e1-kube-api-access-tql6b\") pod \"maas-api-e2e-mt-6111ecc0-5985b5684-lxz65\" (UID: \"58e0a2c3-78bf-41e6-85ea-9101d6de94e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6111ecc0-5985b5684-lxz65"
Sep 15 10:36:56.033676 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:56.033629    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58e0a2c3-78bf-41e6-85ea-9101d6de94e1-openshift-service-ca\") pod \"maas-api-e2e-mt-6111ecc0-5985b5684-lxz65\" (UID: \"58e0a2c3-78bf-41e6-85ea-9101d6de94e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6111ecc0-5985b5684-lxz65"
Sep 15 10:36:56.033676 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:56.033662    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58e0a2c3-78bf-41e6-85ea-9101d6de94e1-metrics-certs\") pod \"maas-api-e2e-mt-6111ecc0-5985b5684-lxz65\" (UID: \"58e0a2c3-78bf-41e6-85ea-9101d6de94e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6111ecc0-5985b5684-lxz65"
Sep 15 10:36:56.033891 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:56.033727    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58e0a2c3-78bf-41e6-85ea-9101d6de94e1-maas-api-tls\") pod \"maas-api-e2e-mt-6111ecc0-5985b5684-lxz65\" (UID: \"58e0a2c3-78bf-41e6-85ea-9101d6de94e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6111ecc0-5985b5684-lxz65"
Sep 15 10:36:56.033891 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:36:56.033830    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-6111ecc0-serving-cert: secret "maas-api-e2e-mt-6111ecc0-serving-cert" not found
Sep 15 10:36:56.033957 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:36:56.033901    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58e0a2c3-78bf-41e6-85ea-9101d6de94e1-maas-api-tls podName:58e0a2c3-78bf-41e6-85ea-9101d6de94e1 nodeName:}" failed. No retries permitted until 2026-09-15 10:36:56.533883411 +0000 UTC m=+1131.780850363 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/58e0a2c3-78bf-41e6-85ea-9101d6de94e1-maas-api-tls") pod "maas-api-e2e-mt-6111ecc0-5985b5684-lxz65" (UID: "58e0a2c3-78bf-41e6-85ea-9101d6de94e1") : secret "maas-api-e2e-mt-6111ecc0-serving-cert" not found
Sep 15 10:36:56.034333 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:56.034309    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58e0a2c3-78bf-41e6-85ea-9101d6de94e1-openshift-service-ca\") pod \"maas-api-e2e-mt-6111ecc0-5985b5684-lxz65\" (UID: \"58e0a2c3-78bf-41e6-85ea-9101d6de94e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6111ecc0-5985b5684-lxz65"
Sep 15 10:36:56.036249 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:56.036225    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58e0a2c3-78bf-41e6-85ea-9101d6de94e1-metrics-certs\") pod \"maas-api-e2e-mt-6111ecc0-5985b5684-lxz65\" (UID: \"58e0a2c3-78bf-41e6-85ea-9101d6de94e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6111ecc0-5985b5684-lxz65"
Sep 15 10:36:56.043542 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:56.043519    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tql6b\" (UniqueName: \"kubernetes.io/projected/58e0a2c3-78bf-41e6-85ea-9101d6de94e1-kube-api-access-tql6b\") pod \"maas-api-e2e-mt-6111ecc0-5985b5684-lxz65\" (UID: \"58e0a2c3-78bf-41e6-85ea-9101d6de94e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6111ecc0-5985b5684-lxz65"
Sep 15 10:36:56.287457 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:56.287360    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:56.394014 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:56.393976    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:56.459353 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:56.459321    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:36:56.539484 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:56.539361    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58e0a2c3-78bf-41e6-85ea-9101d6de94e1-maas-api-tls\") pod \"maas-api-e2e-mt-6111ecc0-5985b5684-lxz65\" (UID: \"58e0a2c3-78bf-41e6-85ea-9101d6de94e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6111ecc0-5985b5684-lxz65"
Sep 15 10:36:56.543022 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:56.542991    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58e0a2c3-78bf-41e6-85ea-9101d6de94e1-maas-api-tls\") pod \"maas-api-e2e-mt-6111ecc0-5985b5684-lxz65\" (UID: \"58e0a2c3-78bf-41e6-85ea-9101d6de94e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6111ecc0-5985b5684-lxz65"
Sep 15 10:36:56.696494 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:56.696412    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6111ecc0-5985b5684-lxz65"
Sep 15 10:36:56.967372 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:56.967344    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6111ecc0-5985b5684-lxz65"]
Sep 15 10:36:56.968459 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:36:56.968412    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58e0a2c3_78bf_41e6_85ea_9101d6de94e1.slice/crio-388038bb0f6cad837a1364ba11b0f4a5261f6161ca11476db7b7c7033d59ea56 WatchSource:0}: Error finding container 388038bb0f6cad837a1364ba11b0f4a5261f6161ca11476db7b7c7033d59ea56: Status 404 returned error can't find the container with id 388038bb0f6cad837a1364ba11b0f4a5261f6161ca11476db7b7c7033d59ea56
Sep 15 10:36:57.383104 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:57.383015    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6111ecc0-5985b5684-lxz65" event={"ID":"58e0a2c3-78bf-41e6-85ea-9101d6de94e1","Type":"ContainerStarted","Data":"388038bb0f6cad837a1364ba11b0f4a5261f6161ca11476db7b7c7033d59ea56"}
Sep 15 10:36:58.390714 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:58.390681    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6111ecc0-5985b5684-lxz65" event={"ID":"58e0a2c3-78bf-41e6-85ea-9101d6de94e1","Type":"ContainerStarted","Data":"e1b41bf4382614da369424e80fa83dc54d2f6b4517e1f226da2810aef2a2dfcb"}
Sep 15 10:36:58.391109 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:58.390743    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6111ecc0-5985b5684-lxz65"
Sep 15 10:36:58.410046 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:36:58.409995    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6111ecc0-5985b5684-lxz65" podStartSLOduration=2.597848207 podStartE2EDuration="3.409980142s" totalImagesPullingTime="812.131935ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:36:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:36:58.407109396 +0000 UTC m=+1133.654076366" watchObservedRunningTime="2026-09-15 10:36:58.409980142 +0000 UTC m=+1133.656947175"
Sep 15 10:37:01.809558 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:01.809531    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:37:04.402291 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:04.402261    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6111ecc0-5985b5684-lxz65"
Sep 15 10:37:15.522260 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:15.522226    2583 generic.go:356] "Generic (PLEG): container finished" podID="ac3cc4d8-ad26-4bfa-8690-dad507e6e055" containerID="b11d7ac7ec2b3856c2941bdffb010bbf28295187a85e5c9e1cc021cb63e0c9a1" exitCode=0
Sep 15 10:37:15.522685 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:15.522302    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw" event={"ID":"ac3cc4d8-ad26-4bfa-8690-dad507e6e055","Type":"ContainerDied","Data":"b11d7ac7ec2b3856c2941bdffb010bbf28295187a85e5c9e1cc021cb63e0c9a1"}
Sep 15 10:37:15.522765 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:15.522750    2583 scope.go:117] "RemoveContainer" containerID="b11d7ac7ec2b3856c2941bdffb010bbf28295187a85e5c9e1cc021cb63e0c9a1"
Sep 15 10:37:15.523984 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:15.523957    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:37:15.524037 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:15.524026    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:37:15.524074 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:15.524055    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:37:16.530518 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:16.530481    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw" event={"ID":"ac3cc4d8-ad26-4bfa-8690-dad507e6e055","Type":"ContainerStarted","Data":"713269bb8818f7d7368b2ee616ea97dd61d7711e7dbdc4a77930134f25b41c59"}
Sep 15 10:37:16.599823 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:16.599787    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:37:16.601051 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:16.601017    2583 patch_prober.go:28] interesting pod/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 15 10:37:16.601196 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:16.601077    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw" podUID="ac3cc4d8-ad26-4bfa-8690-dad507e6e055" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 15 10:37:17.599730 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:17.599692    2583 patch_prober.go:28] interesting pod/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 15 10:37:17.600137 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:17.599747    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw" podUID="ac3cc4d8-ad26-4bfa-8690-dad507e6e055" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 15 10:37:17.669850 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:17.669814    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:37:18.385178 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:18.385145    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:37:18.599373 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:18.599338    2583 patch_prober.go:28] interesting pod/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 15 10:37:18.599571 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:18.599393    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw" podUID="ac3cc4d8-ad26-4bfa-8690-dad507e6e055" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 15 10:37:18.988227 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:18.988191    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:37:19.183299 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:19.183270    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:37:19.579489 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:19.579453    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:37:19.599828 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:19.599798    2583 patch_prober.go:28] interesting pod/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 15 10:37:19.599968 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:19.599852    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw" podUID="ac3cc4d8-ad26-4bfa-8690-dad507e6e055" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 15 10:37:20.339624 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:20.339591    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:37:20.372908 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:20.372834    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:37:20.600068 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:20.599985    2583 patch_prober.go:28] interesting pod/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 15 10:37:20.600068 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:20.600036    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw" podUID="ac3cc4d8-ad26-4bfa-8690-dad507e6e055" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 15 10:37:20.997824 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:20.997792    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:37:21.167847 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:21.167806    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:37:21.402364 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:21.402337    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:37:21.703850 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:21.703770    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:37:21.800170 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:21.800132    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:37:21.801594 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:21.801568    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:37:21.805055 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:21.805023    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:37:22.370438 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:22.370407    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:37:40.318889 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:40.318806    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:37:40.375243 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:40.375208    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:37:45.396029 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:45.395984    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:37:50.197671 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:50.197636    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:37:52.938180 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:52.938144    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:37:53.039364 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:53.039329    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:37:53.189704 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:53.189622    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:37:53.605681 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:53.605594    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:37:53.888970 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:53.888940    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:37:54.031169 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:54.031136    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:37:55.245379 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:55.245336    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:37:55.341917 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:55.341889    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:37:55.447206 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:37:55.447178    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:38:10.756587 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:10.756555    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:38:13.388890 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:13.388845    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:38:29.458235 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:29.458192    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:38:30.119057 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:30.119022    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:38:30.869382 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:30.869352    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:38:31.474737 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:31.474709    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:38:31.653260 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:31.653229    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf"]
Sep 15 10:38:31.656934 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:31.656918    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf"
Sep 15 10:38:31.659365 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:31.659333    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6111ecc0\"/\"e2e-lifecycle-model-6111ecc0-kserve-self-signed-certs\""
Sep 15 10:38:31.660706 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:31.660680    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6111ecc0\"/\"kube-root-ca.crt\""
Sep 15 10:38:31.660807 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:31.660738    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6111ecc0\"/\"default-dockercfg-2pd8h\""
Sep 15 10:38:31.660853 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:31.660681    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6111ecc0\"/\"openshift-service-ca.crt\""
Sep 15 10:38:31.669841 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:31.669807    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf"]
Sep 15 10:38:31.690979 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:31.690950    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:38:31.712137 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:31.712116    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b91056a-8d16-41e1-a9e3-589d7482133d-dshm\") pod \"e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf\" (UID: \"1b91056a-8d16-41e1-a9e3-589d7482133d\") " pod="ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf"
Sep 15 10:38:31.712247 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:31.712147    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b91056a-8d16-41e1-a9e3-589d7482133d-tls-certs\") pod \"e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf\" (UID: \"1b91056a-8d16-41e1-a9e3-589d7482133d\") " pod="ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf"
Sep 15 10:38:31.712247 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:31.712165    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjsqc\" (UniqueName: \"kubernetes.io/projected/1b91056a-8d16-41e1-a9e3-589d7482133d-kube-api-access-kjsqc\") pod \"e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf\" (UID: \"1b91056a-8d16-41e1-a9e3-589d7482133d\") " pod="ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf"
Sep 15 10:38:31.712321 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:31.712254    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b91056a-8d16-41e1-a9e3-589d7482133d-tmp-dir\") pod \"e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf\" (UID: \"1b91056a-8d16-41e1-a9e3-589d7482133d\") " pod="ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf"
Sep 15 10:38:31.712321 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:31.712291    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b91056a-8d16-41e1-a9e3-589d7482133d-model-cache\") pod \"e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf\" (UID: \"1b91056a-8d16-41e1-a9e3-589d7482133d\") " pod="ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf"
Sep 15 10:38:31.712387 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:31.712332    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b91056a-8d16-41e1-a9e3-589d7482133d-home\") pod \"e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf\" (UID: \"1b91056a-8d16-41e1-a9e3-589d7482133d\") " pod="ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf"
Sep 15 10:38:31.813299 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:31.813232    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b91056a-8d16-41e1-a9e3-589d7482133d-dshm\") pod \"e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf\" (UID: \"1b91056a-8d16-41e1-a9e3-589d7482133d\") " pod="ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf"
Sep 15 10:38:31.813299 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:31.813265    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b91056a-8d16-41e1-a9e3-589d7482133d-tls-certs\") pod \"e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf\" (UID: \"1b91056a-8d16-41e1-a9e3-589d7482133d\") " pod="ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf"
Sep 15 10:38:31.813299 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:31.813281    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjsqc\" (UniqueName: \"kubernetes.io/projected/1b91056a-8d16-41e1-a9e3-589d7482133d-kube-api-access-kjsqc\") pod \"e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf\" (UID: \"1b91056a-8d16-41e1-a9e3-589d7482133d\") " pod="ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf"
Sep 15 10:38:31.813557 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:31.813331    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b91056a-8d16-41e1-a9e3-589d7482133d-tmp-dir\") pod \"e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf\" (UID: \"1b91056a-8d16-41e1-a9e3-589d7482133d\") " pod="ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf"
Sep 15 10:38:31.813557 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:31.813379    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b91056a-8d16-41e1-a9e3-589d7482133d-model-cache\") pod \"e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf\" (UID: \"1b91056a-8d16-41e1-a9e3-589d7482133d\") " pod="ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf"
Sep 15 10:38:31.813557 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:31.813422    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b91056a-8d16-41e1-a9e3-589d7482133d-home\") pod \"e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf\" (UID: \"1b91056a-8d16-41e1-a9e3-589d7482133d\") " pod="ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf"
Sep 15 10:38:31.813773 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:31.813754    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b91056a-8d16-41e1-a9e3-589d7482133d-tmp-dir\") pod \"e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf\" (UID: \"1b91056a-8d16-41e1-a9e3-589d7482133d\") " pod="ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf"
Sep 15 10:38:31.813840 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:31.813795    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b91056a-8d16-41e1-a9e3-589d7482133d-model-cache\") pod \"e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf\" (UID: \"1b91056a-8d16-41e1-a9e3-589d7482133d\") " pod="ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf"
Sep 15 10:38:31.813880 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:31.813854    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b91056a-8d16-41e1-a9e3-589d7482133d-home\") pod \"e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf\" (UID: \"1b91056a-8d16-41e1-a9e3-589d7482133d\") " pod="ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf"
Sep 15 10:38:31.815655 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:31.815636    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b91056a-8d16-41e1-a9e3-589d7482133d-dshm\") pod \"e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf\" (UID: \"1b91056a-8d16-41e1-a9e3-589d7482133d\") " pod="ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf"
Sep 15 10:38:31.815826 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:31.815809    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b91056a-8d16-41e1-a9e3-589d7482133d-tls-certs\") pod \"e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf\" (UID: \"1b91056a-8d16-41e1-a9e3-589d7482133d\") " pod="ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf"
Sep 15 10:38:31.822620 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:31.822602    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjsqc\" (UniqueName: \"kubernetes.io/projected/1b91056a-8d16-41e1-a9e3-589d7482133d-kube-api-access-kjsqc\") pod \"e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf\" (UID: \"1b91056a-8d16-41e1-a9e3-589d7482133d\") " pod="ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf"
Sep 15 10:38:31.966841 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:31.966809    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf"
Sep 15 10:38:32.193813 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:32.193770    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:38:32.240536 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:32.240503    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf"]
Sep 15 10:38:32.241510 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:38:32.241483    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1b91056a_8d16_41e1_a9e3_589d7482133d.slice/crio-06c0454dab6e4c80ec1667fbf90c54cb947f24a3eb1c7c3a4ba5210db32df4e8 WatchSource:0}: Error finding container 06c0454dab6e4c80ec1667fbf90c54cb947f24a3eb1c7c3a4ba5210db32df4e8: Status 404 returned error can't find the container with id 06c0454dab6e4c80ec1667fbf90c54cb947f24a3eb1c7c3a4ba5210db32df4e8
Sep 15 10:38:32.982659 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:32.982627    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:38:33.017555 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:33.017516    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf" event={"ID":"1b91056a-8d16-41e1-a9e3-589d7482133d","Type":"ContainerStarted","Data":"786c4032de10f31162099932ba5aba06ca0c922278ff23546c40c7c0b358478a"}
Sep 15 10:38:33.017715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:33.017565    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf" event={"ID":"1b91056a-8d16-41e1-a9e3-589d7482133d","Type":"ContainerStarted","Data":"06c0454dab6e4c80ec1667fbf90c54cb947f24a3eb1c7c3a4ba5210db32df4e8"}
Sep 15 10:38:33.038272 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:33.038217    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf" podStartSLOduration=2.038201937 podStartE2EDuration="2.038201937s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:38:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:38:33.034917737 +0000 UTC m=+1228.281884706" watchObservedRunningTime="2026-09-15 10:38:33.038201937 +0000 UTC m=+1228.285168907"
Sep 15 10:38:34.151835 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:34.151803    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:38:34.252414 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:34.252380    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:38:34.974858 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:34.974825    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:38:35.353920 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:35.353837    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:38:39.433151 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:39.433114    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj"]
Sep 15 10:38:39.433637 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:39.433384    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj" podUID="60fe0698-4c41-4f62-9389-f8b03cd9086a" containerName="maas-api" containerID="cri-o://c5631c76951d92cc875a3c52acca970c3d7f8fbea5699338b7d9278760ddb4b8" gracePeriod=30
Sep 15 10:38:39.459927 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:39.459868    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:38:39.561487 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:39.561407    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:38:39.641459 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:39.641399    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:38:39.690027 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:39.689965    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj"
Sep 15 10:38:39.695373 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:39.695353    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:38:39.799137 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:39.799096    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60fe0698-4c41-4f62-9389-f8b03cd9086a-openshift-service-ca\") pod \"60fe0698-4c41-4f62-9389-f8b03cd9086a\" (UID: \"60fe0698-4c41-4f62-9389-f8b03cd9086a\") "
Sep 15 10:38:39.799329 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:39.799175    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60fe0698-4c41-4f62-9389-f8b03cd9086a-maas-api-tls\") pod \"60fe0698-4c41-4f62-9389-f8b03cd9086a\" (UID: \"60fe0698-4c41-4f62-9389-f8b03cd9086a\") "
Sep 15 10:38:39.799329 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:39.799270    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqh4l\" (UniqueName: \"kubernetes.io/projected/60fe0698-4c41-4f62-9389-f8b03cd9086a-kube-api-access-wqh4l\") pod \"60fe0698-4c41-4f62-9389-f8b03cd9086a\" (UID: \"60fe0698-4c41-4f62-9389-f8b03cd9086a\") "
Sep 15 10:38:39.799329 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:39.799323    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60fe0698-4c41-4f62-9389-f8b03cd9086a-metrics-certs\") pod \"60fe0698-4c41-4f62-9389-f8b03cd9086a\" (UID: \"60fe0698-4c41-4f62-9389-f8b03cd9086a\") "
Sep 15 10:38:39.799541 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:39.799509    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60fe0698-4c41-4f62-9389-f8b03cd9086a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "60fe0698-4c41-4f62-9389-f8b03cd9086a" (UID: "60fe0698-4c41-4f62-9389-f8b03cd9086a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:38:39.799744 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:39.799725    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60fe0698-4c41-4f62-9389-f8b03cd9086a-openshift-service-ca\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:38:39.801375 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:39.801335    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60fe0698-4c41-4f62-9389-f8b03cd9086a-kube-api-access-wqh4l" (OuterVolumeSpecName: "kube-api-access-wqh4l") pod "60fe0698-4c41-4f62-9389-f8b03cd9086a" (UID: "60fe0698-4c41-4f62-9389-f8b03cd9086a"). InnerVolumeSpecName "kube-api-access-wqh4l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:38:39.801375 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:39.801354    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60fe0698-4c41-4f62-9389-f8b03cd9086a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "60fe0698-4c41-4f62-9389-f8b03cd9086a" (UID: "60fe0698-4c41-4f62-9389-f8b03cd9086a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:38:39.801540 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:39.801402    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60fe0698-4c41-4f62-9389-f8b03cd9086a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "60fe0698-4c41-4f62-9389-f8b03cd9086a" (UID: "60fe0698-4c41-4f62-9389-f8b03cd9086a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:38:39.900785 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:39.900742    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60fe0698-4c41-4f62-9389-f8b03cd9086a-maas-api-tls\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:38:39.900785 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:39.900776    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqh4l\" (UniqueName: \"kubernetes.io/projected/60fe0698-4c41-4f62-9389-f8b03cd9086a-kube-api-access-wqh4l\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:38:39.900785 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:39.900789    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60fe0698-4c41-4f62-9389-f8b03cd9086a-metrics-certs\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:38:39.991544 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:39.991462    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:38:40.061299 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:40.061266    2583 generic.go:356] "Generic (PLEG): container finished" podID="60fe0698-4c41-4f62-9389-f8b03cd9086a" containerID="c5631c76951d92cc875a3c52acca970c3d7f8fbea5699338b7d9278760ddb4b8" exitCode=0
Sep 15 10:38:40.061527 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:40.061322    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj"
Sep 15 10:38:40.061527 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:40.061358    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj" event={"ID":"60fe0698-4c41-4f62-9389-f8b03cd9086a","Type":"ContainerDied","Data":"c5631c76951d92cc875a3c52acca970c3d7f8fbea5699338b7d9278760ddb4b8"}
Sep 15 10:38:40.061527 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:40.061413    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj" event={"ID":"60fe0698-4c41-4f62-9389-f8b03cd9086a","Type":"ContainerDied","Data":"4efd2d1c84bb1fb37349d6a1cb176ff747bc0ed130b87c6762f8843b3b548a1d"}
Sep 15 10:38:40.061527 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:40.061436    2583 scope.go:117] "RemoveContainer" containerID="c5631c76951d92cc875a3c52acca970c3d7f8fbea5699338b7d9278760ddb4b8"
Sep 15 10:38:40.072018 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:40.071994    2583 scope.go:117] "RemoveContainer" containerID="c5631c76951d92cc875a3c52acca970c3d7f8fbea5699338b7d9278760ddb4b8"
Sep 15 10:38:40.072359 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:38:40.072331    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5631c76951d92cc875a3c52acca970c3d7f8fbea5699338b7d9278760ddb4b8\": container with ID starting with c5631c76951d92cc875a3c52acca970c3d7f8fbea5699338b7d9278760ddb4b8 not found: ID does not exist" containerID="c5631c76951d92cc875a3c52acca970c3d7f8fbea5699338b7d9278760ddb4b8"
Sep 15 10:38:40.072471 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:40.072371    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5631c76951d92cc875a3c52acca970c3d7f8fbea5699338b7d9278760ddb4b8"} err="failed to get container status \"c5631c76951d92cc875a3c52acca970c3d7f8fbea5699338b7d9278760ddb4b8\": rpc error: code = NotFound desc = could not find container \"c5631c76951d92cc875a3c52acca970c3d7f8fbea5699338b7d9278760ddb4b8\": container with ID starting with c5631c76951d92cc875a3c52acca970c3d7f8fbea5699338b7d9278760ddb4b8 not found: ID does not exist"
Sep 15 10:38:40.084372 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:40.084146    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj"]
Sep 15 10:38:40.087469 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:40.087425    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-86a8f5-c64cd744b-jh9kj"]
Sep 15 10:38:40.149120 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:40.149083    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:38:40.786221 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:40.786192    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:38:41.300876 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:41.300841    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60fe0698-4c41-4f62-9389-f8b03cd9086a" path="/var/lib/kubelet/pods/60fe0698-4c41-4f62-9389-f8b03cd9086a/volumes"
Sep 15 10:38:41.577291 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:41.577199    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:38:41.967417 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:41.967372    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf"
Sep 15 10:38:41.967837 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:41.967433    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf"
Sep 15 10:38:41.980369 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:41.980339    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf"
Sep 15 10:38:42.094069 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:42.094039    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf"
Sep 15 10:38:42.288744 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:42.288650    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:38:42.905961 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:42.905925    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:38:43.499509 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:43.499422    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:38:44.196188 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:44.196153    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:38:45.075006 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:45.074974    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:38:59.095202 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:59.095127    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:38:59.311287 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:59.311248    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:38:59.602787 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:59.602754    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"]
Sep 15 10:38:59.602996 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:59.602973    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx" podUID="3cead169-6453-4e87-8ab8-d1d47e6502d3" containerName="istio-proxy" containerID="cri-o://7fe6ab28c7b017a26b128d7675f6f12de8b05e45c0a6522a1a7a459a2fa1254e" gracePeriod=30
Sep 15 10:38:59.643523 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:38:59.643490    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:00.091918 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:00.091886    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:00.889165 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:00.889137    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:01.477687 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:01.477653    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:02.184425 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:02.184395    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:02.804391 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:02.804357    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:03.043287 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:03.043252    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6"]
Sep 15 10:39:03.060227 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:03.060132    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:03.593719 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:03.593687    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:04.182456 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:04.182412    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:04.860783 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:04.860756    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:39:04.966135 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:04.966098    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:04.972093 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:04.972070    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3cead169-6453-4e87-8ab8-d1d47e6502d3-istio-envoy\") pod \"3cead169-6453-4e87-8ab8-d1d47e6502d3\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") "
Sep 15 10:39:04.972213 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:04.972118    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3cead169-6453-4e87-8ab8-d1d47e6502d3-istiod-ca-cert\") pod \"3cead169-6453-4e87-8ab8-d1d47e6502d3\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") "
Sep 15 10:39:04.972213 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:04.972156    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3cead169-6453-4e87-8ab8-d1d47e6502d3-istio-data\") pod \"3cead169-6453-4e87-8ab8-d1d47e6502d3\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") "
Sep 15 10:39:04.972213 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:04.972173    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3cead169-6453-4e87-8ab8-d1d47e6502d3-istio-podinfo\") pod \"3cead169-6453-4e87-8ab8-d1d47e6502d3\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") "
Sep 15 10:39:04.972336 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:04.972236    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cead169-6453-4e87-8ab8-d1d47e6502d3-workload-socket\") pod \"3cead169-6453-4e87-8ab8-d1d47e6502d3\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") "
Sep 15 10:39:04.972336 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:04.972293    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkz99\" (UniqueName: \"kubernetes.io/projected/3cead169-6453-4e87-8ab8-d1d47e6502d3-kube-api-access-xkz99\") pod \"3cead169-6453-4e87-8ab8-d1d47e6502d3\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") "
Sep 15 10:39:04.972455 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:04.972375    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3cead169-6453-4e87-8ab8-d1d47e6502d3-istio-token\") pod \"3cead169-6453-4e87-8ab8-d1d47e6502d3\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") "
Sep 15 10:39:04.972455 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:04.972399    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3cead169-6453-4e87-8ab8-d1d47e6502d3-workload-certs\") pod \"3cead169-6453-4e87-8ab8-d1d47e6502d3\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") "
Sep 15 10:39:04.972564 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:04.972434    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cead169-6453-4e87-8ab8-d1d47e6502d3-credential-socket\") pod \"3cead169-6453-4e87-8ab8-d1d47e6502d3\" (UID: \"3cead169-6453-4e87-8ab8-d1d47e6502d3\") "
Sep 15 10:39:04.972621 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:04.972563    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cead169-6453-4e87-8ab8-d1d47e6502d3-istio-data" (OuterVolumeSpecName: "istio-data") pod "3cead169-6453-4e87-8ab8-d1d47e6502d3" (UID: "3cead169-6453-4e87-8ab8-d1d47e6502d3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:39:04.972621 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:04.972604    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cead169-6453-4e87-8ab8-d1d47e6502d3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3cead169-6453-4e87-8ab8-d1d47e6502d3" (UID: "3cead169-6453-4e87-8ab8-d1d47e6502d3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:39:04.972732 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:04.972655    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cead169-6453-4e87-8ab8-d1d47e6502d3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3cead169-6453-4e87-8ab8-d1d47e6502d3" (UID: "3cead169-6453-4e87-8ab8-d1d47e6502d3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:39:04.972847 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:04.972814    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cead169-6453-4e87-8ab8-d1d47e6502d3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3cead169-6453-4e87-8ab8-d1d47e6502d3" (UID: "3cead169-6453-4e87-8ab8-d1d47e6502d3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:39:04.972928 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:04.972857    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cead169-6453-4e87-8ab8-d1d47e6502d3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3cead169-6453-4e87-8ab8-d1d47e6502d3" (UID: "3cead169-6453-4e87-8ab8-d1d47e6502d3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:39:04.973045 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:04.973024    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3cead169-6453-4e87-8ab8-d1d47e6502d3-workload-certs\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:39:04.973106 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:04.973051    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cead169-6453-4e87-8ab8-d1d47e6502d3-credential-socket\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:39:04.973106 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:04.973067    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3cead169-6453-4e87-8ab8-d1d47e6502d3-istiod-ca-cert\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:39:04.973106 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:04.973081    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3cead169-6453-4e87-8ab8-d1d47e6502d3-istio-data\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:39:04.973106 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:04.973093    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3cead169-6453-4e87-8ab8-d1d47e6502d3-workload-socket\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:39:04.974343 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:04.974318    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cead169-6453-4e87-8ab8-d1d47e6502d3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3cead169-6453-4e87-8ab8-d1d47e6502d3" (UID: "3cead169-6453-4e87-8ab8-d1d47e6502d3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:39:04.974626 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:04.974609    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cead169-6453-4e87-8ab8-d1d47e6502d3-kube-api-access-xkz99" (OuterVolumeSpecName: "kube-api-access-xkz99") pod "3cead169-6453-4e87-8ab8-d1d47e6502d3" (UID: "3cead169-6453-4e87-8ab8-d1d47e6502d3"). InnerVolumeSpecName "kube-api-access-xkz99". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:39:04.974877 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:04.974849    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cead169-6453-4e87-8ab8-d1d47e6502d3-istio-token" (OuterVolumeSpecName: "istio-token") pod "3cead169-6453-4e87-8ab8-d1d47e6502d3" (UID: "3cead169-6453-4e87-8ab8-d1d47e6502d3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:39:04.974877 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:04.974863    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3cead169-6453-4e87-8ab8-d1d47e6502d3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3cead169-6453-4e87-8ab8-d1d47e6502d3" (UID: "3cead169-6453-4e87-8ab8-d1d47e6502d3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 10:39:05.073888 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:05.073847    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3cead169-6453-4e87-8ab8-d1d47e6502d3-istio-podinfo\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:39:05.073888 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:05.073881    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkz99\" (UniqueName: \"kubernetes.io/projected/3cead169-6453-4e87-8ab8-d1d47e6502d3-kube-api-access-xkz99\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:39:05.073888 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:05.073892    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3cead169-6453-4e87-8ab8-d1d47e6502d3-istio-token\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:39:05.073888 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:05.073900    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3cead169-6453-4e87-8ab8-d1d47e6502d3-istio-envoy\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:39:05.230055 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:05.229966    2583 generic.go:356] "Generic (PLEG): container finished" podID="3cead169-6453-4e87-8ab8-d1d47e6502d3" containerID="7fe6ab28c7b017a26b128d7675f6f12de8b05e45c0a6522a1a7a459a2fa1254e" exitCode=0
Sep 15 10:39:05.230055 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:05.230017    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"
Sep 15 10:39:05.230055 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:05.230046    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx" event={"ID":"3cead169-6453-4e87-8ab8-d1d47e6502d3","Type":"ContainerDied","Data":"7fe6ab28c7b017a26b128d7675f6f12de8b05e45c0a6522a1a7a459a2fa1254e"}
Sep 15 10:39:05.230276 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:05.230141    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx" event={"ID":"3cead169-6453-4e87-8ab8-d1d47e6502d3","Type":"ContainerDied","Data":"2ca1266b161e93a474fb156329719c6cf6f7c365937886ad97bc9104e934970d"}
Sep 15 10:39:05.230276 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:05.230172    2583 scope.go:117] "RemoveContainer" containerID="7fe6ab28c7b017a26b128d7675f6f12de8b05e45c0a6522a1a7a459a2fa1254e"
Sep 15 10:39:05.240519 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:05.240499    2583 scope.go:117] "RemoveContainer" containerID="9a2e012a54a8ee76ab5206e79a33d5033ad77b03ec384037906a11e09f263cf4"
Sep 15 10:39:05.249946 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:05.249922    2583 scope.go:117] "RemoveContainer" containerID="7fe6ab28c7b017a26b128d7675f6f12de8b05e45c0a6522a1a7a459a2fa1254e"
Sep 15 10:39:05.250240 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:39:05.250218    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fe6ab28c7b017a26b128d7675f6f12de8b05e45c0a6522a1a7a459a2fa1254e\": container with ID starting with 7fe6ab28c7b017a26b128d7675f6f12de8b05e45c0a6522a1a7a459a2fa1254e not found: ID does not exist" containerID="7fe6ab28c7b017a26b128d7675f6f12de8b05e45c0a6522a1a7a459a2fa1254e"
Sep 15 10:39:05.250299 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:05.250248    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fe6ab28c7b017a26b128d7675f6f12de8b05e45c0a6522a1a7a459a2fa1254e"} err="failed to get container status \"7fe6ab28c7b017a26b128d7675f6f12de8b05e45c0a6522a1a7a459a2fa1254e\": rpc error: code = NotFound desc = could not find container \"7fe6ab28c7b017a26b128d7675f6f12de8b05e45c0a6522a1a7a459a2fa1254e\": container with ID starting with 7fe6ab28c7b017a26b128d7675f6f12de8b05e45c0a6522a1a7a459a2fa1254e not found: ID does not exist"
Sep 15 10:39:05.250299 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:05.250266    2583 scope.go:117] "RemoveContainer" containerID="9a2e012a54a8ee76ab5206e79a33d5033ad77b03ec384037906a11e09f263cf4"
Sep 15 10:39:05.250562 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:39:05.250546    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a2e012a54a8ee76ab5206e79a33d5033ad77b03ec384037906a11e09f263cf4\": container with ID starting with 9a2e012a54a8ee76ab5206e79a33d5033ad77b03ec384037906a11e09f263cf4 not found: ID does not exist" containerID="9a2e012a54a8ee76ab5206e79a33d5033ad77b03ec384037906a11e09f263cf4"
Sep 15 10:39:05.250616 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:05.250568    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a2e012a54a8ee76ab5206e79a33d5033ad77b03ec384037906a11e09f263cf4"} err="failed to get container status \"9a2e012a54a8ee76ab5206e79a33d5033ad77b03ec384037906a11e09f263cf4\": rpc error: code = NotFound desc = could not find container \"9a2e012a54a8ee76ab5206e79a33d5033ad77b03ec384037906a11e09f263cf4\": container with ID starting with 9a2e012a54a8ee76ab5206e79a33d5033ad77b03ec384037906a11e09f263cf4 not found: ID does not exist"
Sep 15 10:39:05.252538 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:05.252517    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"]
Sep 15 10:39:05.257296 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:05.257272    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-86a8f5-openshift-default-77bdc7c74b-6twkx"]
Sep 15 10:39:05.302539 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:05.302508    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cead169-6453-4e87-8ab8-d1d47e6502d3" path="/var/lib/kubelet/pods/3cead169-6453-4e87-8ab8-d1d47e6502d3/volumes"
Sep 15 10:39:10.348219 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:10.348182    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg"]
Sep 15 10:39:10.348740 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:10.348423    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg" podUID="2db6e97f-345b-4fa3-9b48-127cd97bb0d4" containerName="maas-api" containerID="cri-o://57a4e042eee80c63717957eb7b9c48d4d7c74b80fd6f5d90de6861ee0d4e03c1" gracePeriod=30
Sep 15 10:39:10.377516 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:10.377481    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:10.477678 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:10.477599    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:10.568555 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:10.568524    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:10.609622 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:10.609559    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg"
Sep 15 10:39:10.624251 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:10.624209    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:10.738756 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:10.738722    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2db6e97f-345b-4fa3-9b48-127cd97bb0d4-metrics-certs\") pod \"2db6e97f-345b-4fa3-9b48-127cd97bb0d4\" (UID: \"2db6e97f-345b-4fa3-9b48-127cd97bb0d4\") "
Sep 15 10:39:10.738966 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:10.738802    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2db6e97f-345b-4fa3-9b48-127cd97bb0d4-openshift-service-ca\") pod \"2db6e97f-345b-4fa3-9b48-127cd97bb0d4\" (UID: \"2db6e97f-345b-4fa3-9b48-127cd97bb0d4\") "
Sep 15 10:39:10.738966 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:10.738904    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sz5d6\" (UniqueName: \"kubernetes.io/projected/2db6e97f-345b-4fa3-9b48-127cd97bb0d4-kube-api-access-sz5d6\") pod \"2db6e97f-345b-4fa3-9b48-127cd97bb0d4\" (UID: \"2db6e97f-345b-4fa3-9b48-127cd97bb0d4\") "
Sep 15 10:39:10.738966 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:10.738952    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2db6e97f-345b-4fa3-9b48-127cd97bb0d4-maas-api-tls\") pod \"2db6e97f-345b-4fa3-9b48-127cd97bb0d4\" (UID: \"2db6e97f-345b-4fa3-9b48-127cd97bb0d4\") "
Sep 15 10:39:10.739177 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:10.739157    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2db6e97f-345b-4fa3-9b48-127cd97bb0d4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2db6e97f-345b-4fa3-9b48-127cd97bb0d4" (UID: "2db6e97f-345b-4fa3-9b48-127cd97bb0d4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:39:10.739419 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:10.739397    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2db6e97f-345b-4fa3-9b48-127cd97bb0d4-openshift-service-ca\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:39:10.741017 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:10.740994    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2db6e97f-345b-4fa3-9b48-127cd97bb0d4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2db6e97f-345b-4fa3-9b48-127cd97bb0d4" (UID: "2db6e97f-345b-4fa3-9b48-127cd97bb0d4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:39:10.741110 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:10.741044    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2db6e97f-345b-4fa3-9b48-127cd97bb0d4-kube-api-access-sz5d6" (OuterVolumeSpecName: "kube-api-access-sz5d6") pod "2db6e97f-345b-4fa3-9b48-127cd97bb0d4" (UID: "2db6e97f-345b-4fa3-9b48-127cd97bb0d4"). InnerVolumeSpecName "kube-api-access-sz5d6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:39:10.741110 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:10.741044    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2db6e97f-345b-4fa3-9b48-127cd97bb0d4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2db6e97f-345b-4fa3-9b48-127cd97bb0d4" (UID: "2db6e97f-345b-4fa3-9b48-127cd97bb0d4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:39:10.839982 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:10.839951    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2db6e97f-345b-4fa3-9b48-127cd97bb0d4-metrics-certs\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:39:10.839982 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:10.839977    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sz5d6\" (UniqueName: \"kubernetes.io/projected/2db6e97f-345b-4fa3-9b48-127cd97bb0d4-kube-api-access-sz5d6\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:39:10.839982 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:10.839986    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2db6e97f-345b-4fa3-9b48-127cd97bb0d4-maas-api-tls\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:39:10.955532 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:10.955490    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:11.057406 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:11.057380    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:11.266976 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:11.266875    2583 generic.go:356] "Generic (PLEG): container finished" podID="2db6e97f-345b-4fa3-9b48-127cd97bb0d4" containerID="57a4e042eee80c63717957eb7b9c48d4d7c74b80fd6f5d90de6861ee0d4e03c1" exitCode=0
Sep 15 10:39:11.266976 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:11.266918    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg"
Sep 15 10:39:11.266976 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:11.266960    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg" event={"ID":"2db6e97f-345b-4fa3-9b48-127cd97bb0d4","Type":"ContainerDied","Data":"57a4e042eee80c63717957eb7b9c48d4d7c74b80fd6f5d90de6861ee0d4e03c1"}
Sep 15 10:39:11.267253 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:11.267014    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg" event={"ID":"2db6e97f-345b-4fa3-9b48-127cd97bb0d4","Type":"ContainerDied","Data":"3f08879986c6934ea3eeaa28f1f258b27228e277736bb40254a76922eea37145"}
Sep 15 10:39:11.267253 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:11.267037    2583 scope.go:117] "RemoveContainer" containerID="57a4e042eee80c63717957eb7b9c48d4d7c74b80fd6f5d90de6861ee0d4e03c1"
Sep 15 10:39:11.277223 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:11.277203    2583 scope.go:117] "RemoveContainer" containerID="57a4e042eee80c63717957eb7b9c48d4d7c74b80fd6f5d90de6861ee0d4e03c1"
Sep 15 10:39:11.277532 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:39:11.277514    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57a4e042eee80c63717957eb7b9c48d4d7c74b80fd6f5d90de6861ee0d4e03c1\": container with ID starting with 57a4e042eee80c63717957eb7b9c48d4d7c74b80fd6f5d90de6861ee0d4e03c1 not found: ID does not exist" containerID="57a4e042eee80c63717957eb7b9c48d4d7c74b80fd6f5d90de6861ee0d4e03c1"
Sep 15 10:39:11.277595 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:11.277542    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57a4e042eee80c63717957eb7b9c48d4d7c74b80fd6f5d90de6861ee0d4e03c1"} err="failed to get container status \"57a4e042eee80c63717957eb7b9c48d4d7c74b80fd6f5d90de6861ee0d4e03c1\": rpc error: code = NotFound desc = could not find container \"57a4e042eee80c63717957eb7b9c48d4d7c74b80fd6f5d90de6861ee0d4e03c1\": container with ID starting with 57a4e042eee80c63717957eb7b9c48d4d7c74b80fd6f5d90de6861ee0d4e03c1 not found: ID does not exist"
Sep 15 10:39:11.288772 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:11.288749    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg"]
Sep 15 10:39:11.300127 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:11.300108    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-234840-5b76c7c8b4-6dsrg"]
Sep 15 10:39:11.681775 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:11.681744    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:12.495968 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:12.495939    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:13.112013 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:13.111976    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:13.300552 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:13.300521    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2db6e97f-345b-4fa3-9b48-127cd97bb0d4" path="/var/lib/kubelet/pods/2db6e97f-345b-4fa3-9b48-127cd97bb0d4/volumes"
Sep 15 10:39:13.974382 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:13.974352    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:14.572376 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:14.572342    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:16.030150 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:16.030119    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:17.700587 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:17.700556    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:18.063415 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:18.063314    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6" podUID="fcedb166-03b2-448c-b397-2d8f44f799ba" containerName="main" containerID="cri-o://f4b8d9f135d230db4995920620855b58090c46c4ec70dda6463527d7d069b00b" gracePeriod=45
Sep 15 10:39:18.310962 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:18.310935    2583 generic.go:356] "Generic (PLEG): container finished" podID="fcedb166-03b2-448c-b397-2d8f44f799ba" containerID="f4b8d9f135d230db4995920620855b58090c46c4ec70dda6463527d7d069b00b" exitCode=0
Sep 15 10:39:18.311081 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:18.311005    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6" event={"ID":"fcedb166-03b2-448c-b397-2d8f44f799ba","Type":"ContainerDied","Data":"f4b8d9f135d230db4995920620855b58090c46c4ec70dda6463527d7d069b00b"}
Sep 15 10:39:18.311081 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:18.311041    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6" event={"ID":"fcedb166-03b2-448c-b397-2d8f44f799ba","Type":"ContainerDied","Data":"840a675c60e6db962bbc6550bd0285c86cd96e7beec7549e45338b27a1784cca"}
Sep 15 10:39:18.311081 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:18.311053    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="840a675c60e6db962bbc6550bd0285c86cd96e7beec7549e45338b27a1784cca"
Sep 15 10:39:18.318462 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:18.318395    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6"
Sep 15 10:39:18.416101 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:18.416076    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b44d7\" (UniqueName: \"kubernetes.io/projected/fcedb166-03b2-448c-b397-2d8f44f799ba-kube-api-access-b44d7\") pod \"fcedb166-03b2-448c-b397-2d8f44f799ba\" (UID: \"fcedb166-03b2-448c-b397-2d8f44f799ba\") "
Sep 15 10:39:18.417400 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:18.416151    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcedb166-03b2-448c-b397-2d8f44f799ba-tmp-dir\") pod \"fcedb166-03b2-448c-b397-2d8f44f799ba\" (UID: \"fcedb166-03b2-448c-b397-2d8f44f799ba\") "
Sep 15 10:39:18.417400 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:18.416725    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcedb166-03b2-448c-b397-2d8f44f799ba-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fcedb166-03b2-448c-b397-2d8f44f799ba" (UID: "fcedb166-03b2-448c-b397-2d8f44f799ba"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:39:18.417400 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:18.416776    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcedb166-03b2-448c-b397-2d8f44f799ba-dshm\") pod \"fcedb166-03b2-448c-b397-2d8f44f799ba\" (UID: \"fcedb166-03b2-448c-b397-2d8f44f799ba\") "
Sep 15 10:39:18.417400 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:18.416840    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcedb166-03b2-448c-b397-2d8f44f799ba-model-cache\") pod \"fcedb166-03b2-448c-b397-2d8f44f799ba\" (UID: \"fcedb166-03b2-448c-b397-2d8f44f799ba\") "
Sep 15 10:39:18.417400 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:18.417170    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcedb166-03b2-448c-b397-2d8f44f799ba-model-cache" (OuterVolumeSpecName: "model-cache") pod "fcedb166-03b2-448c-b397-2d8f44f799ba" (UID: "fcedb166-03b2-448c-b397-2d8f44f799ba"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:39:18.417400 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:18.417313    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcedb166-03b2-448c-b397-2d8f44f799ba-home\") pod \"fcedb166-03b2-448c-b397-2d8f44f799ba\" (UID: \"fcedb166-03b2-448c-b397-2d8f44f799ba\") "
Sep 15 10:39:18.417400 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:18.417398    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcedb166-03b2-448c-b397-2d8f44f799ba-tls-certs\") pod \"fcedb166-03b2-448c-b397-2d8f44f799ba\" (UID: \"fcedb166-03b2-448c-b397-2d8f44f799ba\") "
Sep 15 10:39:18.418584 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:18.418563    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcedb166-03b2-448c-b397-2d8f44f799ba-model-cache\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:39:18.418766 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:18.418754    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcedb166-03b2-448c-b397-2d8f44f799ba-tmp-dir\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:39:18.421484 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:18.419557    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcedb166-03b2-448c-b397-2d8f44f799ba-home" (OuterVolumeSpecName: "home") pod "fcedb166-03b2-448c-b397-2d8f44f799ba" (UID: "fcedb166-03b2-448c-b397-2d8f44f799ba"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:39:18.421484 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:18.419613    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcedb166-03b2-448c-b397-2d8f44f799ba-kube-api-access-b44d7" (OuterVolumeSpecName: "kube-api-access-b44d7") pod "fcedb166-03b2-448c-b397-2d8f44f799ba" (UID: "fcedb166-03b2-448c-b397-2d8f44f799ba"). InnerVolumeSpecName "kube-api-access-b44d7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:39:18.421484 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:18.419798    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcedb166-03b2-448c-b397-2d8f44f799ba-dshm" (OuterVolumeSpecName: "dshm") pod "fcedb166-03b2-448c-b397-2d8f44f799ba" (UID: "fcedb166-03b2-448c-b397-2d8f44f799ba"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:39:18.421484 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:18.420435    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcedb166-03b2-448c-b397-2d8f44f799ba-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fcedb166-03b2-448c-b397-2d8f44f799ba" (UID: "fcedb166-03b2-448c-b397-2d8f44f799ba"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:39:18.520290 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:18.520253    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcedb166-03b2-448c-b397-2d8f44f799ba-dshm\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:39:18.520290 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:18.520283    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcedb166-03b2-448c-b397-2d8f44f799ba-home\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:39:18.520290 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:18.520291    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcedb166-03b2-448c-b397-2d8f44f799ba-tls-certs\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:39:18.520564 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:18.520306    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b44d7\" (UniqueName: \"kubernetes.io/projected/fcedb166-03b2-448c-b397-2d8f44f799ba-kube-api-access-b44d7\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:39:19.316849 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:19.316813    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6"
Sep 15 10:39:19.333321 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:19.333273    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6"]
Sep 15 10:39:19.336361 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:19.336331    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-234840/test-model-234840-kserve-74c775dc8c-xfvx6"]
Sep 15 10:39:21.301331 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:21.301294    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcedb166-03b2-448c-b397-2d8f44f799ba" path="/var/lib/kubelet/pods/fcedb166-03b2-448c-b397-2d8f44f799ba/volumes"
Sep 15 10:39:21.829129 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:21.829099    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:22.110493 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:22.110389    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:22.688670 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:22.688630    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:23.003142 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.003061    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"]
Sep 15 10:39:23.003555 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.003539    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcedb166-03b2-448c-b397-2d8f44f799ba" containerName="main"
Sep 15 10:39:23.003606 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.003557    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcedb166-03b2-448c-b397-2d8f44f799ba" containerName="main"
Sep 15 10:39:23.003606 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.003573    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cead169-6453-4e87-8ab8-d1d47e6502d3" containerName="istio-proxy"
Sep 15 10:39:23.003606 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.003579    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cead169-6453-4e87-8ab8-d1d47e6502d3" containerName="istio-proxy"
Sep 15 10:39:23.003606 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.003590    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60fe0698-4c41-4f62-9389-f8b03cd9086a" containerName="maas-api"
Sep 15 10:39:23.003606 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.003595    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="60fe0698-4c41-4f62-9389-f8b03cd9086a" containerName="maas-api"
Sep 15 10:39:23.003763 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.003613    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2db6e97f-345b-4fa3-9b48-127cd97bb0d4" containerName="maas-api"
Sep 15 10:39:23.003763 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.003621    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="2db6e97f-345b-4fa3-9b48-127cd97bb0d4" containerName="maas-api"
Sep 15 10:39:23.003763 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.003689    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="2db6e97f-345b-4fa3-9b48-127cd97bb0d4" containerName="maas-api"
Sep 15 10:39:23.003763 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.003697    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="60fe0698-4c41-4f62-9389-f8b03cd9086a" containerName="maas-api"
Sep 15 10:39:23.003763 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.003708    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cead169-6453-4e87-8ab8-d1d47e6502d3" containerName="istio-proxy"
Sep 15 10:39:23.003763 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.003715    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cead169-6453-4e87-8ab8-d1d47e6502d3" containerName="istio-proxy"
Sep 15 10:39:23.003763 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.003721    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcedb166-03b2-448c-b397-2d8f44f799ba" containerName="main"
Sep 15 10:39:23.007112 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.007090    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:39:23.009909 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.009885    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-83e814-openshift-default-dockercfg-mq2j5\""
Sep 15 10:39:23.021233 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.021211    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"]
Sep 15 10:39:23.033142 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.033123    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:23.169694 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.169663    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f66d9141-89c8-40cf-a691-d8d19d2fd610-istiod-ca-cert\") pod \"e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") " pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:39:23.169886 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.169701    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f66d9141-89c8-40cf-a691-d8d19d2fd610-workload-socket\") pod \"e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") " pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:39:23.169886 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.169727    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f66d9141-89c8-40cf-a691-d8d19d2fd610-istio-data\") pod \"e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") " pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:39:23.169886 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.169752    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f66d9141-89c8-40cf-a691-d8d19d2fd610-istio-token\") pod \"e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") " pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:39:23.169886 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.169840    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f66d9141-89c8-40cf-a691-d8d19d2fd610-workload-certs\") pod \"e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") " pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:39:23.170103 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.169962    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnxjm\" (UniqueName: \"kubernetes.io/projected/f66d9141-89c8-40cf-a691-d8d19d2fd610-kube-api-access-qnxjm\") pod \"e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") " pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:39:23.170103 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.170007    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f66d9141-89c8-40cf-a691-d8d19d2fd610-credential-socket\") pod \"e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") " pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:39:23.170103 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.170051    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f66d9141-89c8-40cf-a691-d8d19d2fd610-istio-podinfo\") pod \"e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") " pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:39:23.170216 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.170127    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f66d9141-89c8-40cf-a691-d8d19d2fd610-istio-envoy\") pod \"e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") " pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:39:23.263207 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.263138    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:23.271467 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.271413    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f66d9141-89c8-40cf-a691-d8d19d2fd610-istio-envoy\") pod \"e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") " pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:39:23.271594 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.271476    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f66d9141-89c8-40cf-a691-d8d19d2fd610-istiod-ca-cert\") pod \"e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") " pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:39:23.271594 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.271511    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f66d9141-89c8-40cf-a691-d8d19d2fd610-workload-socket\") pod \"e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") " pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:39:23.271594 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.271543    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f66d9141-89c8-40cf-a691-d8d19d2fd610-istio-data\") pod \"e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") " pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:39:23.271776 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.271596    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f66d9141-89c8-40cf-a691-d8d19d2fd610-istio-token\") pod \"e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") " pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:39:23.271776 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.271761    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f66d9141-89c8-40cf-a691-d8d19d2fd610-workload-certs\") pod \"e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") " pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:39:23.271912 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.271895    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnxjm\" (UniqueName: \"kubernetes.io/projected/f66d9141-89c8-40cf-a691-d8d19d2fd610-kube-api-access-qnxjm\") pod \"e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") " pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:39:23.271966 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.271924    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f66d9141-89c8-40cf-a691-d8d19d2fd610-credential-socket\") pod \"e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") " pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:39:23.271966 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.271951    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f66d9141-89c8-40cf-a691-d8d19d2fd610-istio-podinfo\") pod \"e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") " pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:39:23.272073 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.272060    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f66d9141-89c8-40cf-a691-d8d19d2fd610-workload-certs\") pod \"e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") " pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:39:23.272135 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.272076    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f66d9141-89c8-40cf-a691-d8d19d2fd610-workload-socket\") pod \"e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") " pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:39:23.272190 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.272131    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f66d9141-89c8-40cf-a691-d8d19d2fd610-istio-data\") pod \"e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") " pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:39:23.272322 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.272298    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f66d9141-89c8-40cf-a691-d8d19d2fd610-credential-socket\") pod \"e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") " pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:39:23.272596 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.272580    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f66d9141-89c8-40cf-a691-d8d19d2fd610-istiod-ca-cert\") pod \"e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") " pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:39:23.273740 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.273718    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f66d9141-89c8-40cf-a691-d8d19d2fd610-istio-envoy\") pod \"e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") " pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:39:23.274216 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.274197    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f66d9141-89c8-40cf-a691-d8d19d2fd610-istio-podinfo\") pod \"e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") " pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:39:23.278929 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.278902    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f66d9141-89c8-40cf-a691-d8d19d2fd610-istio-token\") pod \"e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") " pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:39:23.279413 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.279397    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnxjm\" (UniqueName: \"kubernetes.io/projected/f66d9141-89c8-40cf-a691-d8d19d2fd610-kube-api-access-qnxjm\") pod \"e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") " pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:39:23.317701 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.317673    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:39:23.598295 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.598263    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"]
Sep 15 10:39:23.599345 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:39:23.599319    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf66d9141_89c8_40cf_a691_d8d19d2fd610.slice/crio-72ad55c07fb65c5948d012882a2efe123bfa52f09a66b61f7f9f82fa810da442 WatchSource:0}: Error finding container 72ad55c07fb65c5948d012882a2efe123bfa52f09a66b61f7f9f82fa810da442: Status 404 returned error can't find the container with id 72ad55c07fb65c5948d012882a2efe123bfa52f09a66b61f7f9f82fa810da442
Sep 15 10:39:23.601758 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.601727    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:39:23.601889 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.601793    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:39:23.601889 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:23.601822    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:39:24.351406 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:24.351366    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5" event={"ID":"f66d9141-89c8-40cf-a691-d8d19d2fd610","Type":"ContainerStarted","Data":"d33a3316bcfdd1aca57469f83ba9e4a5049a25578b8e905e562a8d98d8b54e6a"}
Sep 15 10:39:24.351876 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:24.351424    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5" event={"ID":"f66d9141-89c8-40cf-a691-d8d19d2fd610","Type":"ContainerStarted","Data":"72ad55c07fb65c5948d012882a2efe123bfa52f09a66b61f7f9f82fa810da442"}
Sep 15 10:39:24.373059 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:24.373003    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5" podStartSLOduration=2.37298734 podStartE2EDuration="2.37298734s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:39:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:39:24.371902631 +0000 UTC m=+1279.618869600" watchObservedRunningTime="2026-09-15 10:39:24.37298734 +0000 UTC m=+1279.619954309"
Sep 15 10:39:25.318310 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:25.318274    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:39:25.323552 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:25.323523    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:39:25.357066 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:25.357033    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:39:25.357930 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:25.357908    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:39:25.599851 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:25.599755    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:26.363627 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:26.363597    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:26.466400 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:26.466367    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:26.570656 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:26.570619    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:27.372497 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:27.372434    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:28.198841 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:28.198805    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:28.959965 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:28.959934    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:29.657525 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:29.657490    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:30.254415 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:30.254383    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9"]
Sep 15 10:39:30.254900 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:30.254884    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cead169-6453-4e87-8ab8-d1d47e6502d3" containerName="istio-proxy"
Sep 15 10:39:30.254954 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:30.254902    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cead169-6453-4e87-8ab8-d1d47e6502d3" containerName="istio-proxy"
Sep 15 10:39:30.258155 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:30.258136    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9"
Sep 15 10:39:30.260229 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:30.260200    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-83e814-serving-cert\""
Sep 15 10:39:30.268212 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:30.268184    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9"]
Sep 15 10:39:30.284452 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:30.284424    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:30.340141 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:30.340105    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4x24\" (UniqueName: \"kubernetes.io/projected/861885eb-eaf6-4765-b1e4-6820797dce8d-kube-api-access-p4x24\") pod \"maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9\" (UID: \"861885eb-eaf6-4765-b1e4-6820797dce8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9"
Sep 15 10:39:30.340316 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:30.340234    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/861885eb-eaf6-4765-b1e4-6820797dce8d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9\" (UID: \"861885eb-eaf6-4765-b1e4-6820797dce8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9"
Sep 15 10:39:30.340368 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:30.340330    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/861885eb-eaf6-4765-b1e4-6820797dce8d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9\" (UID: \"861885eb-eaf6-4765-b1e4-6820797dce8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9"
Sep 15 10:39:30.340368 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:30.340363    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/861885eb-eaf6-4765-b1e4-6820797dce8d-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9\" (UID: \"861885eb-eaf6-4765-b1e4-6820797dce8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9"
Sep 15 10:39:30.385209 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:30.385177    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:30.441490 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:30.441464    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4x24\" (UniqueName: \"kubernetes.io/projected/861885eb-eaf6-4765-b1e4-6820797dce8d-kube-api-access-p4x24\") pod \"maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9\" (UID: \"861885eb-eaf6-4765-b1e4-6820797dce8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9"
Sep 15 10:39:30.441646 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:30.441498    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/861885eb-eaf6-4765-b1e4-6820797dce8d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9\" (UID: \"861885eb-eaf6-4765-b1e4-6820797dce8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9"
Sep 15 10:39:30.441646 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:30.441548    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/861885eb-eaf6-4765-b1e4-6820797dce8d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9\" (UID: \"861885eb-eaf6-4765-b1e4-6820797dce8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9"
Sep 15 10:39:30.441646 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:30.441574    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/861885eb-eaf6-4765-b1e4-6820797dce8d-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9\" (UID: \"861885eb-eaf6-4765-b1e4-6820797dce8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9"
Sep 15 10:39:30.441780 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:39:30.441675    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-83e814-serving-cert: secret "maas-api-e2e-shared-a-w3-83e814-serving-cert" not found
Sep 15 10:39:30.441780 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:39:30.441759    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/861885eb-eaf6-4765-b1e4-6820797dce8d-maas-api-tls podName:861885eb-eaf6-4765-b1e4-6820797dce8d nodeName:}" failed. No retries permitted until 2026-09-15 10:39:30.941736493 +0000 UTC m=+1286.188703446 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/861885eb-eaf6-4765-b1e4-6820797dce8d-maas-api-tls") pod "maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9" (UID: "861885eb-eaf6-4765-b1e4-6820797dce8d") : secret "maas-api-e2e-shared-a-w3-83e814-serving-cert" not found
Sep 15 10:39:30.442137 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:30.442120    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/861885eb-eaf6-4765-b1e4-6820797dce8d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9\" (UID: \"861885eb-eaf6-4765-b1e4-6820797dce8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9"
Sep 15 10:39:30.443991 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:30.443973    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/861885eb-eaf6-4765-b1e4-6820797dce8d-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9\" (UID: \"861885eb-eaf6-4765-b1e4-6820797dce8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9"
Sep 15 10:39:30.452369 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:30.452345    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4x24\" (UniqueName: \"kubernetes.io/projected/861885eb-eaf6-4765-b1e4-6820797dce8d-kube-api-access-p4x24\") pod \"maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9\" (UID: \"861885eb-eaf6-4765-b1e4-6820797dce8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9"
Sep 15 10:39:30.661123 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:30.661085    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:30.819316 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:30.819289    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:30.918569 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:30.918494    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:30.947558 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:30.947528    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/861885eb-eaf6-4765-b1e4-6820797dce8d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9\" (UID: \"861885eb-eaf6-4765-b1e4-6820797dce8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9"
Sep 15 10:39:30.949769 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:30.949752    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/861885eb-eaf6-4765-b1e4-6820797dce8d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9\" (UID: \"861885eb-eaf6-4765-b1e4-6820797dce8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9"
Sep 15 10:39:31.171822 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:31.171735    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9"
Sep 15 10:39:31.449250 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:31.449224    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9"]
Sep 15 10:39:31.450058 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:39:31.450036    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod861885eb_eaf6_4765_b1e4_6820797dce8d.slice/crio-0780624a3680bbfe96e0d5b8bd40460398374c507af52d53860620731ab0ae69 WatchSource:0}: Error finding container 0780624a3680bbfe96e0d5b8bd40460398374c507af52d53860620731ab0ae69: Status 404 returned error can't find the container with id 0780624a3680bbfe96e0d5b8bd40460398374c507af52d53860620731ab0ae69
Sep 15 10:39:32.399111 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:32.399073    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9" event={"ID":"861885eb-eaf6-4765-b1e4-6820797dce8d","Type":"ContainerStarted","Data":"6e6bc17e93e675a2c41ea32baafaaef908156dd681752e87e1d7dfe043a59b37"}
Sep 15 10:39:32.399277 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:32.399133    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9" event={"ID":"861885eb-eaf6-4765-b1e4-6820797dce8d","Type":"ContainerStarted","Data":"0780624a3680bbfe96e0d5b8bd40460398374c507af52d53860620731ab0ae69"}
Sep 15 10:39:32.399322 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:32.399298    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9"
Sep 15 10:39:32.414300 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:32.414256    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9" podStartSLOduration=2.042752346 podStartE2EDuration="2.414239491s" totalImagesPullingTime="371.487145ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:39:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:39:32.413908018 +0000 UTC m=+1287.660874990" watchObservedRunningTime="2026-09-15 10:39:32.414239491 +0000 UTC m=+1287.661206461"
Sep 15 10:39:33.447090 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:33.447065    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:33.713969 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:33.713897    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:33.828411 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:33.828383    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:38.410453 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:38.410422    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9"
Sep 15 10:39:38.857430 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:38.857343    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:50.860729 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:50.860686    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl"]
Sep 15 10:39:50.865123 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:50.865104    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl"
Sep 15 10:39:50.868559 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:50.868523    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-8c1047\"/\"kube-root-ca.crt\""
Sep 15 10:39:50.868697 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:50.868568    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-8c1047\"/\"openshift-service-ca.crt\""
Sep 15 10:39:50.868697 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:50.868523    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-8c1047\"/\"distinct-2-w0-8c1047-kserve-self-signed-certs\""
Sep 15 10:39:50.868835 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:50.868821    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-8c1047\"/\"default-dockercfg-ff5nk\""
Sep 15 10:39:50.872125 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:50.872098    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl"]
Sep 15 10:39:50.884862 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:50.884837    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:50.940085 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:50.940058    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-model-cache\") pod \"distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl\" (UID: \"00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39\") " pod="e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl"
Sep 15 10:39:50.940196 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:50.940097    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-home\") pod \"distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl\" (UID: \"00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39\") " pod="e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl"
Sep 15 10:39:50.940196 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:50.940123    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g99b9\" (UniqueName: \"kubernetes.io/projected/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-kube-api-access-g99b9\") pod \"distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl\" (UID: \"00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39\") " pod="e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl"
Sep 15 10:39:50.940280 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:50.940230    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-tmp-dir\") pod \"distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl\" (UID: \"00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39\") " pod="e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl"
Sep 15 10:39:50.940325 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:50.940284    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-tls-certs\") pod \"distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl\" (UID: \"00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39\") " pod="e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl"
Sep 15 10:39:50.940365 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:50.940330    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-dshm\") pod \"distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl\" (UID: \"00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39\") " pod="e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl"
Sep 15 10:39:51.041158 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:51.041131    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-model-cache\") pod \"distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl\" (UID: \"00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39\") " pod="e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl"
Sep 15 10:39:51.041258 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:51.041169    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-home\") pod \"distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl\" (UID: \"00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39\") " pod="e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl"
Sep 15 10:39:51.041258 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:51.041208    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g99b9\" (UniqueName: \"kubernetes.io/projected/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-kube-api-access-g99b9\") pod \"distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl\" (UID: \"00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39\") " pod="e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl"
Sep 15 10:39:51.041376 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:51.041263    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-tmp-dir\") pod \"distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl\" (UID: \"00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39\") " pod="e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl"
Sep 15 10:39:51.041376 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:51.041296    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-tls-certs\") pod \"distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl\" (UID: \"00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39\") " pod="e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl"
Sep 15 10:39:51.041376 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:51.041323    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-dshm\") pod \"distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl\" (UID: \"00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39\") " pod="e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl"
Sep 15 10:39:51.041598 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:51.041574    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-model-cache\") pod \"distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl\" (UID: \"00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39\") " pod="e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl"
Sep 15 10:39:51.041700 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:51.041674    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-tmp-dir\") pod \"distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl\" (UID: \"00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39\") " pod="e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl"
Sep 15 10:39:51.041762 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:51.041747    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-home\") pod \"distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl\" (UID: \"00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39\") " pod="e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl"
Sep 15 10:39:51.043544 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:51.043524    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-dshm\") pod \"distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl\" (UID: \"00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39\") " pod="e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl"
Sep 15 10:39:51.043937 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:51.043921    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-tls-certs\") pod \"distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl\" (UID: \"00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39\") " pod="e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl"
Sep 15 10:39:51.048959 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:51.048940    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g99b9\" (UniqueName: \"kubernetes.io/projected/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-kube-api-access-g99b9\") pod \"distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl\" (UID: \"00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39\") " pod="e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl"
Sep 15 10:39:51.175998 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:51.175965    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl"
Sep 15 10:39:51.451483 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:51.451459    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl"]
Sep 15 10:39:51.453159 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:39:51.453130    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod00ae72c8_ea8a_42f1_89bf_dd83c3a8cb39.slice/crio-c838d8d28b60598b8d708a2a74cfb70917883429e7d684ca2f8492952236aa00 WatchSource:0}: Error finding container c838d8d28b60598b8d708a2a74cfb70917883429e7d684ca2f8492952236aa00: Status 404 returned error can't find the container with id c838d8d28b60598b8d708a2a74cfb70917883429e7d684ca2f8492952236aa00
Sep 15 10:39:51.508347 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:51.508323    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl" event={"ID":"00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39","Type":"ContainerStarted","Data":"c838d8d28b60598b8d708a2a74cfb70917883429e7d684ca2f8492952236aa00"}
Sep 15 10:39:52.180840 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:52.180809    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:52.517344 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:52.517246    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl" event={"ID":"00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39","Type":"ContainerStarted","Data":"528a3a791c156a061716d3262199adc10e0f3163374f7b940a94611eb6b0c990"}
Sep 15 10:39:52.535718 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:52.535643    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl" podStartSLOduration=2.535627089 podStartE2EDuration="2.535627089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:39:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:39:52.534772775 +0000 UTC m=+1307.781739744" watchObservedRunningTime="2026-09-15 10:39:52.535627089 +0000 UTC m=+1307.782594059"
Sep 15 10:39:53.002316 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:53.002281    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:53.600633 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:53.600603    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:54.209296 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:54.209265    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:54.969059 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:54.969031    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:55.072933 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:55.072900    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:55.249070 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:55.248984    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:55.568523 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:55.568417    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:56.291345 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:56.291315    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:56.464507 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:56.464461    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:56.845471 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:56.845407    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:57.167468 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:57.167374    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:57.369629 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:57.369597    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:39:59.866999 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:39:59.866962    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:00.587588 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:00.587546    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:00.833000 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:00.832971    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:00.931991 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:00.931924    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:01.176856 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:01.176820    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl"
Sep 15 10:40:01.177056 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:01.176878    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl"
Sep 15 10:40:01.189258 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:01.189185    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl"
Sep 15 10:40:01.280178 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:01.280143    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:01.579723 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:01.579633    2583 generic.go:356] "Generic (PLEG): container finished" podID="f66d9141-89c8-40cf-a691-d8d19d2fd610" containerID="d33a3316bcfdd1aca57469f83ba9e4a5049a25578b8e905e562a8d98d8b54e6a" exitCode=0
Sep 15 10:40:01.579723 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:01.579703    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5" event={"ID":"f66d9141-89c8-40cf-a691-d8d19d2fd610","Type":"ContainerDied","Data":"d33a3316bcfdd1aca57469f83ba9e4a5049a25578b8e905e562a8d98d8b54e6a"}
Sep 15 10:40:01.580142 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:01.580123    2583 scope.go:117] "RemoveContainer" containerID="d33a3316bcfdd1aca57469f83ba9e4a5049a25578b8e905e562a8d98d8b54e6a"
Sep 15 10:40:01.581373 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:01.581337    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:40:01.581480 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:01.581411    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:40:01.581480 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:01.581469    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:40:01.591674 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:01.591651    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl"
Sep 15 10:40:02.380616 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:02.380567    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:02.587611 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:02.587578    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5" event={"ID":"f66d9141-89c8-40cf-a691-d8d19d2fd610","Type":"ContainerStarted","Data":"f4b8edc426824e00fafd06d72a745586292378c7bed432e91298ef7130013fd4"}
Sep 15 10:40:02.911768 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:02.911729    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:03.318964 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:03.318877    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:40:03.319133 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:03.319089    2583 patch_prober.go:28] interesting pod/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 15 10:40:03.319179 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:03.319131    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5" podUID="f66d9141-89c8-40cf-a691-d8d19d2fd610" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 15 10:40:03.580358 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:03.580240    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:04.290261 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:04.290229    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:04.318520 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:04.318490    2583 patch_prober.go:28] interesting pod/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 15 10:40:04.318670 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:04.318550    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5" podUID="f66d9141-89c8-40cf-a691-d8d19d2fd610" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 15 10:40:04.886071 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:04.886040    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:05.318492 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:05.318400    2583 patch_prober.go:28] interesting pod/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 15 10:40:05.318492 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:05.318469    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5" podUID="f66d9141-89c8-40cf-a691-d8d19d2fd610" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 15 10:40:05.496681 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:05.496650    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:05.598963 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:05.598876    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:05.955105 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:05.955063    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:06.059668 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:06.057062    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:06.128712 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:06.128683    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:06.318583 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:06.318495    2583 patch_prober.go:28] interesting pod/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 15 10:40:06.318583 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:06.318547    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5" podUID="f66d9141-89c8-40cf-a691-d8d19d2fd610" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 15 10:40:07.318214 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:07.318183    2583 patch_prober.go:28] interesting pod/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 15 10:40:07.318617 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:07.318235    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5" podUID="f66d9141-89c8-40cf-a691-d8d19d2fd610" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 15 10:40:08.319027 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:08.318992    2583 patch_prober.go:28] interesting pod/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 15 10:40:08.319401 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:08.319044    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5" podUID="f66d9141-89c8-40cf-a691-d8d19d2fd610" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 15 10:40:09.322160 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:09.322132    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:40:09.322567 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:09.322380    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:40:09.323124 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:09.323104    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:40:09.681605 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:09.681580    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:15.194769 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:15.194734    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6111ecc0-5985b5684-lxz65"]
Sep 15 10:40:15.195119 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:15.194974    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6111ecc0-5985b5684-lxz65" podUID="58e0a2c3-78bf-41e6-85ea-9101d6de94e1" containerName="maas-api" containerID="cri-o://e1b41bf4382614da369424e80fa83dc54d2f6b4517e1f226da2810aef2a2dfcb" gracePeriod=30
Sep 15 10:40:15.219102 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:15.219075    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:15.286713 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:15.286681    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:15.391065 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:15.391035    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:15.436383 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:15.436362    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6111ecc0-5985b5684-lxz65"
Sep 15 10:40:15.496368 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:15.496111    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tql6b\" (UniqueName: \"kubernetes.io/projected/58e0a2c3-78bf-41e6-85ea-9101d6de94e1-kube-api-access-tql6b\") pod \"58e0a2c3-78bf-41e6-85ea-9101d6de94e1\" (UID: \"58e0a2c3-78bf-41e6-85ea-9101d6de94e1\") "
Sep 15 10:40:15.496368 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:15.496154    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58e0a2c3-78bf-41e6-85ea-9101d6de94e1-maas-api-tls\") pod \"58e0a2c3-78bf-41e6-85ea-9101d6de94e1\" (UID: \"58e0a2c3-78bf-41e6-85ea-9101d6de94e1\") "
Sep 15 10:40:15.496368 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:15.496243    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58e0a2c3-78bf-41e6-85ea-9101d6de94e1-metrics-certs\") pod \"58e0a2c3-78bf-41e6-85ea-9101d6de94e1\" (UID: \"58e0a2c3-78bf-41e6-85ea-9101d6de94e1\") "
Sep 15 10:40:15.496368 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:15.496335    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58e0a2c3-78bf-41e6-85ea-9101d6de94e1-openshift-service-ca\") pod \"58e0a2c3-78bf-41e6-85ea-9101d6de94e1\" (UID: \"58e0a2c3-78bf-41e6-85ea-9101d6de94e1\") "
Sep 15 10:40:15.496758 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:15.496708    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:15.497279 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:15.497253    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58e0a2c3-78bf-41e6-85ea-9101d6de94e1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "58e0a2c3-78bf-41e6-85ea-9101d6de94e1" (UID: "58e0a2c3-78bf-41e6-85ea-9101d6de94e1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:40:15.499189 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:15.499162    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58e0a2c3-78bf-41e6-85ea-9101d6de94e1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "58e0a2c3-78bf-41e6-85ea-9101d6de94e1" (UID: "58e0a2c3-78bf-41e6-85ea-9101d6de94e1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:40:15.499189 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:15.499180    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58e0a2c3-78bf-41e6-85ea-9101d6de94e1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "58e0a2c3-78bf-41e6-85ea-9101d6de94e1" (UID: "58e0a2c3-78bf-41e6-85ea-9101d6de94e1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:40:15.499519 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:15.499492    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58e0a2c3-78bf-41e6-85ea-9101d6de94e1-kube-api-access-tql6b" (OuterVolumeSpecName: "kube-api-access-tql6b") pod "58e0a2c3-78bf-41e6-85ea-9101d6de94e1" (UID: "58e0a2c3-78bf-41e6-85ea-9101d6de94e1"). InnerVolumeSpecName "kube-api-access-tql6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:40:15.598318 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:15.598283    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58e0a2c3-78bf-41e6-85ea-9101d6de94e1-openshift-service-ca\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:40:15.598318 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:15.598319    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tql6b\" (UniqueName: \"kubernetes.io/projected/58e0a2c3-78bf-41e6-85ea-9101d6de94e1-kube-api-access-tql6b\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:40:15.598537 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:15.598332    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58e0a2c3-78bf-41e6-85ea-9101d6de94e1-maas-api-tls\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:40:15.598537 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:15.598344    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58e0a2c3-78bf-41e6-85ea-9101d6de94e1-metrics-certs\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:40:15.665346 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:15.665305    2583 generic.go:356] "Generic (PLEG): container finished" podID="58e0a2c3-78bf-41e6-85ea-9101d6de94e1" containerID="e1b41bf4382614da369424e80fa83dc54d2f6b4517e1f226da2810aef2a2dfcb" exitCode=0
Sep 15 10:40:15.665549 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:15.665362    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6111ecc0-5985b5684-lxz65"
Sep 15 10:40:15.665549 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:15.665405    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6111ecc0-5985b5684-lxz65" event={"ID":"58e0a2c3-78bf-41e6-85ea-9101d6de94e1","Type":"ContainerDied","Data":"e1b41bf4382614da369424e80fa83dc54d2f6b4517e1f226da2810aef2a2dfcb"}
Sep 15 10:40:15.665549 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:15.665485    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6111ecc0-5985b5684-lxz65" event={"ID":"58e0a2c3-78bf-41e6-85ea-9101d6de94e1","Type":"ContainerDied","Data":"388038bb0f6cad837a1364ba11b0f4a5261f6161ca11476db7b7c7033d59ea56"}
Sep 15 10:40:15.665549 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:15.665511    2583 scope.go:117] "RemoveContainer" containerID="e1b41bf4382614da369424e80fa83dc54d2f6b4517e1f226da2810aef2a2dfcb"
Sep 15 10:40:15.675209 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:15.675194    2583 scope.go:117] "RemoveContainer" containerID="e1b41bf4382614da369424e80fa83dc54d2f6b4517e1f226da2810aef2a2dfcb"
Sep 15 10:40:15.675469 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:40:15.675430    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1b41bf4382614da369424e80fa83dc54d2f6b4517e1f226da2810aef2a2dfcb\": container with ID starting with e1b41bf4382614da369424e80fa83dc54d2f6b4517e1f226da2810aef2a2dfcb not found: ID does not exist" containerID="e1b41bf4382614da369424e80fa83dc54d2f6b4517e1f226da2810aef2a2dfcb"
Sep 15 10:40:15.675539 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:15.675470    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1b41bf4382614da369424e80fa83dc54d2f6b4517e1f226da2810aef2a2dfcb"} err="failed to get container status \"e1b41bf4382614da369424e80fa83dc54d2f6b4517e1f226da2810aef2a2dfcb\": rpc error: code = NotFound desc = could not find container \"e1b41bf4382614da369424e80fa83dc54d2f6b4517e1f226da2810aef2a2dfcb\": container with ID starting with e1b41bf4382614da369424e80fa83dc54d2f6b4517e1f226da2810aef2a2dfcb not found: ID does not exist"
Sep 15 10:40:15.687099 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:15.687080    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6111ecc0-5985b5684-lxz65"]
Sep 15 10:40:15.693197 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:15.693177    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6111ecc0-5985b5684-lxz65"]
Sep 15 10:40:15.811630 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:15.811533    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:16.287817 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:16.287784    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:16.603547 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:16.603435    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:17.300941 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:17.300905    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58e0a2c3-78bf-41e6-85ea-9101d6de94e1" path="/var/lib/kubelet/pods/58e0a2c3-78bf-41e6-85ea-9101d6de94e1/volumes"
Sep 15 10:40:17.301534 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:17.301514    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:17.982398 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:17.982337    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:18.326298 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:18.326219    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:18.569457 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:18.569412    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:19.184002 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:19.183969    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:19.869604 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:19.869574    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:20.521630 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:20.521560    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:20.623023 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:20.622993    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:21.189635 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:21.189597    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:22.086857 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:22.086817    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:22.788927 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:22.788890    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:23.310472 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:23.310428    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf"]
Sep 15 10:40:23.351244 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:23.351220    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:23.411087 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:23.411062    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:24.072846 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:24.072809    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:24.848607 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:24.848580    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:26.482344 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:26.482311    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:35.257635 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:35.257541    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:38.330970 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:38.330927    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf" podUID="1b91056a-8d16-41e1-a9e3-589d7482133d" containerName="main" containerID="cri-o://786c4032de10f31162099932ba5aba06ca0c922278ff23546c40c7c0b358478a" gracePeriod=45
Sep 15 10:40:38.578987 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:38.578964    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf"
Sep 15 10:40:38.720004 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:38.719975    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b91056a-8d16-41e1-a9e3-589d7482133d-home\") pod \"1b91056a-8d16-41e1-a9e3-589d7482133d\" (UID: \"1b91056a-8d16-41e1-a9e3-589d7482133d\") "
Sep 15 10:40:38.720178 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:38.720067    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b91056a-8d16-41e1-a9e3-589d7482133d-tls-certs\") pod \"1b91056a-8d16-41e1-a9e3-589d7482133d\" (UID: \"1b91056a-8d16-41e1-a9e3-589d7482133d\") "
Sep 15 10:40:38.720178 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:38.720091    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b91056a-8d16-41e1-a9e3-589d7482133d-tmp-dir\") pod \"1b91056a-8d16-41e1-a9e3-589d7482133d\" (UID: \"1b91056a-8d16-41e1-a9e3-589d7482133d\") "
Sep 15 10:40:38.720178 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:38.720128    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b91056a-8d16-41e1-a9e3-589d7482133d-dshm\") pod \"1b91056a-8d16-41e1-a9e3-589d7482133d\" (UID: \"1b91056a-8d16-41e1-a9e3-589d7482133d\") "
Sep 15 10:40:38.720415 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:38.720237    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b91056a-8d16-41e1-a9e3-589d7482133d-home" (OuterVolumeSpecName: "home") pod "1b91056a-8d16-41e1-a9e3-589d7482133d" (UID: "1b91056a-8d16-41e1-a9e3-589d7482133d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:40:38.720415 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:38.720266    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjsqc\" (UniqueName: \"kubernetes.io/projected/1b91056a-8d16-41e1-a9e3-589d7482133d-kube-api-access-kjsqc\") pod \"1b91056a-8d16-41e1-a9e3-589d7482133d\" (UID: \"1b91056a-8d16-41e1-a9e3-589d7482133d\") "
Sep 15 10:40:38.720415 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:38.720330    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b91056a-8d16-41e1-a9e3-589d7482133d-model-cache\") pod \"1b91056a-8d16-41e1-a9e3-589d7482133d\" (UID: \"1b91056a-8d16-41e1-a9e3-589d7482133d\") "
Sep 15 10:40:38.720415 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:38.720362    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b91056a-8d16-41e1-a9e3-589d7482133d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1b91056a-8d16-41e1-a9e3-589d7482133d" (UID: "1b91056a-8d16-41e1-a9e3-589d7482133d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:40:38.720660 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:38.720639    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b91056a-8d16-41e1-a9e3-589d7482133d-model-cache" (OuterVolumeSpecName: "model-cache") pod "1b91056a-8d16-41e1-a9e3-589d7482133d" (UID: "1b91056a-8d16-41e1-a9e3-589d7482133d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:40:38.720784 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:38.720764    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b91056a-8d16-41e1-a9e3-589d7482133d-tmp-dir\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:40:38.720833 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:38.720789    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b91056a-8d16-41e1-a9e3-589d7482133d-model-cache\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:40:38.720833 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:38.720802    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b91056a-8d16-41e1-a9e3-589d7482133d-home\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:40:38.722374 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:38.722343    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b91056a-8d16-41e1-a9e3-589d7482133d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1b91056a-8d16-41e1-a9e3-589d7482133d" (UID: "1b91056a-8d16-41e1-a9e3-589d7482133d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:40:38.722374 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:38.722368    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b91056a-8d16-41e1-a9e3-589d7482133d-dshm" (OuterVolumeSpecName: "dshm") pod "1b91056a-8d16-41e1-a9e3-589d7482133d" (UID: "1b91056a-8d16-41e1-a9e3-589d7482133d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:40:38.722552 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:38.722387    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b91056a-8d16-41e1-a9e3-589d7482133d-kube-api-access-kjsqc" (OuterVolumeSpecName: "kube-api-access-kjsqc") pod "1b91056a-8d16-41e1-a9e3-589d7482133d" (UID: "1b91056a-8d16-41e1-a9e3-589d7482133d"). InnerVolumeSpecName "kube-api-access-kjsqc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:40:38.803894 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:38.803867    2583 generic.go:356] "Generic (PLEG): container finished" podID="1b91056a-8d16-41e1-a9e3-589d7482133d" containerID="786c4032de10f31162099932ba5aba06ca0c922278ff23546c40c7c0b358478a" exitCode=0
Sep 15 10:40:38.804030 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:38.803903    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf"
Sep 15 10:40:38.804030 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:38.803924    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf" event={"ID":"1b91056a-8d16-41e1-a9e3-589d7482133d","Type":"ContainerDied","Data":"786c4032de10f31162099932ba5aba06ca0c922278ff23546c40c7c0b358478a"}
Sep 15 10:40:38.804030 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:38.803965    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf" event={"ID":"1b91056a-8d16-41e1-a9e3-589d7482133d","Type":"ContainerDied","Data":"06c0454dab6e4c80ec1667fbf90c54cb947f24a3eb1c7c3a4ba5210db32df4e8"}
Sep 15 10:40:38.804030 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:38.803988    2583 scope.go:117] "RemoveContainer" containerID="786c4032de10f31162099932ba5aba06ca0c922278ff23546c40c7c0b358478a"
Sep 15 10:40:38.813004 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:38.812988    2583 scope.go:117] "RemoveContainer" containerID="786c4032de10f31162099932ba5aba06ca0c922278ff23546c40c7c0b358478a"
Sep 15 10:40:38.813238 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:40:38.813216    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"786c4032de10f31162099932ba5aba06ca0c922278ff23546c40c7c0b358478a\": container with ID starting with 786c4032de10f31162099932ba5aba06ca0c922278ff23546c40c7c0b358478a not found: ID does not exist" containerID="786c4032de10f31162099932ba5aba06ca0c922278ff23546c40c7c0b358478a"
Sep 15 10:40:38.813341 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:38.813242    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"786c4032de10f31162099932ba5aba06ca0c922278ff23546c40c7c0b358478a"} err="failed to get container status \"786c4032de10f31162099932ba5aba06ca0c922278ff23546c40c7c0b358478a\": rpc error: code = NotFound desc = could not find container \"786c4032de10f31162099932ba5aba06ca0c922278ff23546c40c7c0b358478a\": container with ID starting with 786c4032de10f31162099932ba5aba06ca0c922278ff23546c40c7c0b358478a not found: ID does not exist"
Sep 15 10:40:38.821498 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:38.821481    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b91056a-8d16-41e1-a9e3-589d7482133d-tls-certs\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:40:38.821498 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:38.821499    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b91056a-8d16-41e1-a9e3-589d7482133d-dshm\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:40:38.821591 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:38.821508    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjsqc\" (UniqueName: \"kubernetes.io/projected/1b91056a-8d16-41e1-a9e3-589d7482133d-kube-api-access-kjsqc\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:40:38.825834 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:38.825809    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf"]
Sep 15 10:40:38.828433 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:38.828413    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-6111ecc0/e2e-lifecycle-model-6111ecc0-kserve-5cb64fb547-x62bf"]
Sep 15 10:40:39.300949 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:39.300916    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b91056a-8d16-41e1-a9e3-589d7482133d" path="/var/lib/kubelet/pods/1b91056a-8d16-41e1-a9e3-589d7482133d/volumes"
Sep 15 10:40:42.926076 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:42.926040    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:45.755342 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:45.755311    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv"]
Sep 15 10:40:45.755767 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:45.755753    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b91056a-8d16-41e1-a9e3-589d7482133d" containerName="main"
Sep 15 10:40:45.755811 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:45.755768    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b91056a-8d16-41e1-a9e3-589d7482133d" containerName="main"
Sep 15 10:40:45.755811 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:45.755782    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58e0a2c3-78bf-41e6-85ea-9101d6de94e1" containerName="maas-api"
Sep 15 10:40:45.755811 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:45.755788    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="58e0a2c3-78bf-41e6-85ea-9101d6de94e1" containerName="maas-api"
Sep 15 10:40:45.755907 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:45.755843    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b91056a-8d16-41e1-a9e3-589d7482133d" containerName="main"
Sep 15 10:40:45.755907 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:45.755854    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="58e0a2c3-78bf-41e6-85ea-9101d6de94e1" containerName="maas-api"
Sep 15 10:40:45.760078 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:45.760060    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv"
Sep 15 10:40:45.762665 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:45.762642    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-83e814\"/\"openshift-service-ca.crt\""
Sep 15 10:40:45.763360 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:45.763342    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-83e814\"/\"default-dockercfg-84k6n\""
Sep 15 10:40:45.763470 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:45.763382    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-83e814\"/\"auth-test-model-83e814-kserve-self-signed-certs\""
Sep 15 10:40:45.763470 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:45.763384    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-83e814\"/\"kube-root-ca.crt\""
Sep 15 10:40:45.769095 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:45.768861    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv"]
Sep 15 10:40:45.780265 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:45.780247    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:45.786452 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:45.786412    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96039a67-84c4-4287-acbf-88425dca4d3b-home\") pod \"auth-test-model-83e814-kserve-5bc975dbdc-wzpzv\" (UID: \"96039a67-84c4-4287-acbf-88425dca4d3b\") " pod="ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv"
Sep 15 10:40:45.786555 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:45.786488    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96039a67-84c4-4287-acbf-88425dca4d3b-tmp-dir\") pod \"auth-test-model-83e814-kserve-5bc975dbdc-wzpzv\" (UID: \"96039a67-84c4-4287-acbf-88425dca4d3b\") " pod="ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv"
Sep 15 10:40:45.786555 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:45.786530    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96039a67-84c4-4287-acbf-88425dca4d3b-tls-certs\") pod \"auth-test-model-83e814-kserve-5bc975dbdc-wzpzv\" (UID: \"96039a67-84c4-4287-acbf-88425dca4d3b\") " pod="ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv"
Sep 15 10:40:45.786667 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:45.786611    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96039a67-84c4-4287-acbf-88425dca4d3b-dshm\") pod \"auth-test-model-83e814-kserve-5bc975dbdc-wzpzv\" (UID: \"96039a67-84c4-4287-acbf-88425dca4d3b\") " pod="ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv"
Sep 15 10:40:45.786713 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:45.786663    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96039a67-84c4-4287-acbf-88425dca4d3b-model-cache\") pod \"auth-test-model-83e814-kserve-5bc975dbdc-wzpzv\" (UID: \"96039a67-84c4-4287-acbf-88425dca4d3b\") " pod="ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv"
Sep 15 10:40:45.786755 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:45.786725    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzfjz\" (UniqueName: \"kubernetes.io/projected/96039a67-84c4-4287-acbf-88425dca4d3b-kube-api-access-nzfjz\") pod \"auth-test-model-83e814-kserve-5bc975dbdc-wzpzv\" (UID: \"96039a67-84c4-4287-acbf-88425dca4d3b\") " pod="ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv"
Sep 15 10:40:45.887931 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:45.887891    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96039a67-84c4-4287-acbf-88425dca4d3b-home\") pod \"auth-test-model-83e814-kserve-5bc975dbdc-wzpzv\" (UID: \"96039a67-84c4-4287-acbf-88425dca4d3b\") " pod="ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv"
Sep 15 10:40:45.888062 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:45.887942    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96039a67-84c4-4287-acbf-88425dca4d3b-tmp-dir\") pod \"auth-test-model-83e814-kserve-5bc975dbdc-wzpzv\" (UID: \"96039a67-84c4-4287-acbf-88425dca4d3b\") " pod="ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv"
Sep 15 10:40:45.888062 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:45.887971    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96039a67-84c4-4287-acbf-88425dca4d3b-tls-certs\") pod \"auth-test-model-83e814-kserve-5bc975dbdc-wzpzv\" (UID: \"96039a67-84c4-4287-acbf-88425dca4d3b\") " pod="ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv"
Sep 15 10:40:45.888062 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:45.888000    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96039a67-84c4-4287-acbf-88425dca4d3b-dshm\") pod \"auth-test-model-83e814-kserve-5bc975dbdc-wzpzv\" (UID: \"96039a67-84c4-4287-acbf-88425dca4d3b\") " pod="ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv"
Sep 15 10:40:45.888062 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:45.888023    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96039a67-84c4-4287-acbf-88425dca4d3b-model-cache\") pod \"auth-test-model-83e814-kserve-5bc975dbdc-wzpzv\" (UID: \"96039a67-84c4-4287-acbf-88425dca4d3b\") " pod="ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv"
Sep 15 10:40:45.888223 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:45.888088    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzfjz\" (UniqueName: \"kubernetes.io/projected/96039a67-84c4-4287-acbf-88425dca4d3b-kube-api-access-nzfjz\") pod \"auth-test-model-83e814-kserve-5bc975dbdc-wzpzv\" (UID: \"96039a67-84c4-4287-acbf-88425dca4d3b\") " pod="ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv"
Sep 15 10:40:45.888277 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:45.888261    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96039a67-84c4-4287-acbf-88425dca4d3b-home\") pod \"auth-test-model-83e814-kserve-5bc975dbdc-wzpzv\" (UID: \"96039a67-84c4-4287-acbf-88425dca4d3b\") " pod="ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv"
Sep 15 10:40:45.888377 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:45.888358    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96039a67-84c4-4287-acbf-88425dca4d3b-tmp-dir\") pod \"auth-test-model-83e814-kserve-5bc975dbdc-wzpzv\" (UID: \"96039a67-84c4-4287-acbf-88425dca4d3b\") " pod="ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv"
Sep 15 10:40:45.888496 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:45.888477    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96039a67-84c4-4287-acbf-88425dca4d3b-model-cache\") pod \"auth-test-model-83e814-kserve-5bc975dbdc-wzpzv\" (UID: \"96039a67-84c4-4287-acbf-88425dca4d3b\") " pod="ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv"
Sep 15 10:40:45.890263 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:45.890242    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96039a67-84c4-4287-acbf-88425dca4d3b-dshm\") pod \"auth-test-model-83e814-kserve-5bc975dbdc-wzpzv\" (UID: \"96039a67-84c4-4287-acbf-88425dca4d3b\") " pod="ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv"
Sep 15 10:40:45.890578 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:45.890554    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96039a67-84c4-4287-acbf-88425dca4d3b-tls-certs\") pod \"auth-test-model-83e814-kserve-5bc975dbdc-wzpzv\" (UID: \"96039a67-84c4-4287-acbf-88425dca4d3b\") " pod="ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv"
Sep 15 10:40:45.896116 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:45.896096    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzfjz\" (UniqueName: \"kubernetes.io/projected/96039a67-84c4-4287-acbf-88425dca4d3b-kube-api-access-nzfjz\") pod \"auth-test-model-83e814-kserve-5bc975dbdc-wzpzv\" (UID: \"96039a67-84c4-4287-acbf-88425dca4d3b\") " pod="ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv"
Sep 15 10:40:46.071304 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:46.071217    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv"
Sep 15 10:40:46.343224 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:46.343197    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv"]
Sep 15 10:40:46.344431 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:40:46.344406    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod96039a67_84c4_4287_acbf_88425dca4d3b.slice/crio-c1c50662f46ba50255318883ac870efc458700d2b2745c1d0e022d82f7643af4 WatchSource:0}: Error finding container c1c50662f46ba50255318883ac870efc458700d2b2745c1d0e022d82f7643af4: Status 404 returned error can't find the container with id c1c50662f46ba50255318883ac870efc458700d2b2745c1d0e022d82f7643af4
Sep 15 10:40:46.401702 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:46.401662    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:46.855947 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:46.855911    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv" event={"ID":"96039a67-84c4-4287-acbf-88425dca4d3b","Type":"ContainerStarted","Data":"3d37dad555cd4f3ebe749c9454fb573209aa493dc079800581505c5f3e35ccbd"}
Sep 15 10:40:46.856303 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:46.855961    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv" event={"ID":"96039a67-84c4-4287-acbf-88425dca4d3b","Type":"ContainerStarted","Data":"c1c50662f46ba50255318883ac870efc458700d2b2745c1d0e022d82f7643af4"}
Sep 15 10:40:46.872784 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:46.872739    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv" podStartSLOduration=1.872724753 podStartE2EDuration="1.872724753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:40:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:40:46.869913193 +0000 UTC m=+1362.116880164" watchObservedRunningTime="2026-09-15 10:40:46.872724753 +0000 UTC m=+1362.119691745"
Sep 15 10:40:49.271137 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:49.271086    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:40:56.071594 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:56.071560    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv"
Sep 15 10:40:56.071594 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:56.071601    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv"
Sep 15 10:40:56.084015 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:56.083992    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv"
Sep 15 10:40:56.928002 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:40:56.927973    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv"
Sep 15 10:41:00.442788 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:00.442759    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:41:01.140055 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:01.140025    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:41:06.684414 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:06.684380    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:41:09.575002 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:09.574971    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:41:41.478919 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:41.478860    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:41:42.598344 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:42.598292    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:41:46.595633 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:46.595590    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:41:46.696379 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:46.696331    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:41:46.819269 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:46.819235    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"]
Sep 15 10:41:46.819495 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:46.819467    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw" podUID="ac3cc4d8-ad26-4bfa-8690-dad507e6e055" containerName="istio-proxy" containerID="cri-o://713269bb8818f7d7368b2ee616ea97dd61d7711e7dbdc4a77930134f25b41c59" gracePeriod=30
Sep 15 10:41:46.845190 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:46.845156    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:41:48.038430 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:48.038386    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:41:48.136957 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:48.136920    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:41:48.238605 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:48.238538    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:41:48.785815 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:48.785779    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:41:48.885096 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:48.885061    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:41:49.479976 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:49.479947    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:41:50.278042 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:50.278015    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:41:51.802986 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:51.802954    2583 patch_prober.go:28] interesting pod/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 10:41:51.803359 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:51.803020    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw" podUID="ac3cc4d8-ad26-4bfa-8690-dad507e6e055" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 10:41:52.080038 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.080016    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:41:52.142273 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.142245    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-istio-token\") pod \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") "
Sep 15 10:41:52.142403 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.142299    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-workload-socket\") pod \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") "
Sep 15 10:41:52.142403 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.142324    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-istio-data\") pod \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") "
Sep 15 10:41:52.142556 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.142431    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-istio-podinfo\") pod \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") "
Sep 15 10:41:52.142556 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.142528    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-istio-envoy\") pod \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") "
Sep 15 10:41:52.142675 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.142597    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-istiod-ca-cert\") pod \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") "
Sep 15 10:41:52.142675 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.142609    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-istio-data" (OuterVolumeSpecName: "istio-data") pod "ac3cc4d8-ad26-4bfa-8690-dad507e6e055" (UID: "ac3cc4d8-ad26-4bfa-8690-dad507e6e055"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:41:52.142675 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.142616    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ac3cc4d8-ad26-4bfa-8690-dad507e6e055" (UID: "ac3cc4d8-ad26-4bfa-8690-dad507e6e055"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:41:52.142675 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.142624    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffhs8\" (UniqueName: \"kubernetes.io/projected/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-kube-api-access-ffhs8\") pod \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") "
Sep 15 10:41:52.142898 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.142681    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-credential-socket\") pod \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") "
Sep 15 10:41:52.142898 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.142714    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-workload-certs\") pod \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\" (UID: \"ac3cc4d8-ad26-4bfa-8690-dad507e6e055\") "
Sep 15 10:41:52.143003 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.142929    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ac3cc4d8-ad26-4bfa-8690-dad507e6e055" (UID: "ac3cc4d8-ad26-4bfa-8690-dad507e6e055"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:41:52.143003 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.142939    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ac3cc4d8-ad26-4bfa-8690-dad507e6e055" (UID: "ac3cc4d8-ad26-4bfa-8690-dad507e6e055"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:41:52.143165 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.143113    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ac3cc4d8-ad26-4bfa-8690-dad507e6e055" (UID: "ac3cc4d8-ad26-4bfa-8690-dad507e6e055"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:41:52.143264 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.143248    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-istiod-ca-cert\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:41:52.143330 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.143264    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-credential-socket\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:41:52.143330 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.143273    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-workload-certs\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:41:52.143330 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.143283    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-workload-socket\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:41:52.143330 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.143292    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-istio-data\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:41:52.144702 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.144679    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ac3cc4d8-ad26-4bfa-8690-dad507e6e055" (UID: "ac3cc4d8-ad26-4bfa-8690-dad507e6e055"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 10:41:52.144790 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.144701    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ac3cc4d8-ad26-4bfa-8690-dad507e6e055" (UID: "ac3cc4d8-ad26-4bfa-8690-dad507e6e055"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:41:52.144918 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.144901    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-istio-token" (OuterVolumeSpecName: "istio-token") pod "ac3cc4d8-ad26-4bfa-8690-dad507e6e055" (UID: "ac3cc4d8-ad26-4bfa-8690-dad507e6e055"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:41:52.144968 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.144933    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-kube-api-access-ffhs8" (OuterVolumeSpecName: "kube-api-access-ffhs8") pod "ac3cc4d8-ad26-4bfa-8690-dad507e6e055" (UID: "ac3cc4d8-ad26-4bfa-8690-dad507e6e055"). InnerVolumeSpecName "kube-api-access-ffhs8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:41:52.244027 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.243999    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-istio-token\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:41:52.244027 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.244024    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-istio-podinfo\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:41:52.244027 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.244033    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-istio-envoy\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:41:52.244236 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.244043    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffhs8\" (UniqueName: \"kubernetes.io/projected/ac3cc4d8-ad26-4bfa-8690-dad507e6e055-kube-api-access-ffhs8\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:41:52.258629 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.258594    2583 generic.go:356] "Generic (PLEG): container finished" podID="ac3cc4d8-ad26-4bfa-8690-dad507e6e055" containerID="713269bb8818f7d7368b2ee616ea97dd61d7711e7dbdc4a77930134f25b41c59" exitCode=0
Sep 15 10:41:52.258768 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.258646    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"
Sep 15 10:41:52.258768 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.258673    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw" event={"ID":"ac3cc4d8-ad26-4bfa-8690-dad507e6e055","Type":"ContainerDied","Data":"713269bb8818f7d7368b2ee616ea97dd61d7711e7dbdc4a77930134f25b41c59"}
Sep 15 10:41:52.258768 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.258729    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw" event={"ID":"ac3cc4d8-ad26-4bfa-8690-dad507e6e055","Type":"ContainerDied","Data":"99055d120761b8bd884dc758db8890d7525a568dddf08640a28951c2b4000445"}
Sep 15 10:41:52.258768 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.258752    2583 scope.go:117] "RemoveContainer" containerID="713269bb8818f7d7368b2ee616ea97dd61d7711e7dbdc4a77930134f25b41c59"
Sep 15 10:41:52.268746 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.268729    2583 scope.go:117] "RemoveContainer" containerID="b11d7ac7ec2b3856c2941bdffb010bbf28295187a85e5c9e1cc021cb63e0c9a1"
Sep 15 10:41:52.278224 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.278203    2583 scope.go:117] "RemoveContainer" containerID="713269bb8818f7d7368b2ee616ea97dd61d7711e7dbdc4a77930134f25b41c59"
Sep 15 10:41:52.278561 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:41:52.278539    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"713269bb8818f7d7368b2ee616ea97dd61d7711e7dbdc4a77930134f25b41c59\": container with ID starting with 713269bb8818f7d7368b2ee616ea97dd61d7711e7dbdc4a77930134f25b41c59 not found: ID does not exist" containerID="713269bb8818f7d7368b2ee616ea97dd61d7711e7dbdc4a77930134f25b41c59"
Sep 15 10:41:52.278611 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.278570    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"713269bb8818f7d7368b2ee616ea97dd61d7711e7dbdc4a77930134f25b41c59"} err="failed to get container status \"713269bb8818f7d7368b2ee616ea97dd61d7711e7dbdc4a77930134f25b41c59\": rpc error: code = NotFound desc = could not find container \"713269bb8818f7d7368b2ee616ea97dd61d7711e7dbdc4a77930134f25b41c59\": container with ID starting with 713269bb8818f7d7368b2ee616ea97dd61d7711e7dbdc4a77930134f25b41c59 not found: ID does not exist"
Sep 15 10:41:52.278611 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.278588    2583 scope.go:117] "RemoveContainer" containerID="b11d7ac7ec2b3856c2941bdffb010bbf28295187a85e5c9e1cc021cb63e0c9a1"
Sep 15 10:41:52.278844 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:41:52.278827    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b11d7ac7ec2b3856c2941bdffb010bbf28295187a85e5c9e1cc021cb63e0c9a1\": container with ID starting with b11d7ac7ec2b3856c2941bdffb010bbf28295187a85e5c9e1cc021cb63e0c9a1 not found: ID does not exist" containerID="b11d7ac7ec2b3856c2941bdffb010bbf28295187a85e5c9e1cc021cb63e0c9a1"
Sep 15 10:41:52.278900 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.278848    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b11d7ac7ec2b3856c2941bdffb010bbf28295187a85e5c9e1cc021cb63e0c9a1"} err="failed to get container status \"b11d7ac7ec2b3856c2941bdffb010bbf28295187a85e5c9e1cc021cb63e0c9a1\": rpc error: code = NotFound desc = could not find container \"b11d7ac7ec2b3856c2941bdffb010bbf28295187a85e5c9e1cc021cb63e0c9a1\": container with ID starting with b11d7ac7ec2b3856c2941bdffb010bbf28295187a85e5c9e1cc021cb63e0c9a1 not found: ID does not exist"
Sep 15 10:41:52.279812 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.279791    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"]
Sep 15 10:41:52.284885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:52.284862    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-6111ecc0-openshift-default-5c4cc884cb-7mnkw"]
Sep 15 10:41:53.300697 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:53.300667    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac3cc4d8-ad26-4bfa-8690-dad507e6e055" path="/var/lib/kubelet/pods/ac3cc4d8-ad26-4bfa-8690-dad507e6e055/volumes"
Sep 15 10:41:55.800227 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:55.800190    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:41:55.864586 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:55.864549    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:41:57.797002 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:41:57.796931    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:42:02.255705 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:02.255670    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:42:02.356649 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:02.356616    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:42:02.715569 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:02.715538    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:42:03.822898 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:03.822868    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:42:05.401930 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:05.401899    2583 scope.go:117] "RemoveContainer" containerID="f4b8d9f135d230db4995920620855b58090c46c4ec70dda6463527d7d069b00b"
Sep 15 10:42:08.066490 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.066435    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:42:08.140272 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.140239    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:42:08.345330 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.345248    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6"]
Sep 15 10:42:08.345708 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.345694    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac3cc4d8-ad26-4bfa-8690-dad507e6e055" containerName="istio-proxy"
Sep 15 10:42:08.345769 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.345710    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac3cc4d8-ad26-4bfa-8690-dad507e6e055" containerName="istio-proxy"
Sep 15 10:42:08.345769 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.345722    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac3cc4d8-ad26-4bfa-8690-dad507e6e055" containerName="istio-proxy"
Sep 15 10:42:08.345769 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.345728    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac3cc4d8-ad26-4bfa-8690-dad507e6e055" containerName="istio-proxy"
Sep 15 10:42:08.345866 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.345802    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac3cc4d8-ad26-4bfa-8690-dad507e6e055" containerName="istio-proxy"
Sep 15 10:42:08.345866 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.345811    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac3cc4d8-ad26-4bfa-8690-dad507e6e055" containerName="istio-proxy"
Sep 15 10:42:08.348904 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.348885    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6"
Sep 15 10:42:08.352109 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.352080    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-44134e\"/\"openshift-service-ca.crt\""
Sep 15 10:42:08.352237 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.352107    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-44134e\"/\"kube-root-ca.crt\""
Sep 15 10:42:08.352237 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.352182    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-44134e\"/\"default-dockercfg-dkv6h\""
Sep 15 10:42:08.352363 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.352243    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-44134e\"/\"auth-test-model-44134e-kserve-self-signed-certs\""
Sep 15 10:42:08.358520 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.358488    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6"]
Sep 15 10:42:08.399810 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.399782    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20a58489-4663-478f-8a26-09468396c11e-model-cache\") pod \"auth-test-model-44134e-kserve-bb574f8c5-wp7f6\" (UID: \"20a58489-4663-478f-8a26-09468396c11e\") " pod="ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6"
Sep 15 10:42:08.399963 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.399819    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nkjn\" (UniqueName: \"kubernetes.io/projected/20a58489-4663-478f-8a26-09468396c11e-kube-api-access-7nkjn\") pod \"auth-test-model-44134e-kserve-bb574f8c5-wp7f6\" (UID: \"20a58489-4663-478f-8a26-09468396c11e\") " pod="ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6"
Sep 15 10:42:08.399963 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.399892    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20a58489-4663-478f-8a26-09468396c11e-tmp-dir\") pod \"auth-test-model-44134e-kserve-bb574f8c5-wp7f6\" (UID: \"20a58489-4663-478f-8a26-09468396c11e\") " pod="ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6"
Sep 15 10:42:08.400039 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.400021    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20a58489-4663-478f-8a26-09468396c11e-dshm\") pod \"auth-test-model-44134e-kserve-bb574f8c5-wp7f6\" (UID: \"20a58489-4663-478f-8a26-09468396c11e\") " pod="ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6"
Sep 15 10:42:08.400115 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.400098    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20a58489-4663-478f-8a26-09468396c11e-home\") pod \"auth-test-model-44134e-kserve-bb574f8c5-wp7f6\" (UID: \"20a58489-4663-478f-8a26-09468396c11e\") " pod="ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6"
Sep 15 10:42:08.400167 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.400139    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20a58489-4663-478f-8a26-09468396c11e-tls-certs\") pod \"auth-test-model-44134e-kserve-bb574f8c5-wp7f6\" (UID: \"20a58489-4663-478f-8a26-09468396c11e\") " pod="ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6"
Sep 15 10:42:08.449643 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.449607    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:42:08.501304 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.501263    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20a58489-4663-478f-8a26-09468396c11e-tmp-dir\") pod \"auth-test-model-44134e-kserve-bb574f8c5-wp7f6\" (UID: \"20a58489-4663-478f-8a26-09468396c11e\") " pod="ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6"
Sep 15 10:42:08.501471 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.501344    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20a58489-4663-478f-8a26-09468396c11e-dshm\") pod \"auth-test-model-44134e-kserve-bb574f8c5-wp7f6\" (UID: \"20a58489-4663-478f-8a26-09468396c11e\") " pod="ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6"
Sep 15 10:42:08.501471 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.501391    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20a58489-4663-478f-8a26-09468396c11e-home\") pod \"auth-test-model-44134e-kserve-bb574f8c5-wp7f6\" (UID: \"20a58489-4663-478f-8a26-09468396c11e\") " pod="ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6"
Sep 15 10:42:08.501471 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.501410    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20a58489-4663-478f-8a26-09468396c11e-tls-certs\") pod \"auth-test-model-44134e-kserve-bb574f8c5-wp7f6\" (UID: \"20a58489-4663-478f-8a26-09468396c11e\") " pod="ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6"
Sep 15 10:42:08.501666 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.501537    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20a58489-4663-478f-8a26-09468396c11e-model-cache\") pod \"auth-test-model-44134e-kserve-bb574f8c5-wp7f6\" (UID: \"20a58489-4663-478f-8a26-09468396c11e\") " pod="ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6"
Sep 15 10:42:08.501666 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.501588    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nkjn\" (UniqueName: \"kubernetes.io/projected/20a58489-4663-478f-8a26-09468396c11e-kube-api-access-7nkjn\") pod \"auth-test-model-44134e-kserve-bb574f8c5-wp7f6\" (UID: \"20a58489-4663-478f-8a26-09468396c11e\") " pod="ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6"
Sep 15 10:42:08.501779 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.501697    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20a58489-4663-478f-8a26-09468396c11e-tmp-dir\") pod \"auth-test-model-44134e-kserve-bb574f8c5-wp7f6\" (UID: \"20a58489-4663-478f-8a26-09468396c11e\") " pod="ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6"
Sep 15 10:42:08.501834 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.501774    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20a58489-4663-478f-8a26-09468396c11e-home\") pod \"auth-test-model-44134e-kserve-bb574f8c5-wp7f6\" (UID: \"20a58489-4663-478f-8a26-09468396c11e\") " pod="ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6"
Sep 15 10:42:08.501906 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.501886    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20a58489-4663-478f-8a26-09468396c11e-model-cache\") pod \"auth-test-model-44134e-kserve-bb574f8c5-wp7f6\" (UID: \"20a58489-4663-478f-8a26-09468396c11e\") " pod="ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6"
Sep 15 10:42:08.503744 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.503714    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20a58489-4663-478f-8a26-09468396c11e-dshm\") pod \"auth-test-model-44134e-kserve-bb574f8c5-wp7f6\" (UID: \"20a58489-4663-478f-8a26-09468396c11e\") " pod="ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6"
Sep 15 10:42:08.503908 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.503891    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20a58489-4663-478f-8a26-09468396c11e-tls-certs\") pod \"auth-test-model-44134e-kserve-bb574f8c5-wp7f6\" (UID: \"20a58489-4663-478f-8a26-09468396c11e\") " pod="ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6"
Sep 15 10:42:08.508886 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.508864    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nkjn\" (UniqueName: \"kubernetes.io/projected/20a58489-4663-478f-8a26-09468396c11e-kube-api-access-7nkjn\") pod \"auth-test-model-44134e-kserve-bb574f8c5-wp7f6\" (UID: \"20a58489-4663-478f-8a26-09468396c11e\") " pod="ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6"
Sep 15 10:42:08.659395 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.659371    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6"
Sep 15 10:42:08.936026 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:08.935995    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6"]
Sep 15 10:42:08.937274 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:42:08.937251    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod20a58489_4663_478f_8a26_09468396c11e.slice/crio-34368fa7d700fc9848a3b9f3fd6cb394c9ed35c3d560661eaa14bb4f42c76039 WatchSource:0}: Error finding container 34368fa7d700fc9848a3b9f3fd6cb394c9ed35c3d560661eaa14bb4f42c76039: Status 404 returned error can't find the container with id 34368fa7d700fc9848a3b9f3fd6cb394c9ed35c3d560661eaa14bb4f42c76039
Sep 15 10:42:09.360106 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:09.359998    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6" event={"ID":"20a58489-4663-478f-8a26-09468396c11e","Type":"ContainerStarted","Data":"0db213345446457017eee6584e68a0df90b3430fe277877f5ac08bb81725dfba"}
Sep 15 10:42:09.360106 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:09.360045    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6" event={"ID":"20a58489-4663-478f-8a26-09468396c11e","Type":"ContainerStarted","Data":"34368fa7d700fc9848a3b9f3fd6cb394c9ed35c3d560661eaa14bb4f42c76039"}
Sep 15 10:42:09.375817 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:09.375770    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6" podStartSLOduration=1.375753429 podStartE2EDuration="1.375753429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:42:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:42:09.374743914 +0000 UTC m=+1444.621710885" watchObservedRunningTime="2026-09-15 10:42:09.375753429 +0000 UTC m=+1444.622720398"
Sep 15 10:42:12.926497 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:12.926464    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:42:13.173096 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:13.173067    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:42:18.660293 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:18.660258    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6"
Sep 15 10:42:18.660800 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:18.660313    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6"
Sep 15 10:42:18.672665 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:18.672638    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6"
Sep 15 10:42:19.427530 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:19.427504    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6"
Sep 15 10:42:34.050409 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:34.050377    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:42:40.045328 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:40.045272    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:42:40.153986 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:40.153951    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:42:40.386789 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:40.386758    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:42:40.671544 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:40.671455    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:42:41.218809 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:41.218770    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:42:42.970152 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:42.970092    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:42:43.672826 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:43.672783    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:42:43.766900 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:43.766866    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:42:44.375874 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:44.375830    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:42:44.670105 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:44.670078    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:42:45.332384 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:45.332353    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:42:46.073210 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:46.073181    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:42:46.681674 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:46.681641    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:42:47.396409 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:47.396379    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:42:47.993075 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:47.993041    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a81e8d-8699df7474-strhx"]
Sep 15 10:42:47.996909 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:47.996886    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a81e8d-8699df7474-strhx"
Sep 15 10:42:47.999086 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:47.999064    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-a81e8d-serving-cert\""
Sep 15 10:42:48.004811 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:48.004788    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a81e8d-8699df7474-strhx"]
Sep 15 10:42:48.017224 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:48.017204    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:42:48.077501 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:48.077474    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a209f0fc-01ec-4109-a603-a82e996a8434-maas-api-tls\") pod \"maas-api-e2e-api-a-a81e8d-8699df7474-strhx\" (UID: \"a209f0fc-01ec-4109-a603-a82e996a8434\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a81e8d-8699df7474-strhx"
Sep 15 10:42:48.077658 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:48.077522    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a209f0fc-01ec-4109-a603-a82e996a8434-openshift-service-ca\") pod \"maas-api-e2e-api-a-a81e8d-8699df7474-strhx\" (UID: \"a209f0fc-01ec-4109-a603-a82e996a8434\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a81e8d-8699df7474-strhx"
Sep 15 10:42:48.077658 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:48.077548    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a209f0fc-01ec-4109-a603-a82e996a8434-metrics-certs\") pod \"maas-api-e2e-api-a-a81e8d-8699df7474-strhx\" (UID: \"a209f0fc-01ec-4109-a603-a82e996a8434\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a81e8d-8699df7474-strhx"
Sep 15 10:42:48.077777 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:48.077713    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48xrl\" (UniqueName: \"kubernetes.io/projected/a209f0fc-01ec-4109-a603-a82e996a8434-kube-api-access-48xrl\") pod \"maas-api-e2e-api-a-a81e8d-8699df7474-strhx\" (UID: \"a209f0fc-01ec-4109-a603-a82e996a8434\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a81e8d-8699df7474-strhx"
Sep 15 10:42:48.120473 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:48.120427    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:42:48.178842 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:48.178813    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48xrl\" (UniqueName: \"kubernetes.io/projected/a209f0fc-01ec-4109-a603-a82e996a8434-kube-api-access-48xrl\") pod \"maas-api-e2e-api-a-a81e8d-8699df7474-strhx\" (UID: \"a209f0fc-01ec-4109-a603-a82e996a8434\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a81e8d-8699df7474-strhx"
Sep 15 10:42:48.179007 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:48.178857    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a209f0fc-01ec-4109-a603-a82e996a8434-maas-api-tls\") pod \"maas-api-e2e-api-a-a81e8d-8699df7474-strhx\" (UID: \"a209f0fc-01ec-4109-a603-a82e996a8434\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a81e8d-8699df7474-strhx"
Sep 15 10:42:48.179007 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:48.178880    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a209f0fc-01ec-4109-a603-a82e996a8434-openshift-service-ca\") pod \"maas-api-e2e-api-a-a81e8d-8699df7474-strhx\" (UID: \"a209f0fc-01ec-4109-a603-a82e996a8434\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a81e8d-8699df7474-strhx"
Sep 15 10:42:48.179007 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:42:48.178964    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-a81e8d-serving-cert: secret "maas-api-e2e-api-a-a81e8d-serving-cert" not found
Sep 15 10:42:48.179007 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:48.178981    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a209f0fc-01ec-4109-a603-a82e996a8434-metrics-certs\") pod \"maas-api-e2e-api-a-a81e8d-8699df7474-strhx\" (UID: \"a209f0fc-01ec-4109-a603-a82e996a8434\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a81e8d-8699df7474-strhx"
Sep 15 10:42:48.179230 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:42:48.179033    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a209f0fc-01ec-4109-a603-a82e996a8434-maas-api-tls podName:a209f0fc-01ec-4109-a603-a82e996a8434 nodeName:}" failed. No retries permitted until 2026-09-15 10:42:48.67901677 +0000 UTC m=+1483.925983724 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a209f0fc-01ec-4109-a603-a82e996a8434-maas-api-tls") pod "maas-api-e2e-api-a-a81e8d-8699df7474-strhx" (UID: "a209f0fc-01ec-4109-a603-a82e996a8434") : secret "maas-api-e2e-api-a-a81e8d-serving-cert" not found
Sep 15 10:42:48.179645 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:48.179623    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a209f0fc-01ec-4109-a603-a82e996a8434-openshift-service-ca\") pod \"maas-api-e2e-api-a-a81e8d-8699df7474-strhx\" (UID: \"a209f0fc-01ec-4109-a603-a82e996a8434\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a81e8d-8699df7474-strhx"
Sep 15 10:42:48.181329 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:48.181308    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a209f0fc-01ec-4109-a603-a82e996a8434-metrics-certs\") pod \"maas-api-e2e-api-a-a81e8d-8699df7474-strhx\" (UID: \"a209f0fc-01ec-4109-a603-a82e996a8434\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a81e8d-8699df7474-strhx"
Sep 15 10:42:48.187047 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:48.187024    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48xrl\" (UniqueName: \"kubernetes.io/projected/a209f0fc-01ec-4109-a603-a82e996a8434-kube-api-access-48xrl\") pod \"maas-api-e2e-api-a-a81e8d-8699df7474-strhx\" (UID: \"a209f0fc-01ec-4109-a603-a82e996a8434\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a81e8d-8699df7474-strhx"
Sep 15 10:42:48.369386 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:48.369310    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:42:48.473068 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:48.473030    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:42:48.566331 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:48.566297    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:42:48.686931 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:48.686898    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a209f0fc-01ec-4109-a603-a82e996a8434-maas-api-tls\") pod \"maas-api-e2e-api-a-a81e8d-8699df7474-strhx\" (UID: \"a209f0fc-01ec-4109-a603-a82e996a8434\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a81e8d-8699df7474-strhx"
Sep 15 10:42:48.691748 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:48.691708    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a209f0fc-01ec-4109-a603-a82e996a8434-maas-api-tls\") pod \"maas-api-e2e-api-a-a81e8d-8699df7474-strhx\" (UID: \"a209f0fc-01ec-4109-a603-a82e996a8434\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a81e8d-8699df7474-strhx"
Sep 15 10:42:48.909712 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:48.909677    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a81e8d-8699df7474-strhx"
Sep 15 10:42:49.186199 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:49.186174    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a81e8d-8699df7474-strhx"]
Sep 15 10:42:49.187175 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:42:49.187157    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda209f0fc_01ec_4109_a603_a82e996a8434.slice/crio-abe733171a690bf41834359a801eec585d604ee26ef849c41010fa85b3ce9ca2 WatchSource:0}: Error finding container abe733171a690bf41834359a801eec585d604ee26ef849c41010fa85b3ce9ca2: Status 404 returned error can't find the container with id abe733171a690bf41834359a801eec585d604ee26ef849c41010fa85b3ce9ca2
Sep 15 10:42:49.188698 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:49.188684    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 10:42:49.589111 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:49.589088    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a81e8d-8699df7474-strhx" event={"ID":"a209f0fc-01ec-4109-a603-a82e996a8434","Type":"ContainerStarted","Data":"abe733171a690bf41834359a801eec585d604ee26ef849c41010fa85b3ce9ca2"}
Sep 15 10:42:50.596503 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:50.596465    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a81e8d-8699df7474-strhx" event={"ID":"a209f0fc-01ec-4109-a603-a82e996a8434","Type":"ContainerStarted","Data":"8a90f48c5cc865c50f01b613e56cb0df658429798ee26e151ea49e44f6adc8db"}
Sep 15 10:42:50.596964 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:50.596521    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a81e8d-8699df7474-strhx"
Sep 15 10:42:50.616007 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:50.615963    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a81e8d-8699df7474-strhx" podStartSLOduration=3.266938012 podStartE2EDuration="3.615949567s" totalImagesPullingTime="349.011555ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:42:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:42:50.613052626 +0000 UTC m=+1485.860019599" watchObservedRunningTime="2026-09-15 10:42:50.615949567 +0000 UTC m=+1485.862916581"
Sep 15 10:42:50.941019 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:50.940963    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:42:55.162635 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:55.162598    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:42:56.607913 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:42:56.607888    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a81e8d-8699df7474-strhx"
Sep 15 10:43:02.038579 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:02.038544    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:08.839650 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:08.839610    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:10.139229 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:10.139200    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:10.775175 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:10.775144    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:11.367583 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:11.367554    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:12.097573 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:12.097539    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:12.806769 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:12.806739    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:14.348012 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.347981    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"]
Sep 15 10:43:14.351225 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.351204    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:14.353560 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.353539    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-069380-openshift-default-dockercfg-z62f9\""
Sep 15 10:43:14.363108 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.363084    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"]
Sep 15 10:43:14.449837 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.449806    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:14.460880 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.460845    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3508e4ea-2813-4c8b-8007-fec91eb76f79-istio-data\") pod \"e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") " pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:14.461025 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.460902    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3508e4ea-2813-4c8b-8007-fec91eb76f79-credential-socket\") pod \"e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") " pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:14.461025 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.460955    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3508e4ea-2813-4c8b-8007-fec91eb76f79-workload-socket\") pod \"e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") " pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:14.461025 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.461012    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3508e4ea-2813-4c8b-8007-fec91eb76f79-workload-certs\") pod \"e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") " pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:14.461125 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.461048    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3508e4ea-2813-4c8b-8007-fec91eb76f79-istio-podinfo\") pod \"e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") " pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:14.461125 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.461096    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3508e4ea-2813-4c8b-8007-fec91eb76f79-istio-token\") pod \"e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") " pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:14.461191 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.461133    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3508e4ea-2813-4c8b-8007-fec91eb76f79-istiod-ca-cert\") pod \"e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") " pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:14.461226 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.461210    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3508e4ea-2813-4c8b-8007-fec91eb76f79-istio-envoy\") pod \"e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") " pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:14.461307 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.461281    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ln9p9\" (UniqueName: \"kubernetes.io/projected/3508e4ea-2813-4c8b-8007-fec91eb76f79-kube-api-access-ln9p9\") pod \"e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") " pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:14.562543 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.562512    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3508e4ea-2813-4c8b-8007-fec91eb76f79-workload-certs\") pod \"e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") " pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:14.562543 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.562549    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3508e4ea-2813-4c8b-8007-fec91eb76f79-istio-podinfo\") pod \"e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") " pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:14.562776 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.562568    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3508e4ea-2813-4c8b-8007-fec91eb76f79-istio-token\") pod \"e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") " pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:14.562776 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.562593    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3508e4ea-2813-4c8b-8007-fec91eb76f79-istiod-ca-cert\") pod \"e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") " pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:14.562776 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.562637    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3508e4ea-2813-4c8b-8007-fec91eb76f79-istio-envoy\") pod \"e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") " pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:14.562776 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.562668    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ln9p9\" (UniqueName: \"kubernetes.io/projected/3508e4ea-2813-4c8b-8007-fec91eb76f79-kube-api-access-ln9p9\") pod \"e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") " pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:14.562776 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.562716    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3508e4ea-2813-4c8b-8007-fec91eb76f79-istio-data\") pod \"e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") " pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:14.562776 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.562752    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3508e4ea-2813-4c8b-8007-fec91eb76f79-credential-socket\") pod \"e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") " pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:14.562776 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.562776    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3508e4ea-2813-4c8b-8007-fec91eb76f79-workload-socket\") pod \"e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") " pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:14.563123 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.562959    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3508e4ea-2813-4c8b-8007-fec91eb76f79-workload-certs\") pod \"e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") " pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:14.563123 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.563098    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3508e4ea-2813-4c8b-8007-fec91eb76f79-istio-data\") pod \"e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") " pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:14.563229 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.563160    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3508e4ea-2813-4c8b-8007-fec91eb76f79-workload-socket\") pod \"e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") " pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:14.563229 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.563219    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3508e4ea-2813-4c8b-8007-fec91eb76f79-credential-socket\") pod \"e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") " pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:14.563365 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.563343    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3508e4ea-2813-4c8b-8007-fec91eb76f79-istiod-ca-cert\") pod \"e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") " pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:14.564907 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.564882    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3508e4ea-2813-4c8b-8007-fec91eb76f79-istio-envoy\") pod \"e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") " pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:14.565242 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.565226    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3508e4ea-2813-4c8b-8007-fec91eb76f79-istio-podinfo\") pod \"e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") " pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:14.569602 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.569573    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3508e4ea-2813-4c8b-8007-fec91eb76f79-istio-token\") pod \"e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") " pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:14.569740 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.569722    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ln9p9\" (UniqueName: \"kubernetes.io/projected/3508e4ea-2813-4c8b-8007-fec91eb76f79-kube-api-access-ln9p9\") pod \"e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") " pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:14.622233 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.622165    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:14.662584 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.662547    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:14.897475 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.897413    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:14.941335 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.941314    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"]
Sep 15 10:43:14.943205 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:43:14.943180    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3508e4ea_2813_4c8b_8007_fec91eb76f79.slice/crio-b23b06adeced77ad4886c02c8594e10d4936c68b6546c07ac9b8e0c2a240fbb7 WatchSource:0}: Error finding container b23b06adeced77ad4886c02c8594e10d4936c68b6546c07ac9b8e0c2a240fbb7: Status 404 returned error can't find the container with id b23b06adeced77ad4886c02c8594e10d4936c68b6546c07ac9b8e0c2a240fbb7
Sep 15 10:43:14.945558 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.945525    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:43:14.945665 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.945604    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:43:14.945665 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:14.945646    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:43:15.750844 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:15.750810    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch" event={"ID":"3508e4ea-2813-4c8b-8007-fec91eb76f79","Type":"ContainerStarted","Data":"9f7e909f3b5a125216e1f29f8efb50f485aa26d26b5011edfbd4aa1a345b2bd6"}
Sep 15 10:43:15.751281 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:15.750872    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch" event={"ID":"3508e4ea-2813-4c8b-8007-fec91eb76f79","Type":"ContainerStarted","Data":"b23b06adeced77ad4886c02c8594e10d4936c68b6546c07ac9b8e0c2a240fbb7"}
Sep 15 10:43:15.770790 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:15.770741    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch" podStartSLOduration=1.77072601 podStartE2EDuration="1.77072601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:43:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:43:15.770136439 +0000 UTC m=+1511.017103419" watchObservedRunningTime="2026-09-15 10:43:15.77072601 +0000 UTC m=+1511.017692979"
Sep 15 10:43:16.663130 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:16.663093    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:16.668330 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:16.668301    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:16.756527 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:16.756495    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:16.757373 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:16.757352    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:16.995811 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:16.995734    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:17.577935 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:17.577900    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:17.680464 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:17.680420    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:17.778642 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:17.778606    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:18.461257 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:18.461228    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:19.089875 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:19.089843    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:19.976190 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:19.976157    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:20.623129 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:20.623100    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:21.307040 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:21.307011    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:21.990916 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:21.990886    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:22.684776 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:22.684744    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-069380-544df7c984-mdlq4"]
Sep 15 10:43:22.689340 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:22.689311    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-069380-544df7c984-mdlq4"
Sep 15 10:43:22.691677 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:22.691658    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-069380-serving-cert\""
Sep 15 10:43:22.697081 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:22.697056    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-069380-544df7c984-mdlq4"]
Sep 15 10:43:22.714380 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:22.714347    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:22.769345 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:22.769317    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:22.847031 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:22.846997    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24rlr\" (UniqueName: \"kubernetes.io/projected/863cf327-3188-4b77-8ba5-90aafc821a77-kube-api-access-24rlr\") pod \"maas-api-e2e-api-b-069380-544df7c984-mdlq4\" (UID: \"863cf327-3188-4b77-8ba5-90aafc821a77\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-069380-544df7c984-mdlq4"
Sep 15 10:43:22.847284 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:22.847066    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/863cf327-3188-4b77-8ba5-90aafc821a77-openshift-service-ca\") pod \"maas-api-e2e-api-b-069380-544df7c984-mdlq4\" (UID: \"863cf327-3188-4b77-8ba5-90aafc821a77\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-069380-544df7c984-mdlq4"
Sep 15 10:43:22.847284 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:22.847145    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/863cf327-3188-4b77-8ba5-90aafc821a77-maas-api-tls\") pod \"maas-api-e2e-api-b-069380-544df7c984-mdlq4\" (UID: \"863cf327-3188-4b77-8ba5-90aafc821a77\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-069380-544df7c984-mdlq4"
Sep 15 10:43:22.847284 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:22.847179    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/863cf327-3188-4b77-8ba5-90aafc821a77-metrics-certs\") pod \"maas-api-e2e-api-b-069380-544df7c984-mdlq4\" (UID: \"863cf327-3188-4b77-8ba5-90aafc821a77\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-069380-544df7c984-mdlq4"
Sep 15 10:43:22.902975 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:22.902937    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:22.948304 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:22.948234    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/863cf327-3188-4b77-8ba5-90aafc821a77-maas-api-tls\") pod \"maas-api-e2e-api-b-069380-544df7c984-mdlq4\" (UID: \"863cf327-3188-4b77-8ba5-90aafc821a77\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-069380-544df7c984-mdlq4"
Sep 15 10:43:22.948304 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:22.948271    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/863cf327-3188-4b77-8ba5-90aafc821a77-metrics-certs\") pod \"maas-api-e2e-api-b-069380-544df7c984-mdlq4\" (UID: \"863cf327-3188-4b77-8ba5-90aafc821a77\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-069380-544df7c984-mdlq4"
Sep 15 10:43:22.948513 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:22.948359    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24rlr\" (UniqueName: \"kubernetes.io/projected/863cf327-3188-4b77-8ba5-90aafc821a77-kube-api-access-24rlr\") pod \"maas-api-e2e-api-b-069380-544df7c984-mdlq4\" (UID: \"863cf327-3188-4b77-8ba5-90aafc821a77\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-069380-544df7c984-mdlq4"
Sep 15 10:43:22.948513 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:22.948481    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/863cf327-3188-4b77-8ba5-90aafc821a77-openshift-service-ca\") pod \"maas-api-e2e-api-b-069380-544df7c984-mdlq4\" (UID: \"863cf327-3188-4b77-8ba5-90aafc821a77\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-069380-544df7c984-mdlq4"
Sep 15 10:43:22.949136 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:22.949109    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/863cf327-3188-4b77-8ba5-90aafc821a77-openshift-service-ca\") pod \"maas-api-e2e-api-b-069380-544df7c984-mdlq4\" (UID: \"863cf327-3188-4b77-8ba5-90aafc821a77\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-069380-544df7c984-mdlq4"
Sep 15 10:43:22.950848 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:22.950826    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/863cf327-3188-4b77-8ba5-90aafc821a77-metrics-certs\") pod \"maas-api-e2e-api-b-069380-544df7c984-mdlq4\" (UID: \"863cf327-3188-4b77-8ba5-90aafc821a77\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-069380-544df7c984-mdlq4"
Sep 15 10:43:22.950951 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:22.950884    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/863cf327-3188-4b77-8ba5-90aafc821a77-maas-api-tls\") pod \"maas-api-e2e-api-b-069380-544df7c984-mdlq4\" (UID: \"863cf327-3188-4b77-8ba5-90aafc821a77\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-069380-544df7c984-mdlq4"
Sep 15 10:43:22.957080 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:22.957058    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24rlr\" (UniqueName: \"kubernetes.io/projected/863cf327-3188-4b77-8ba5-90aafc821a77-kube-api-access-24rlr\") pod \"maas-api-e2e-api-b-069380-544df7c984-mdlq4\" (UID: \"863cf327-3188-4b77-8ba5-90aafc821a77\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-069380-544df7c984-mdlq4"
Sep 15 10:43:23.003246 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:23.003223    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-069380-544df7c984-mdlq4"
Sep 15 10:43:23.283549 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:23.283510    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-069380-544df7c984-mdlq4"]
Sep 15 10:43:23.285019 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:43:23.284992    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod863cf327_3188_4b77_8ba5_90aafc821a77.slice/crio-160a246ab99b3b99e82c46950a42caecf8e1c4ea118d0408f3f965d6d946e836 WatchSource:0}: Error finding container 160a246ab99b3b99e82c46950a42caecf8e1c4ea118d0408f3f965d6d946e836: Status 404 returned error can't find the container with id 160a246ab99b3b99e82c46950a42caecf8e1c4ea118d0408f3f965d6d946e836
Sep 15 10:43:23.307130 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:23.307107    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:23.389797 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:23.389772    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:23.809036 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:23.809001    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-069380-544df7c984-mdlq4" event={"ID":"863cf327-3188-4b77-8ba5-90aafc821a77","Type":"ContainerStarted","Data":"03813663bd5161200acd1d4382844d15cae07e961eab0181a7ab7a9947835044"}
Sep 15 10:43:23.809193 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:23.809063    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-069380-544df7c984-mdlq4" event={"ID":"863cf327-3188-4b77-8ba5-90aafc821a77","Type":"ContainerStarted","Data":"160a246ab99b3b99e82c46950a42caecf8e1c4ea118d0408f3f965d6d946e836"}
Sep 15 10:43:23.809193 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:23.809176    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-069380-544df7c984-mdlq4"
Sep 15 10:43:23.826775 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:23.826729    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-069380-544df7c984-mdlq4" podStartSLOduration=1.493828728 podStartE2EDuration="1.826713604s" totalImagesPullingTime="332.884876ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:43:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:43:23.824112701 +0000 UTC m=+1519.071079701" watchObservedRunningTime="2026-09-15 10:43:23.826713604 +0000 UTC m=+1519.073680614"
Sep 15 10:43:29.820057 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:29.819983    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-069380-544df7c984-mdlq4"
Sep 15 10:43:44.720074 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:44.720040    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:45.496263 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:45.496231    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:46.109083 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:46.109053    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:46.791226 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:46.791138    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:47.552609 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:47.552576    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:47.717126 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:47.717092    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:47.797611 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:47.797571    2583 patch_prober.go:28] interesting pod/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 15 10:43:47.797785 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:47.797635    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch" podUID="3508e4ea-2813-4c8b-8007-fec91eb76f79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 15 10:43:48.203892 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:48.203861    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:48.892723 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:48.892684    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:49.552308 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:49.552220    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:49.966147 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:49.966116    2583 generic.go:356] "Generic (PLEG): container finished" podID="3508e4ea-2813-4c8b-8007-fec91eb76f79" containerID="9f7e909f3b5a125216e1f29f8efb50f485aa26d26b5011edfbd4aa1a345b2bd6" exitCode=0
Sep 15 10:43:49.966524 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:49.966189    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch" event={"ID":"3508e4ea-2813-4c8b-8007-fec91eb76f79","Type":"ContainerDied","Data":"9f7e909f3b5a125216e1f29f8efb50f485aa26d26b5011edfbd4aa1a345b2bd6"}
Sep 15 10:43:49.966667 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:49.966652    2583 scope.go:117] "RemoveContainer" containerID="9f7e909f3b5a125216e1f29f8efb50f485aa26d26b5011edfbd4aa1a345b2bd6"
Sep 15 10:43:49.967995 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:49.967971    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:43:49.968062 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:49.968034    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:43:49.968098 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:49.968065    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:43:50.182411 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:50.182376    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:50.975233 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:50.975195    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch" event={"ID":"3508e4ea-2813-4c8b-8007-fec91eb76f79","Type":"ContainerStarted","Data":"59aa3c32c2c55e6f5d46f7441d1c8837fa8a368fdc6982c26c668f7b616b3e92"}
Sep 15 10:43:51.086550 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:51.086521    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:51.663501 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:51.663454    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:51.664101 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:51.664075    2583 patch_prober.go:28] interesting pod/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 15 10:43:51.664287 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:51.664264    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch" podUID="3508e4ea-2813-4c8b-8007-fec91eb76f79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 15 10:43:51.683430 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:51.683404    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:52.306764 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:52.306728    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:52.663349 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:52.663302    2583 patch_prober.go:28] interesting pod/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 15 10:43:52.663572 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:52.663386    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch" podUID="3508e4ea-2813-4c8b-8007-fec91eb76f79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 15 10:43:52.998660 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:52.998578    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:53.593679 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:53.593634    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:53.664304 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:53.663846    2583 patch_prober.go:28] interesting pod/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 15 10:43:53.664304 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:53.663923    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch" podUID="3508e4ea-2813-4c8b-8007-fec91eb76f79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 15 10:43:54.290972 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:54.290938    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:54.663462 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:54.663401    2583 patch_prober.go:28] interesting pod/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 15 10:43:54.664067 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:54.664035    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch" podUID="3508e4ea-2813-4c8b-8007-fec91eb76f79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 15 10:43:54.875785 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:54.875751    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:55.467784 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:55.467747    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:55.663145 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:55.663105    2583 patch_prober.go:28] interesting pod/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 15 10:43:55.663856 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:55.663418    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch" podUID="3508e4ea-2813-4c8b-8007-fec91eb76f79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 15 10:43:56.150301 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:56.150266    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:43:56.666532 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:56.666503    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:56.666894 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:56.666826    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:43:56.667545 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:43:56.667528    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:44:00.583981 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:00.583943    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:03.613011 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:03.612975    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:06.069021 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:06.068988    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a81e8d-8699df7474-strhx"]
Sep 15 10:44:06.069478 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:06.069233    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a81e8d-8699df7474-strhx" podUID="a209f0fc-01ec-4109-a603-a82e996a8434" containerName="maas-api" containerID="cri-o://8a90f48c5cc865c50f01b613e56cb0df658429798ee26e151ea49e44f6adc8db" gracePeriod=30
Sep 15 10:44:06.093133 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:06.093108    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:06.157319 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:06.157270    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:06.220846 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:06.220818    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:06.277679 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:06.277651    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:06.321940 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:06.321880    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a81e8d-8699df7474-strhx"
Sep 15 10:44:06.395592 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:06.395559    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48xrl\" (UniqueName: \"kubernetes.io/projected/a209f0fc-01ec-4109-a603-a82e996a8434-kube-api-access-48xrl\") pod \"a209f0fc-01ec-4109-a603-a82e996a8434\" (UID: \"a209f0fc-01ec-4109-a603-a82e996a8434\") "
Sep 15 10:44:06.395789 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:06.395612    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a209f0fc-01ec-4109-a603-a82e996a8434-metrics-certs\") pod \"a209f0fc-01ec-4109-a603-a82e996a8434\" (UID: \"a209f0fc-01ec-4109-a603-a82e996a8434\") "
Sep 15 10:44:06.395789 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:06.395746    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a209f0fc-01ec-4109-a603-a82e996a8434-openshift-service-ca\") pod \"a209f0fc-01ec-4109-a603-a82e996a8434\" (UID: \"a209f0fc-01ec-4109-a603-a82e996a8434\") "
Sep 15 10:44:06.395908 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:06.395817    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a209f0fc-01ec-4109-a603-a82e996a8434-maas-api-tls\") pod \"a209f0fc-01ec-4109-a603-a82e996a8434\" (UID: \"a209f0fc-01ec-4109-a603-a82e996a8434\") "
Sep 15 10:44:06.396261 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:06.396235    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a209f0fc-01ec-4109-a603-a82e996a8434-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a209f0fc-01ec-4109-a603-a82e996a8434" (UID: "a209f0fc-01ec-4109-a603-a82e996a8434"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:44:06.398055 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:06.398024    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a209f0fc-01ec-4109-a603-a82e996a8434-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a209f0fc-01ec-4109-a603-a82e996a8434" (UID: "a209f0fc-01ec-4109-a603-a82e996a8434"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:44:06.398171 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:06.398048    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a209f0fc-01ec-4109-a603-a82e996a8434-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a209f0fc-01ec-4109-a603-a82e996a8434" (UID: "a209f0fc-01ec-4109-a603-a82e996a8434"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:44:06.398171 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:06.398136    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a209f0fc-01ec-4109-a603-a82e996a8434-kube-api-access-48xrl" (OuterVolumeSpecName: "kube-api-access-48xrl") pod "a209f0fc-01ec-4109-a603-a82e996a8434" (UID: "a209f0fc-01ec-4109-a603-a82e996a8434"). InnerVolumeSpecName "kube-api-access-48xrl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:44:06.497522 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:06.497488    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a209f0fc-01ec-4109-a603-a82e996a8434-openshift-service-ca\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:06.497522 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:06.497519    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a209f0fc-01ec-4109-a603-a82e996a8434-maas-api-tls\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:06.497522 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:06.497529    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48xrl\" (UniqueName: \"kubernetes.io/projected/a209f0fc-01ec-4109-a603-a82e996a8434-kube-api-access-48xrl\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:06.497758 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:06.497538    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a209f0fc-01ec-4109-a603-a82e996a8434-metrics-certs\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:06.587471 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:06.587343    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:06.775923 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:06.775886    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:07.078919 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:07.078880    2583 generic.go:356] "Generic (PLEG): container finished" podID="a209f0fc-01ec-4109-a603-a82e996a8434" containerID="8a90f48c5cc865c50f01b613e56cb0df658429798ee26e151ea49e44f6adc8db" exitCode=0
Sep 15 10:44:07.079341 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:07.078925    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a81e8d-8699df7474-strhx"
Sep 15 10:44:07.079341 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:07.078952    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a81e8d-8699df7474-strhx" event={"ID":"a209f0fc-01ec-4109-a603-a82e996a8434","Type":"ContainerDied","Data":"8a90f48c5cc865c50f01b613e56cb0df658429798ee26e151ea49e44f6adc8db"}
Sep 15 10:44:07.079341 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:07.078998    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a81e8d-8699df7474-strhx" event={"ID":"a209f0fc-01ec-4109-a603-a82e996a8434","Type":"ContainerDied","Data":"abe733171a690bf41834359a801eec585d604ee26ef849c41010fa85b3ce9ca2"}
Sep 15 10:44:07.079341 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:07.079020    2583 scope.go:117] "RemoveContainer" containerID="8a90f48c5cc865c50f01b613e56cb0df658429798ee26e151ea49e44f6adc8db"
Sep 15 10:44:07.089097 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:07.089081    2583 scope.go:117] "RemoveContainer" containerID="8a90f48c5cc865c50f01b613e56cb0df658429798ee26e151ea49e44f6adc8db"
Sep 15 10:44:07.089327 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:44:07.089300    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a90f48c5cc865c50f01b613e56cb0df658429798ee26e151ea49e44f6adc8db\": container with ID starting with 8a90f48c5cc865c50f01b613e56cb0df658429798ee26e151ea49e44f6adc8db not found: ID does not exist" containerID="8a90f48c5cc865c50f01b613e56cb0df658429798ee26e151ea49e44f6adc8db"
Sep 15 10:44:07.089375 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:07.089335    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a90f48c5cc865c50f01b613e56cb0df658429798ee26e151ea49e44f6adc8db"} err="failed to get container status \"8a90f48c5cc865c50f01b613e56cb0df658429798ee26e151ea49e44f6adc8db\": rpc error: code = NotFound desc = could not find container \"8a90f48c5cc865c50f01b613e56cb0df658429798ee26e151ea49e44f6adc8db\": container with ID starting with 8a90f48c5cc865c50f01b613e56cb0df658429798ee26e151ea49e44f6adc8db not found: ID does not exist"
Sep 15 10:44:07.101860 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:07.101837    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a81e8d-8699df7474-strhx"]
Sep 15 10:44:07.107073 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:07.107047    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a81e8d-8699df7474-strhx"]
Sep 15 10:44:07.300519 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:07.300490    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a209f0fc-01ec-4109-a603-a82e996a8434" path="/var/lib/kubelet/pods/a209f0fc-01ec-4109-a603-a82e996a8434/volumes"
Sep 15 10:44:07.582178 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:07.582145    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:08.372985 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:08.372951    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:08.725294 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:08.725265    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:08.799101 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:08.799072    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:09.093437 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:09.093358    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:09.777468 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:09.777417    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:10.482271 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:10.482236    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:11.077987 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:11.077946    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:11.668133 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:11.668104    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:12.544150 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:12.544122    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:12.646467 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:12.646417    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:12.771992 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:12.771963    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:13.994736 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:13.994702    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:14.604837 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:14.604807    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:15.190422 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:15.190394    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:15.978491 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:15.978460    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:16.689500 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:16.689468    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:17.378455 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:17.378412    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:17.981664 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:17.981632    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:18.667955 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:18.667919    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:25.467899 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:25.467867    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:25.777190 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:25.777110    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:26.683466 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:26.683408    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:27.285141 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:27.285102    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:27.763245 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:27.763208    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-069380-544df7c984-mdlq4"]
Sep 15 10:44:27.763719 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:27.763434    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-069380-544df7c984-mdlq4" podUID="863cf327-3188-4b77-8ba5-90aafc821a77" containerName="maas-api" containerID="cri-o://03813663bd5161200acd1d4382844d15cae07e961eab0181a7ab7a9947835044" gracePeriod=30
Sep 15 10:44:27.787537 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:27.787507    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:27.850708 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:27.850643    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:27.905633 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:27.905596    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:27.965679 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:27.965645    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:28.001619 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:28.001597    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-069380-544df7c984-mdlq4"
Sep 15 10:44:28.054653 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:28.054570    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:28.112235 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:28.112203    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24rlr\" (UniqueName: \"kubernetes.io/projected/863cf327-3188-4b77-8ba5-90aafc821a77-kube-api-access-24rlr\") pod \"863cf327-3188-4b77-8ba5-90aafc821a77\" (UID: \"863cf327-3188-4b77-8ba5-90aafc821a77\") "
Sep 15 10:44:28.112235 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:28.112241    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/863cf327-3188-4b77-8ba5-90aafc821a77-openshift-service-ca\") pod \"863cf327-3188-4b77-8ba5-90aafc821a77\" (UID: \"863cf327-3188-4b77-8ba5-90aafc821a77\") "
Sep 15 10:44:28.112497 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:28.112333    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/863cf327-3188-4b77-8ba5-90aafc821a77-metrics-certs\") pod \"863cf327-3188-4b77-8ba5-90aafc821a77\" (UID: \"863cf327-3188-4b77-8ba5-90aafc821a77\") "
Sep 15 10:44:28.112497 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:28.112361    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/863cf327-3188-4b77-8ba5-90aafc821a77-maas-api-tls\") pod \"863cf327-3188-4b77-8ba5-90aafc821a77\" (UID: \"863cf327-3188-4b77-8ba5-90aafc821a77\") "
Sep 15 10:44:28.112702 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:28.112679    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/863cf327-3188-4b77-8ba5-90aafc821a77-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "863cf327-3188-4b77-8ba5-90aafc821a77" (UID: "863cf327-3188-4b77-8ba5-90aafc821a77"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:44:28.114581 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:28.114549    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/863cf327-3188-4b77-8ba5-90aafc821a77-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "863cf327-3188-4b77-8ba5-90aafc821a77" (UID: "863cf327-3188-4b77-8ba5-90aafc821a77"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:44:28.114581 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:28.114567    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/863cf327-3188-4b77-8ba5-90aafc821a77-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "863cf327-3188-4b77-8ba5-90aafc821a77" (UID: "863cf327-3188-4b77-8ba5-90aafc821a77"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:44:28.114720 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:28.114633    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/863cf327-3188-4b77-8ba5-90aafc821a77-kube-api-access-24rlr" (OuterVolumeSpecName: "kube-api-access-24rlr") pod "863cf327-3188-4b77-8ba5-90aafc821a77" (UID: "863cf327-3188-4b77-8ba5-90aafc821a77"). InnerVolumeSpecName "kube-api-access-24rlr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:44:28.213908 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:28.213867    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/863cf327-3188-4b77-8ba5-90aafc821a77-metrics-certs\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:28.213908 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:28.213902    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/863cf327-3188-4b77-8ba5-90aafc821a77-maas-api-tls\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:28.213908 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:28.213918    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24rlr\" (UniqueName: \"kubernetes.io/projected/863cf327-3188-4b77-8ba5-90aafc821a77-kube-api-access-24rlr\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:28.214267 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:28.213932    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/863cf327-3188-4b77-8ba5-90aafc821a77-openshift-service-ca\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:28.214956 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:28.214928    2583 generic.go:356] "Generic (PLEG): container finished" podID="863cf327-3188-4b77-8ba5-90aafc821a77" containerID="03813663bd5161200acd1d4382844d15cae07e961eab0181a7ab7a9947835044" exitCode=0
Sep 15 10:44:28.215080 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:28.214991    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-069380-544df7c984-mdlq4"
Sep 15 10:44:28.215080 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:28.215008    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-069380-544df7c984-mdlq4" event={"ID":"863cf327-3188-4b77-8ba5-90aafc821a77","Type":"ContainerDied","Data":"03813663bd5161200acd1d4382844d15cae07e961eab0181a7ab7a9947835044"}
Sep 15 10:44:28.215080 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:28.215066    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-069380-544df7c984-mdlq4" event={"ID":"863cf327-3188-4b77-8ba5-90aafc821a77","Type":"ContainerDied","Data":"160a246ab99b3b99e82c46950a42caecf8e1c4ea118d0408f3f965d6d946e836"}
Sep 15 10:44:28.215236 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:28.215089    2583 scope.go:117] "RemoveContainer" containerID="03813663bd5161200acd1d4382844d15cae07e961eab0181a7ab7a9947835044"
Sep 15 10:44:28.225175 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:28.225149    2583 scope.go:117] "RemoveContainer" containerID="03813663bd5161200acd1d4382844d15cae07e961eab0181a7ab7a9947835044"
Sep 15 10:44:28.225575 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:44:28.225543    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03813663bd5161200acd1d4382844d15cae07e961eab0181a7ab7a9947835044\": container with ID starting with 03813663bd5161200acd1d4382844d15cae07e961eab0181a7ab7a9947835044 not found: ID does not exist" containerID="03813663bd5161200acd1d4382844d15cae07e961eab0181a7ab7a9947835044"
Sep 15 10:44:28.225666 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:28.225589    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03813663bd5161200acd1d4382844d15cae07e961eab0181a7ab7a9947835044"} err="failed to get container status \"03813663bd5161200acd1d4382844d15cae07e961eab0181a7ab7a9947835044\": rpc error: code = NotFound desc = could not find container \"03813663bd5161200acd1d4382844d15cae07e961eab0181a7ab7a9947835044\": container with ID starting with 03813663bd5161200acd1d4382844d15cae07e961eab0181a7ab7a9947835044 not found: ID does not exist"
Sep 15 10:44:28.246382 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:28.246346    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-069380-544df7c984-mdlq4"]
Sep 15 10:44:28.251807 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:28.251775    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-069380-544df7c984-mdlq4"]
Sep 15 10:44:28.330227 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:28.330149    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:28.781684 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:28.781657    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:29.301020 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:29.300987    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="863cf327-3188-4b77-8ba5-90aafc821a77" path="/var/lib/kubelet/pods/863cf327-3188-4b77-8ba5-90aafc821a77/volumes"
Sep 15 10:44:29.478350 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:29.478317    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:30.066994 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:30.066955    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:30.452878 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:30.452840    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl"]
Sep 15 10:44:30.550900 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:30.550867    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:30.697466 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:30.697405    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:31.451145 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:31.451113    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:32.074706 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:32.074669    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:32.670670 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:32.670635    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:34.073252 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:34.073219    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:34.177681 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:34.177649    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:34.293087 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:34.293057    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"]
Sep 15 10:44:34.293299 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:34.293263    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch" podUID="3508e4ea-2813-4c8b-8007-fec91eb76f79" containerName="istio-proxy" containerID="cri-o://59aa3c32c2c55e6f5d46f7441d1c8837fa8a368fdc6982c26c668f7b616b3e92" gracePeriod=30
Sep 15 10:44:34.307333 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:34.307308    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:34.952566 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:34.952536    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:35.774266 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:35.774233    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:35.895692 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:35.895662    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:35.996400 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:35.996367    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:36.400061 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:36.400030    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:37.002197 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:37.002164    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:37.715081 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:37.715054    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:38.493765 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:38.493733    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:39.252180 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:39.252143    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:39.548343 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:39.548318    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:44:39.630150 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:39.630117    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3508e4ea-2813-4c8b-8007-fec91eb76f79-credential-socket\") pod \"3508e4ea-2813-4c8b-8007-fec91eb76f79\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") "
Sep 15 10:44:39.630298 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:39.630165    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3508e4ea-2813-4c8b-8007-fec91eb76f79-istio-envoy\") pod \"3508e4ea-2813-4c8b-8007-fec91eb76f79\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") "
Sep 15 10:44:39.630298 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:39.630200    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3508e4ea-2813-4c8b-8007-fec91eb76f79-istiod-ca-cert\") pod \"3508e4ea-2813-4c8b-8007-fec91eb76f79\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") "
Sep 15 10:44:39.630298 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:39.630246    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3508e4ea-2813-4c8b-8007-fec91eb76f79-istio-token\") pod \"3508e4ea-2813-4c8b-8007-fec91eb76f79\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") "
Sep 15 10:44:39.630298 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:39.630273    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3508e4ea-2813-4c8b-8007-fec91eb76f79-workload-socket\") pod \"3508e4ea-2813-4c8b-8007-fec91eb76f79\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") "
Sep 15 10:44:39.630548 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:39.630301    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ln9p9\" (UniqueName: \"kubernetes.io/projected/3508e4ea-2813-4c8b-8007-fec91eb76f79-kube-api-access-ln9p9\") pod \"3508e4ea-2813-4c8b-8007-fec91eb76f79\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") "
Sep 15 10:44:39.630548 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:39.630336    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3508e4ea-2813-4c8b-8007-fec91eb76f79-istio-data\") pod \"3508e4ea-2813-4c8b-8007-fec91eb76f79\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") "
Sep 15 10:44:39.630548 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:39.630416    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3508e4ea-2813-4c8b-8007-fec91eb76f79-workload-certs\") pod \"3508e4ea-2813-4c8b-8007-fec91eb76f79\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") "
Sep 15 10:44:39.630548 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:39.630485    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3508e4ea-2813-4c8b-8007-fec91eb76f79-istio-podinfo\") pod \"3508e4ea-2813-4c8b-8007-fec91eb76f79\" (UID: \"3508e4ea-2813-4c8b-8007-fec91eb76f79\") "
Sep 15 10:44:39.630771 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:39.630479    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3508e4ea-2813-4c8b-8007-fec91eb76f79-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3508e4ea-2813-4c8b-8007-fec91eb76f79" (UID: "3508e4ea-2813-4c8b-8007-fec91eb76f79"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:44:39.630771 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:39.630580    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3508e4ea-2813-4c8b-8007-fec91eb76f79-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3508e4ea-2813-4c8b-8007-fec91eb76f79" (UID: "3508e4ea-2813-4c8b-8007-fec91eb76f79"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:44:39.630895 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:39.630678    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3508e4ea-2813-4c8b-8007-fec91eb76f79-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3508e4ea-2813-4c8b-8007-fec91eb76f79" (UID: "3508e4ea-2813-4c8b-8007-fec91eb76f79"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:44:39.630895 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:39.630706    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3508e4ea-2813-4c8b-8007-fec91eb76f79-istio-data" (OuterVolumeSpecName: "istio-data") pod "3508e4ea-2813-4c8b-8007-fec91eb76f79" (UID: "3508e4ea-2813-4c8b-8007-fec91eb76f79"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:44:39.630895 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:39.630744    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3508e4ea-2813-4c8b-8007-fec91eb76f79-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3508e4ea-2813-4c8b-8007-fec91eb76f79" (UID: "3508e4ea-2813-4c8b-8007-fec91eb76f79"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:44:39.631061 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:39.630939    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3508e4ea-2813-4c8b-8007-fec91eb76f79-credential-socket\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:39.631061 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:39.630965    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3508e4ea-2813-4c8b-8007-fec91eb76f79-istiod-ca-cert\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:39.631061 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:39.630981    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3508e4ea-2813-4c8b-8007-fec91eb76f79-workload-socket\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:39.631061 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:39.630995    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3508e4ea-2813-4c8b-8007-fec91eb76f79-istio-data\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:39.631061 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:39.631008    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3508e4ea-2813-4c8b-8007-fec91eb76f79-workload-certs\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:39.632563 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:39.632535    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3508e4ea-2813-4c8b-8007-fec91eb76f79-istio-token" (OuterVolumeSpecName: "istio-token") pod "3508e4ea-2813-4c8b-8007-fec91eb76f79" (UID: "3508e4ea-2813-4c8b-8007-fec91eb76f79"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:44:39.632671 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:39.632540    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3508e4ea-2813-4c8b-8007-fec91eb76f79-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3508e4ea-2813-4c8b-8007-fec91eb76f79" (UID: "3508e4ea-2813-4c8b-8007-fec91eb76f79"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:44:39.632992 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:39.632967    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3508e4ea-2813-4c8b-8007-fec91eb76f79-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3508e4ea-2813-4c8b-8007-fec91eb76f79" (UID: "3508e4ea-2813-4c8b-8007-fec91eb76f79"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 10:44:39.633064 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:39.632965    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3508e4ea-2813-4c8b-8007-fec91eb76f79-kube-api-access-ln9p9" (OuterVolumeSpecName: "kube-api-access-ln9p9") pod "3508e4ea-2813-4c8b-8007-fec91eb76f79" (UID: "3508e4ea-2813-4c8b-8007-fec91eb76f79"). InnerVolumeSpecName "kube-api-access-ln9p9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:44:39.732142 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:39.732103    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3508e4ea-2813-4c8b-8007-fec91eb76f79-istio-podinfo\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:39.732142 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:39.732136    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3508e4ea-2813-4c8b-8007-fec91eb76f79-istio-envoy\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:39.732142 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:39.732145    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3508e4ea-2813-4c8b-8007-fec91eb76f79-istio-token\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:39.732381 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:39.732154    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ln9p9\" (UniqueName: \"kubernetes.io/projected/3508e4ea-2813-4c8b-8007-fec91eb76f79-kube-api-access-ln9p9\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:40.288174 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:40.288140    2583 generic.go:356] "Generic (PLEG): container finished" podID="3508e4ea-2813-4c8b-8007-fec91eb76f79" containerID="59aa3c32c2c55e6f5d46f7441d1c8837fa8a368fdc6982c26c668f7b616b3e92" exitCode=0
Sep 15 10:44:40.288174 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:40.288180    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch" event={"ID":"3508e4ea-2813-4c8b-8007-fec91eb76f79","Type":"ContainerDied","Data":"59aa3c32c2c55e6f5d46f7441d1c8837fa8a368fdc6982c26c668f7b616b3e92"}
Sep 15 10:44:40.288392 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:40.288190    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"
Sep 15 10:44:40.288392 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:40.288222    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch" event={"ID":"3508e4ea-2813-4c8b-8007-fec91eb76f79","Type":"ContainerDied","Data":"b23b06adeced77ad4886c02c8594e10d4936c68b6546c07ac9b8e0c2a240fbb7"}
Sep 15 10:44:40.288392 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:40.288244    2583 scope.go:117] "RemoveContainer" containerID="59aa3c32c2c55e6f5d46f7441d1c8837fa8a368fdc6982c26c668f7b616b3e92"
Sep 15 10:44:40.298263 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:40.298245    2583 scope.go:117] "RemoveContainer" containerID="9f7e909f3b5a125216e1f29f8efb50f485aa26d26b5011edfbd4aa1a345b2bd6"
Sep 15 10:44:40.307780 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:40.307759    2583 scope.go:117] "RemoveContainer" containerID="59aa3c32c2c55e6f5d46f7441d1c8837fa8a368fdc6982c26c668f7b616b3e92"
Sep 15 10:44:40.308087 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:44:40.308066    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59aa3c32c2c55e6f5d46f7441d1c8837fa8a368fdc6982c26c668f7b616b3e92\": container with ID starting with 59aa3c32c2c55e6f5d46f7441d1c8837fa8a368fdc6982c26c668f7b616b3e92 not found: ID does not exist" containerID="59aa3c32c2c55e6f5d46f7441d1c8837fa8a368fdc6982c26c668f7b616b3e92"
Sep 15 10:44:40.308149 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:40.308095    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59aa3c32c2c55e6f5d46f7441d1c8837fa8a368fdc6982c26c668f7b616b3e92"} err="failed to get container status \"59aa3c32c2c55e6f5d46f7441d1c8837fa8a368fdc6982c26c668f7b616b3e92\": rpc error: code = NotFound desc = could not find container \"59aa3c32c2c55e6f5d46f7441d1c8837fa8a368fdc6982c26c668f7b616b3e92\": container with ID starting with 59aa3c32c2c55e6f5d46f7441d1c8837fa8a368fdc6982c26c668f7b616b3e92 not found: ID does not exist"
Sep 15 10:44:40.308149 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:40.308111    2583 scope.go:117] "RemoveContainer" containerID="9f7e909f3b5a125216e1f29f8efb50f485aa26d26b5011edfbd4aa1a345b2bd6"
Sep 15 10:44:40.308405 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:44:40.308382    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f7e909f3b5a125216e1f29f8efb50f485aa26d26b5011edfbd4aa1a345b2bd6\": container with ID starting with 9f7e909f3b5a125216e1f29f8efb50f485aa26d26b5011edfbd4aa1a345b2bd6 not found: ID does not exist" containerID="9f7e909f3b5a125216e1f29f8efb50f485aa26d26b5011edfbd4aa1a345b2bd6"
Sep 15 10:44:40.308575 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:40.308415    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f7e909f3b5a125216e1f29f8efb50f485aa26d26b5011edfbd4aa1a345b2bd6"} err="failed to get container status \"9f7e909f3b5a125216e1f29f8efb50f485aa26d26b5011edfbd4aa1a345b2bd6\": rpc error: code = NotFound desc = could not find container \"9f7e909f3b5a125216e1f29f8efb50f485aa26d26b5011edfbd4aa1a345b2bd6\": container with ID starting with 9f7e909f3b5a125216e1f29f8efb50f485aa26d26b5011edfbd4aa1a345b2bd6 not found: ID does not exist"
Sep 15 10:44:40.310487 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:40.310467    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"]
Sep 15 10:44:40.313964 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:40.313944    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-069380-openshift-default-7bbf45cbf8-8xfch"]
Sep 15 10:44:41.301091 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:41.301056    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3508e4ea-2813-4c8b-8007-fec91eb76f79" path="/var/lib/kubelet/pods/3508e4ea-2813-4c8b-8007-fec91eb76f79/volumes"
Sep 15 10:44:43.338421 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:43.338390    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc"]
Sep 15 10:44:43.338844 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:43.338618    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc" podUID="48677bfc-860f-49f9-87b3-a83cce52f760" containerName="maas-api" containerID="cri-o://c5b0f378bea1395c41e143e671aaec7756dd41eb09619661f202e262f9a076f9" gracePeriod=30
Sep 15 10:44:43.378103 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:43.378069    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:43.477329 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:43.477239    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:43.549422 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:43.549388    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:43.607090 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:43.607052    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:43.786811 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:43.786786    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc"
Sep 15 10:44:43.876908 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:43.876873    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k5m5j\" (UniqueName: \"kubernetes.io/projected/48677bfc-860f-49f9-87b3-a83cce52f760-kube-api-access-k5m5j\") pod \"48677bfc-860f-49f9-87b3-a83cce52f760\" (UID: \"48677bfc-860f-49f9-87b3-a83cce52f760\") "
Sep 15 10:44:43.877111 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:43.876933    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48677bfc-860f-49f9-87b3-a83cce52f760-metrics-certs\") pod \"48677bfc-860f-49f9-87b3-a83cce52f760\" (UID: \"48677bfc-860f-49f9-87b3-a83cce52f760\") "
Sep 15 10:44:43.877111 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:43.876976    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48677bfc-860f-49f9-87b3-a83cce52f760-openshift-service-ca\") pod \"48677bfc-860f-49f9-87b3-a83cce52f760\" (UID: \"48677bfc-860f-49f9-87b3-a83cce52f760\") "
Sep 15 10:44:43.877111 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:43.877032    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48677bfc-860f-49f9-87b3-a83cce52f760-maas-api-tls\") pod \"48677bfc-860f-49f9-87b3-a83cce52f760\" (UID: \"48677bfc-860f-49f9-87b3-a83cce52f760\") "
Sep 15 10:44:43.877514 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:43.877392    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48677bfc-860f-49f9-87b3-a83cce52f760-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "48677bfc-860f-49f9-87b3-a83cce52f760" (UID: "48677bfc-860f-49f9-87b3-a83cce52f760"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:44:43.879411 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:43.879380    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48677bfc-860f-49f9-87b3-a83cce52f760-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "48677bfc-860f-49f9-87b3-a83cce52f760" (UID: "48677bfc-860f-49f9-87b3-a83cce52f760"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:44:43.879550 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:43.879503    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48677bfc-860f-49f9-87b3-a83cce52f760-kube-api-access-k5m5j" (OuterVolumeSpecName: "kube-api-access-k5m5j") pod "48677bfc-860f-49f9-87b3-a83cce52f760" (UID: "48677bfc-860f-49f9-87b3-a83cce52f760"). InnerVolumeSpecName "kube-api-access-k5m5j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:44:43.879550 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:43.879522    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48677bfc-860f-49f9-87b3-a83cce52f760-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "48677bfc-860f-49f9-87b3-a83cce52f760" (UID: "48677bfc-860f-49f9-87b3-a83cce52f760"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:44:43.964649 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:43.964614    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:43.978920 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:43.978890    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k5m5j\" (UniqueName: \"kubernetes.io/projected/48677bfc-860f-49f9-87b3-a83cce52f760-kube-api-access-k5m5j\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:43.978920 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:43.978921    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48677bfc-860f-49f9-87b3-a83cce52f760-metrics-certs\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:43.979093 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:43.978934    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48677bfc-860f-49f9-87b3-a83cce52f760-openshift-service-ca\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:43.979093 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:43.978946    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48677bfc-860f-49f9-87b3-a83cce52f760-maas-api-tls\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:44.063788 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:44.063704    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:44.314363 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:44.314276    2583 generic.go:356] "Generic (PLEG): container finished" podID="48677bfc-860f-49f9-87b3-a83cce52f760" containerID="c5b0f378bea1395c41e143e671aaec7756dd41eb09619661f202e262f9a076f9" exitCode=0
Sep 15 10:44:44.314363 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:44.314330    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc"
Sep 15 10:44:44.314571 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:44.314361    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc" event={"ID":"48677bfc-860f-49f9-87b3-a83cce52f760","Type":"ContainerDied","Data":"c5b0f378bea1395c41e143e671aaec7756dd41eb09619661f202e262f9a076f9"}
Sep 15 10:44:44.314571 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:44.314410    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc" event={"ID":"48677bfc-860f-49f9-87b3-a83cce52f760","Type":"ContainerDied","Data":"b1b91fef02d7e6d06791932acd2f8cdaac0a77432ba2ce2fca09f2020933d4c3"}
Sep 15 10:44:44.314571 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:44.314433    2583 scope.go:117] "RemoveContainer" containerID="c5b0f378bea1395c41e143e671aaec7756dd41eb09619661f202e262f9a076f9"
Sep 15 10:44:44.324630 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:44.324603    2583 scope.go:117] "RemoveContainer" containerID="c5b0f378bea1395c41e143e671aaec7756dd41eb09619661f202e262f9a076f9"
Sep 15 10:44:44.324933 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:44:44.324912    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5b0f378bea1395c41e143e671aaec7756dd41eb09619661f202e262f9a076f9\": container with ID starting with c5b0f378bea1395c41e143e671aaec7756dd41eb09619661f202e262f9a076f9 not found: ID does not exist" containerID="c5b0f378bea1395c41e143e671aaec7756dd41eb09619661f202e262f9a076f9"
Sep 15 10:44:44.324992 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:44.324942    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5b0f378bea1395c41e143e671aaec7756dd41eb09619661f202e262f9a076f9"} err="failed to get container status \"c5b0f378bea1395c41e143e671aaec7756dd41eb09619661f202e262f9a076f9\": rpc error: code = NotFound desc = could not find container \"c5b0f378bea1395c41e143e671aaec7756dd41eb09619661f202e262f9a076f9\": container with ID starting with c5b0f378bea1395c41e143e671aaec7756dd41eb09619661f202e262f9a076f9 not found: ID does not exist"
Sep 15 10:44:44.335988 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:44.335962    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc"]
Sep 15 10:44:44.341125 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:44.341101    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-8c1047-7d49fc6744-klkvc"]
Sep 15 10:44:44.687611 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:44.687569    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:45.300659 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:45.300628    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48677bfc-860f-49f9-87b3-a83cce52f760" path="/var/lib/kubelet/pods/48677bfc-860f-49f9-87b3-a83cce52f760/volumes"
Sep 15 10:44:45.308304 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:45.308277    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:45.472499 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:45.472458    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl" podUID="00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39" containerName="main" containerID="cri-o://528a3a791c156a061716d3262199adc10e0f3163374f7b940a94611eb6b0c990" gracePeriod=45
Sep 15 10:44:45.727560 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:45.727530    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl"
Sep 15 10:44:45.794561 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:45.794527    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g99b9\" (UniqueName: \"kubernetes.io/projected/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-kube-api-access-g99b9\") pod \"00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39\" (UID: \"00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39\") "
Sep 15 10:44:45.794561 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:45.794560    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-tmp-dir\") pod \"00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39\" (UID: \"00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39\") "
Sep 15 10:44:45.794818 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:45.794595    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-home\") pod \"00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39\" (UID: \"00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39\") "
Sep 15 10:44:45.794818 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:45.794664    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-model-cache\") pod \"00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39\" (UID: \"00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39\") "
Sep 15 10:44:45.794818 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:45.794704    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-tls-certs\") pod \"00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39\" (UID: \"00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39\") "
Sep 15 10:44:45.794818 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:45.794759    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-dshm\") pod \"00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39\" (UID: \"00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39\") "
Sep 15 10:44:45.795049 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:45.794912    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39" (UID: "00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:44:45.795233 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:45.795212    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-tmp-dir\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:45.795420 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:45.795241    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-home" (OuterVolumeSpecName: "home") pod "00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39" (UID: "00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:44:45.795420 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:45.795258    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-model-cache" (OuterVolumeSpecName: "model-cache") pod "00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39" (UID: "00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:44:45.797104 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:45.797073    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-kube-api-access-g99b9" (OuterVolumeSpecName: "kube-api-access-g99b9") pod "00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39" (UID: "00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39"). InnerVolumeSpecName "kube-api-access-g99b9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:44:45.797232 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:45.797158    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-dshm" (OuterVolumeSpecName: "dshm") pod "00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39" (UID: "00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:44:45.797304 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:45.797236    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39" (UID: "00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:44:45.896835 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:45.896799    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-dshm\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:45.896835 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:45.896836    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g99b9\" (UniqueName: \"kubernetes.io/projected/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-kube-api-access-g99b9\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:45.897029 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:45.896851    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-home\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:45.897029 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:45.896863    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-model-cache\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:45.897029 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:45.896874    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39-tls-certs\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:45.999184 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:45.999154    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:46.065087 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:46.065061    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:46.119306 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:46.119272    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:46.176570 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:46.176466    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:46.281503 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:46.281467    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:46.329175 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:46.329146    2583 generic.go:356] "Generic (PLEG): container finished" podID="00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39" containerID="528a3a791c156a061716d3262199adc10e0f3163374f7b940a94611eb6b0c990" exitCode=0
Sep 15 10:44:46.329373 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:46.329188    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl"
Sep 15 10:44:46.329373 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:46.329235    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl" event={"ID":"00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39","Type":"ContainerDied","Data":"528a3a791c156a061716d3262199adc10e0f3163374f7b940a94611eb6b0c990"}
Sep 15 10:44:46.329373 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:46.329281    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl" event={"ID":"00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39","Type":"ContainerDied","Data":"c838d8d28b60598b8d708a2a74cfb70917883429e7d684ca2f8492952236aa00"}
Sep 15 10:44:46.329373 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:46.329299    2583 scope.go:117] "RemoveContainer" containerID="528a3a791c156a061716d3262199adc10e0f3163374f7b940a94611eb6b0c990"
Sep 15 10:44:46.339187 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:46.339169    2583 scope.go:117] "RemoveContainer" containerID="528a3a791c156a061716d3262199adc10e0f3163374f7b940a94611eb6b0c990"
Sep 15 10:44:46.339469 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:44:46.339436    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"528a3a791c156a061716d3262199adc10e0f3163374f7b940a94611eb6b0c990\": container with ID starting with 528a3a791c156a061716d3262199adc10e0f3163374f7b940a94611eb6b0c990 not found: ID does not exist" containerID="528a3a791c156a061716d3262199adc10e0f3163374f7b940a94611eb6b0c990"
Sep 15 10:44:46.339542 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:46.339481    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"528a3a791c156a061716d3262199adc10e0f3163374f7b940a94611eb6b0c990"} err="failed to get container status \"528a3a791c156a061716d3262199adc10e0f3163374f7b940a94611eb6b0c990\": rpc error: code = NotFound desc = could not find container \"528a3a791c156a061716d3262199adc10e0f3163374f7b940a94611eb6b0c990\": container with ID starting with 528a3a791c156a061716d3262199adc10e0f3163374f7b940a94611eb6b0c990 not found: ID does not exist"
Sep 15 10:44:46.352590 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:46.352545    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl"]
Sep 15 10:44:46.358511 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:46.358432    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-8c1047/distinct-2-w0-8c1047-kserve-74948f6cbf-gl6zl"]
Sep 15 10:44:46.512158 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:46.512075    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:47.088358 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:47.088316    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:47.301376 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:47.301342    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39" path="/var/lib/kubelet/pods/00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39/volumes"
Sep 15 10:44:47.874205 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:47.874155    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:48.607150 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:48.607107    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:49.480403 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:49.480367    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:50.051301 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:50.051269    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:50.907680 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:50.907652    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:51.007702 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:51.007674    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:51.683787 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:51.683744    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:52.319783 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:52.319752    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:52.379953 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:52.379913    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:52.556456 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:52.556410    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"]
Sep 15 10:44:52.556660 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:52.556638    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk" podUID="5e854721-2229-44c8-b91e-2ab76dd1a162" containerName="istio-proxy" containerID="cri-o://bd2f8fb841046c2c9419e0f8bbdf022c27aeb4c8fa57399acc851f96d9055baf" gracePeriod=30
Sep 15 10:44:52.580191 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:52.580125    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:53.080619 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:53.080581    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:53.810396 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:53.810357    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:54.499000 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:54.498963    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:55.198684 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:55.198654    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:55.954113 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:55.954081    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:56.372792 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:56.372696    2583 patch_prober.go:28] interesting pod/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 10:44:56.372792 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:56.372763    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk" podUID="5e854721-2229-44c8-b91e-2ab76dd1a162" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 10:44:56.572523 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:56.572491    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:44:57.811065 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:57.811039    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:44:57.918070 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:57.918032    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e854721-2229-44c8-b91e-2ab76dd1a162-workload-certs\") pod \"5e854721-2229-44c8-b91e-2ab76dd1a162\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") "
Sep 15 10:44:57.918070 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:57.918072    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e854721-2229-44c8-b91e-2ab76dd1a162-istio-data\") pod \"5e854721-2229-44c8-b91e-2ab76dd1a162\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") "
Sep 15 10:44:57.918347 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:57.918119    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e854721-2229-44c8-b91e-2ab76dd1a162-istio-token\") pod \"5e854721-2229-44c8-b91e-2ab76dd1a162\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") "
Sep 15 10:44:57.918347 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:57.918148    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e854721-2229-44c8-b91e-2ab76dd1a162-workload-socket\") pod \"5e854721-2229-44c8-b91e-2ab76dd1a162\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") "
Sep 15 10:44:57.918347 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:57.918190    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e854721-2229-44c8-b91e-2ab76dd1a162-istiod-ca-cert\") pod \"5e854721-2229-44c8-b91e-2ab76dd1a162\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") "
Sep 15 10:44:57.918347 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:57.918244    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e854721-2229-44c8-b91e-2ab76dd1a162-credential-socket\") pod \"5e854721-2229-44c8-b91e-2ab76dd1a162\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") "
Sep 15 10:44:57.918347 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:57.918297    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cc99w\" (UniqueName: \"kubernetes.io/projected/5e854721-2229-44c8-b91e-2ab76dd1a162-kube-api-access-cc99w\") pod \"5e854721-2229-44c8-b91e-2ab76dd1a162\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") "
Sep 15 10:44:57.918656 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:57.918418    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e854721-2229-44c8-b91e-2ab76dd1a162-istio-envoy\") pod \"5e854721-2229-44c8-b91e-2ab76dd1a162\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") "
Sep 15 10:44:57.918656 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:57.918379    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e854721-2229-44c8-b91e-2ab76dd1a162-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5e854721-2229-44c8-b91e-2ab76dd1a162" (UID: "5e854721-2229-44c8-b91e-2ab76dd1a162"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:44:57.918656 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:57.918494    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e854721-2229-44c8-b91e-2ab76dd1a162-istio-podinfo\") pod \"5e854721-2229-44c8-b91e-2ab76dd1a162\" (UID: \"5e854721-2229-44c8-b91e-2ab76dd1a162\") "
Sep 15 10:44:57.918656 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:57.918477    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e854721-2229-44c8-b91e-2ab76dd1a162-istio-data" (OuterVolumeSpecName: "istio-data") pod "5e854721-2229-44c8-b91e-2ab76dd1a162" (UID: "5e854721-2229-44c8-b91e-2ab76dd1a162"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:44:57.918656 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:57.918601    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e854721-2229-44c8-b91e-2ab76dd1a162-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5e854721-2229-44c8-b91e-2ab76dd1a162" (UID: "5e854721-2229-44c8-b91e-2ab76dd1a162"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:44:57.918927 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:57.918691    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e854721-2229-44c8-b91e-2ab76dd1a162-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5e854721-2229-44c8-b91e-2ab76dd1a162" (UID: "5e854721-2229-44c8-b91e-2ab76dd1a162"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:44:57.918927 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:57.918794    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e854721-2229-44c8-b91e-2ab76dd1a162-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5e854721-2229-44c8-b91e-2ab76dd1a162" (UID: "5e854721-2229-44c8-b91e-2ab76dd1a162"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:44:57.919035 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:57.919000    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e854721-2229-44c8-b91e-2ab76dd1a162-credential-socket\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:57.919035 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:57.919022    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e854721-2229-44c8-b91e-2ab76dd1a162-workload-certs\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:57.919138 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:57.919036    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e854721-2229-44c8-b91e-2ab76dd1a162-istio-data\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:57.919138 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:57.919047    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e854721-2229-44c8-b91e-2ab76dd1a162-workload-socket\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:57.919138 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:57.919059    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e854721-2229-44c8-b91e-2ab76dd1a162-istiod-ca-cert\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:57.920659 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:57.920638    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e854721-2229-44c8-b91e-2ab76dd1a162-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5e854721-2229-44c8-b91e-2ab76dd1a162" (UID: "5e854721-2229-44c8-b91e-2ab76dd1a162"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:44:57.920755 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:57.920672    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5e854721-2229-44c8-b91e-2ab76dd1a162-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5e854721-2229-44c8-b91e-2ab76dd1a162" (UID: "5e854721-2229-44c8-b91e-2ab76dd1a162"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 10:44:57.920955 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:57.920929    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e854721-2229-44c8-b91e-2ab76dd1a162-istio-token" (OuterVolumeSpecName: "istio-token") pod "5e854721-2229-44c8-b91e-2ab76dd1a162" (UID: "5e854721-2229-44c8-b91e-2ab76dd1a162"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:44:57.920955 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:57.920940    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e854721-2229-44c8-b91e-2ab76dd1a162-kube-api-access-cc99w" (OuterVolumeSpecName: "kube-api-access-cc99w") pod "5e854721-2229-44c8-b91e-2ab76dd1a162" (UID: "5e854721-2229-44c8-b91e-2ab76dd1a162"). InnerVolumeSpecName "kube-api-access-cc99w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:44:58.019778 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:58.019740    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e854721-2229-44c8-b91e-2ab76dd1a162-istio-token\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:58.019778 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:58.019769    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cc99w\" (UniqueName: \"kubernetes.io/projected/5e854721-2229-44c8-b91e-2ab76dd1a162-kube-api-access-cc99w\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:58.019778 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:58.019779    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e854721-2229-44c8-b91e-2ab76dd1a162-istio-envoy\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:58.019778 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:58.019787    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e854721-2229-44c8-b91e-2ab76dd1a162-istio-podinfo\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:44:58.406348 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:58.406313    2583 generic.go:356] "Generic (PLEG): container finished" podID="5e854721-2229-44c8-b91e-2ab76dd1a162" containerID="bd2f8fb841046c2c9419e0f8bbdf022c27aeb4c8fa57399acc851f96d9055baf" exitCode=0
Sep 15 10:44:58.406574 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:58.406365    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"
Sep 15 10:44:58.406574 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:58.406392    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk" event={"ID":"5e854721-2229-44c8-b91e-2ab76dd1a162","Type":"ContainerDied","Data":"bd2f8fb841046c2c9419e0f8bbdf022c27aeb4c8fa57399acc851f96d9055baf"}
Sep 15 10:44:58.406574 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:58.406460    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk" event={"ID":"5e854721-2229-44c8-b91e-2ab76dd1a162","Type":"ContainerDied","Data":"7e4f7a347a42e313e37f31c6953102c3d5b09ea6e30abf18a657287a8c7731a3"}
Sep 15 10:44:58.406574 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:58.406492    2583 scope.go:117] "RemoveContainer" containerID="bd2f8fb841046c2c9419e0f8bbdf022c27aeb4c8fa57399acc851f96d9055baf"
Sep 15 10:44:58.416509 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:58.416490    2583 scope.go:117] "RemoveContainer" containerID="ca4dc2de215d2a79a4371d21d7aab8643722161d4440b8edf1531a715a191dde"
Sep 15 10:44:58.425191 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:58.425175    2583 scope.go:117] "RemoveContainer" containerID="bd2f8fb841046c2c9419e0f8bbdf022c27aeb4c8fa57399acc851f96d9055baf"
Sep 15 10:44:58.425599 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:44:58.425574    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd2f8fb841046c2c9419e0f8bbdf022c27aeb4c8fa57399acc851f96d9055baf\": container with ID starting with bd2f8fb841046c2c9419e0f8bbdf022c27aeb4c8fa57399acc851f96d9055baf not found: ID does not exist" containerID="bd2f8fb841046c2c9419e0f8bbdf022c27aeb4c8fa57399acc851f96d9055baf"
Sep 15 10:44:58.425702 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:58.425612    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd2f8fb841046c2c9419e0f8bbdf022c27aeb4c8fa57399acc851f96d9055baf"} err="failed to get container status \"bd2f8fb841046c2c9419e0f8bbdf022c27aeb4c8fa57399acc851f96d9055baf\": rpc error: code = NotFound desc = could not find container \"bd2f8fb841046c2c9419e0f8bbdf022c27aeb4c8fa57399acc851f96d9055baf\": container with ID starting with bd2f8fb841046c2c9419e0f8bbdf022c27aeb4c8fa57399acc851f96d9055baf not found: ID does not exist"
Sep 15 10:44:58.425702 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:58.425638    2583 scope.go:117] "RemoveContainer" containerID="ca4dc2de215d2a79a4371d21d7aab8643722161d4440b8edf1531a715a191dde"
Sep 15 10:44:58.426088 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:44:58.426044    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca4dc2de215d2a79a4371d21d7aab8643722161d4440b8edf1531a715a191dde\": container with ID starting with ca4dc2de215d2a79a4371d21d7aab8643722161d4440b8edf1531a715a191dde not found: ID does not exist" containerID="ca4dc2de215d2a79a4371d21d7aab8643722161d4440b8edf1531a715a191dde"
Sep 15 10:44:58.426636 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:58.426080    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca4dc2de215d2a79a4371d21d7aab8643722161d4440b8edf1531a715a191dde"} err="failed to get container status \"ca4dc2de215d2a79a4371d21d7aab8643722161d4440b8edf1531a715a191dde\": rpc error: code = NotFound desc = could not find container \"ca4dc2de215d2a79a4371d21d7aab8643722161d4440b8edf1531a715a191dde\": container with ID starting with ca4dc2de215d2a79a4371d21d7aab8643722161d4440b8edf1531a715a191dde not found: ID does not exist"
Sep 15 10:44:58.428829 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:58.428805    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"]
Sep 15 10:44:58.432534 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:58.432515    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-30651f-openshift-default-65f444fcbd-s4kkk"]
Sep 15 10:44:59.300498 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:44:59.300466    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e854721-2229-44c8-b91e-2ab76dd1a162" path="/var/lib/kubelet/pods/5e854721-2229-44c8-b91e-2ab76dd1a162/volumes"
Sep 15 10:45:03.423982 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:03.423945    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87"]
Sep 15 10:45:03.424412 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:03.424228    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87" podUID="d6e82451-e1e6-4a82-be75-dde9c4c906b8" containerName="maas-api" containerID="cri-o://6f4e50fbd0c22d2fa3a9f27cf492904d198b7fa70a40846c4e42c64bf8b38404" gracePeriod=30
Sep 15 10:45:03.455703 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:03.455673    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:45:03.551708 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:03.551676    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:45:03.625056 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:03.625021    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:45:03.669545 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:03.669511    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87"
Sep 15 10:45:03.679942 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:03.679863    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:45:03.776350 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:03.776318    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdpss\" (UniqueName: \"kubernetes.io/projected/d6e82451-e1e6-4a82-be75-dde9c4c906b8-kube-api-access-kdpss\") pod \"d6e82451-e1e6-4a82-be75-dde9c4c906b8\" (UID: \"d6e82451-e1e6-4a82-be75-dde9c4c906b8\") "
Sep 15 10:45:03.776635 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:03.776375    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6e82451-e1e6-4a82-be75-dde9c4c906b8-metrics-certs\") pod \"d6e82451-e1e6-4a82-be75-dde9c4c906b8\" (UID: \"d6e82451-e1e6-4a82-be75-dde9c4c906b8\") "
Sep 15 10:45:03.776635 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:03.776429    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6e82451-e1e6-4a82-be75-dde9c4c906b8-openshift-service-ca\") pod \"d6e82451-e1e6-4a82-be75-dde9c4c906b8\" (UID: \"d6e82451-e1e6-4a82-be75-dde9c4c906b8\") "
Sep 15 10:45:03.776635 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:03.776517    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6e82451-e1e6-4a82-be75-dde9c4c906b8-maas-api-tls\") pod \"d6e82451-e1e6-4a82-be75-dde9c4c906b8\" (UID: \"d6e82451-e1e6-4a82-be75-dde9c4c906b8\") "
Sep 15 10:45:03.776839 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:03.776786    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6e82451-e1e6-4a82-be75-dde9c4c906b8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d6e82451-e1e6-4a82-be75-dde9c4c906b8" (UID: "d6e82451-e1e6-4a82-be75-dde9c4c906b8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:45:03.778584 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:03.778557    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6e82451-e1e6-4a82-be75-dde9c4c906b8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d6e82451-e1e6-4a82-be75-dde9c4c906b8" (UID: "d6e82451-e1e6-4a82-be75-dde9c4c906b8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:45:03.778960 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:03.778940    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6e82451-e1e6-4a82-be75-dde9c4c906b8-kube-api-access-kdpss" (OuterVolumeSpecName: "kube-api-access-kdpss") pod "d6e82451-e1e6-4a82-be75-dde9c4c906b8" (UID: "d6e82451-e1e6-4a82-be75-dde9c4c906b8"). InnerVolumeSpecName "kube-api-access-kdpss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:45:03.779022 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:03.778941    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6e82451-e1e6-4a82-be75-dde9c4c906b8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d6e82451-e1e6-4a82-be75-dde9c4c906b8" (UID: "d6e82451-e1e6-4a82-be75-dde9c4c906b8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:45:03.877414 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:03.877368    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdpss\" (UniqueName: \"kubernetes.io/projected/d6e82451-e1e6-4a82-be75-dde9c4c906b8-kube-api-access-kdpss\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:45:03.877414 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:03.877406    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6e82451-e1e6-4a82-be75-dde9c4c906b8-metrics-certs\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:45:03.877414 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:03.877416    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6e82451-e1e6-4a82-be75-dde9c4c906b8-openshift-service-ca\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:45:03.877414 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:03.877424    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6e82451-e1e6-4a82-be75-dde9c4c906b8-maas-api-tls\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:45:04.004528 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:04.004418    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:45:04.170108 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:04.170077    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:45:04.445577 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:04.445541    2583 generic.go:356] "Generic (PLEG): container finished" podID="d6e82451-e1e6-4a82-be75-dde9c4c906b8" containerID="6f4e50fbd0c22d2fa3a9f27cf492904d198b7fa70a40846c4e42c64bf8b38404" exitCode=0
Sep 15 10:45:04.445998 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:04.445597    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87"
Sep 15 10:45:04.445998 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:04.445629    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87" event={"ID":"d6e82451-e1e6-4a82-be75-dde9c4c906b8","Type":"ContainerDied","Data":"6f4e50fbd0c22d2fa3a9f27cf492904d198b7fa70a40846c4e42c64bf8b38404"}
Sep 15 10:45:04.445998 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:04.445689    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87" event={"ID":"d6e82451-e1e6-4a82-be75-dde9c4c906b8","Type":"ContainerDied","Data":"329c169345c74203f87c17d76bff9e939667d001d80498d9985880cb66dedaf7"}
Sep 15 10:45:04.445998 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:04.445721    2583 scope.go:117] "RemoveContainer" containerID="6f4e50fbd0c22d2fa3a9f27cf492904d198b7fa70a40846c4e42c64bf8b38404"
Sep 15 10:45:04.457772 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:04.457749    2583 scope.go:117] "RemoveContainer" containerID="6f4e50fbd0c22d2fa3a9f27cf492904d198b7fa70a40846c4e42c64bf8b38404"
Sep 15 10:45:04.458202 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:45:04.458182    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f4e50fbd0c22d2fa3a9f27cf492904d198b7fa70a40846c4e42c64bf8b38404\": container with ID starting with 6f4e50fbd0c22d2fa3a9f27cf492904d198b7fa70a40846c4e42c64bf8b38404 not found: ID does not exist" containerID="6f4e50fbd0c22d2fa3a9f27cf492904d198b7fa70a40846c4e42c64bf8b38404"
Sep 15 10:45:04.458268 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:04.458212    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f4e50fbd0c22d2fa3a9f27cf492904d198b7fa70a40846c4e42c64bf8b38404"} err="failed to get container status \"6f4e50fbd0c22d2fa3a9f27cf492904d198b7fa70a40846c4e42c64bf8b38404\": rpc error: code = NotFound desc = could not find container \"6f4e50fbd0c22d2fa3a9f27cf492904d198b7fa70a40846c4e42c64bf8b38404\": container with ID starting with 6f4e50fbd0c22d2fa3a9f27cf492904d198b7fa70a40846c4e42c64bf8b38404 not found: ID does not exist"
Sep 15 10:45:04.468222 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:04.468186    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87"]
Sep 15 10:45:04.471706 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:04.471680    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-ecf56f-6db6c748db-2mz87"]
Sep 15 10:45:04.792147 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:04.792060    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:45:05.301383 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:05.301346    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6e82451-e1e6-4a82-be75-dde9c4c906b8" path="/var/lib/kubelet/pods/d6e82451-e1e6-4a82-be75-dde9c4c906b8/volumes"
Sep 15 10:45:05.377328 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:05.377289    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:45:06.266788 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:06.266752    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:45:07.718675 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:07.718621    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:45:08.295424 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:08.295391    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:45:08.393472 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:08.393413    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:45:10.193468 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:10.193407    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:45:10.574351 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:10.574267    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:45:10.673490 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:10.673432    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:45:14.235692 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:14.235649    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:45:31.886681 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:31.886646    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:45:31.987301 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:31.987269    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:45:47.358688 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:45:47.358649    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:46:17.918808 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:17.918770    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:46:18.049249 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:18.049212    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv"]
Sep 15 10:46:18.068796 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:18.068753    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:46:18.773017 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:18.772973    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:46:19.451647 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:19.451592    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:46:19.548028 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:19.547995    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6"]
Sep 15 10:46:19.566602 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:19.566569    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:46:20.070875 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:20.070841    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:46:20.786882 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:20.786851    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:46:22.977223 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:22.977184    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:46:33.069004 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.068890    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv" podUID="96039a67-84c4-4287-acbf-88425dca4d3b" containerName="main" containerID="cri-o://3d37dad555cd4f3ebe749c9454fb573209aa493dc079800581505c5f3e35ccbd" gracePeriod=45
Sep 15 10:46:33.317383 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.317357    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv"
Sep 15 10:46:33.485199 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.485164    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96039a67-84c4-4287-acbf-88425dca4d3b-tmp-dir\") pod \"96039a67-84c4-4287-acbf-88425dca4d3b\" (UID: \"96039a67-84c4-4287-acbf-88425dca4d3b\") "
Sep 15 10:46:33.485411 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.485272    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzfjz\" (UniqueName: \"kubernetes.io/projected/96039a67-84c4-4287-acbf-88425dca4d3b-kube-api-access-nzfjz\") pod \"96039a67-84c4-4287-acbf-88425dca4d3b\" (UID: \"96039a67-84c4-4287-acbf-88425dca4d3b\") "
Sep 15 10:46:33.485411 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.485393    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96039a67-84c4-4287-acbf-88425dca4d3b-model-cache\") pod \"96039a67-84c4-4287-acbf-88425dca4d3b\" (UID: \"96039a67-84c4-4287-acbf-88425dca4d3b\") "
Sep 15 10:46:33.485562 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.485425    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96039a67-84c4-4287-acbf-88425dca4d3b-tls-certs\") pod \"96039a67-84c4-4287-acbf-88425dca4d3b\" (UID: \"96039a67-84c4-4287-acbf-88425dca4d3b\") "
Sep 15 10:46:33.485562 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.485507    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96039a67-84c4-4287-acbf-88425dca4d3b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "96039a67-84c4-4287-acbf-88425dca4d3b" (UID: "96039a67-84c4-4287-acbf-88425dca4d3b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:46:33.485663 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.485569    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96039a67-84c4-4287-acbf-88425dca4d3b-dshm\") pod \"96039a67-84c4-4287-acbf-88425dca4d3b\" (UID: \"96039a67-84c4-4287-acbf-88425dca4d3b\") "
Sep 15 10:46:33.485663 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.485610    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96039a67-84c4-4287-acbf-88425dca4d3b-home\") pod \"96039a67-84c4-4287-acbf-88425dca4d3b\" (UID: \"96039a67-84c4-4287-acbf-88425dca4d3b\") "
Sep 15 10:46:33.485778 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.485716    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96039a67-84c4-4287-acbf-88425dca4d3b-model-cache" (OuterVolumeSpecName: "model-cache") pod "96039a67-84c4-4287-acbf-88425dca4d3b" (UID: "96039a67-84c4-4287-acbf-88425dca4d3b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:46:33.485946 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.485918    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96039a67-84c4-4287-acbf-88425dca4d3b-home" (OuterVolumeSpecName: "home") pod "96039a67-84c4-4287-acbf-88425dca4d3b" (UID: "96039a67-84c4-4287-acbf-88425dca4d3b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:46:33.486050 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.486027    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96039a67-84c4-4287-acbf-88425dca4d3b-home\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:46:33.486111 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.486057    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96039a67-84c4-4287-acbf-88425dca4d3b-tmp-dir\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:46:33.486111 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.486074    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96039a67-84c4-4287-acbf-88425dca4d3b-model-cache\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:46:33.487658 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.487631    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96039a67-84c4-4287-acbf-88425dca4d3b-dshm" (OuterVolumeSpecName: "dshm") pod "96039a67-84c4-4287-acbf-88425dca4d3b" (UID: "96039a67-84c4-4287-acbf-88425dca4d3b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:46:33.487658 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.487649    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96039a67-84c4-4287-acbf-88425dca4d3b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "96039a67-84c4-4287-acbf-88425dca4d3b" (UID: "96039a67-84c4-4287-acbf-88425dca4d3b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:46:33.487818 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.487702    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96039a67-84c4-4287-acbf-88425dca4d3b-kube-api-access-nzfjz" (OuterVolumeSpecName: "kube-api-access-nzfjz") pod "96039a67-84c4-4287-acbf-88425dca4d3b" (UID: "96039a67-84c4-4287-acbf-88425dca4d3b"). InnerVolumeSpecName "kube-api-access-nzfjz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:46:33.521676 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.521641    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d"]
Sep 15 10:46:33.521924 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.521868    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d" podUID="0eabace0-ccad-44ee-a69f-ab95ed30a1c4" containerName="maas-api" containerID="cri-o://1137483d61c11df6a10fb6d17b058d9c982efb10b8e438491f49ebaba00ac204" gracePeriod=30
Sep 15 10:46:33.557714 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.557676    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:46:33.587413 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.587368    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzfjz\" (UniqueName: \"kubernetes.io/projected/96039a67-84c4-4287-acbf-88425dca4d3b-kube-api-access-nzfjz\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:46:33.587413 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.587410    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96039a67-84c4-4287-acbf-88425dca4d3b-tls-certs\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:46:33.587413 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.587425    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96039a67-84c4-4287-acbf-88425dca4d3b-dshm\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:46:33.622824 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.622783    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:46:33.689903 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.689867    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:46:33.761160 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.761072    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:46:33.962215 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.962189    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d"
Sep 15 10:46:33.984957 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.984923    2583 generic.go:356] "Generic (PLEG): container finished" podID="96039a67-84c4-4287-acbf-88425dca4d3b" containerID="3d37dad555cd4f3ebe749c9454fb573209aa493dc079800581505c5f3e35ccbd" exitCode=0
Sep 15 10:46:33.984957 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.984962    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv"
Sep 15 10:46:33.985209 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.984999    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv" event={"ID":"96039a67-84c4-4287-acbf-88425dca4d3b","Type":"ContainerDied","Data":"3d37dad555cd4f3ebe749c9454fb573209aa493dc079800581505c5f3e35ccbd"}
Sep 15 10:46:33.985209 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.985041    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv" event={"ID":"96039a67-84c4-4287-acbf-88425dca4d3b","Type":"ContainerDied","Data":"c1c50662f46ba50255318883ac870efc458700d2b2745c1d0e022d82f7643af4"}
Sep 15 10:46:33.985209 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.985061    2583 scope.go:117] "RemoveContainer" containerID="3d37dad555cd4f3ebe749c9454fb573209aa493dc079800581505c5f3e35ccbd"
Sep 15 10:46:33.987383 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.987354    2583 generic.go:356] "Generic (PLEG): container finished" podID="0eabace0-ccad-44ee-a69f-ab95ed30a1c4" containerID="1137483d61c11df6a10fb6d17b058d9c982efb10b8e438491f49ebaba00ac204" exitCode=0
Sep 15 10:46:33.987567 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.987403    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d"
Sep 15 10:46:33.987567 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.987420    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d" event={"ID":"0eabace0-ccad-44ee-a69f-ab95ed30a1c4","Type":"ContainerDied","Data":"1137483d61c11df6a10fb6d17b058d9c982efb10b8e438491f49ebaba00ac204"}
Sep 15 10:46:33.987567 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.987553    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d" event={"ID":"0eabace0-ccad-44ee-a69f-ab95ed30a1c4","Type":"ContainerDied","Data":"e51c1cb163f13777d49c94e6198a90c5ad5cd987f3eedd49189d740097017c84"}
Sep 15 10:46:33.996909 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.996885    2583 scope.go:117] "RemoveContainer" containerID="3d37dad555cd4f3ebe749c9454fb573209aa493dc079800581505c5f3e35ccbd"
Sep 15 10:46:33.997305 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:46:33.997275    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d37dad555cd4f3ebe749c9454fb573209aa493dc079800581505c5f3e35ccbd\": container with ID starting with 3d37dad555cd4f3ebe749c9454fb573209aa493dc079800581505c5f3e35ccbd not found: ID does not exist" containerID="3d37dad555cd4f3ebe749c9454fb573209aa493dc079800581505c5f3e35ccbd"
Sep 15 10:46:33.997436 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.997312    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d37dad555cd4f3ebe749c9454fb573209aa493dc079800581505c5f3e35ccbd"} err="failed to get container status \"3d37dad555cd4f3ebe749c9454fb573209aa493dc079800581505c5f3e35ccbd\": rpc error: code = NotFound desc = could not find container \"3d37dad555cd4f3ebe749c9454fb573209aa493dc079800581505c5f3e35ccbd\": container with ID starting with 3d37dad555cd4f3ebe749c9454fb573209aa493dc079800581505c5f3e35ccbd not found: ID does not exist"
Sep 15 10:46:33.997436 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:33.997331    2583 scope.go:117] "RemoveContainer" containerID="1137483d61c11df6a10fb6d17b058d9c982efb10b8e438491f49ebaba00ac204"
Sep 15 10:46:34.007142 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.007118    2583 scope.go:117] "RemoveContainer" containerID="1137483d61c11df6a10fb6d17b058d9c982efb10b8e438491f49ebaba00ac204"
Sep 15 10:46:34.007566 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:46:34.007543    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1137483d61c11df6a10fb6d17b058d9c982efb10b8e438491f49ebaba00ac204\": container with ID starting with 1137483d61c11df6a10fb6d17b058d9c982efb10b8e438491f49ebaba00ac204 not found: ID does not exist" containerID="1137483d61c11df6a10fb6d17b058d9c982efb10b8e438491f49ebaba00ac204"
Sep 15 10:46:34.007645 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.007573    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1137483d61c11df6a10fb6d17b058d9c982efb10b8e438491f49ebaba00ac204"} err="failed to get container status \"1137483d61c11df6a10fb6d17b058d9c982efb10b8e438491f49ebaba00ac204\": rpc error: code = NotFound desc = could not find container \"1137483d61c11df6a10fb6d17b058d9c982efb10b8e438491f49ebaba00ac204\": container with ID starting with 1137483d61c11df6a10fb6d17b058d9c982efb10b8e438491f49ebaba00ac204 not found: ID does not exist"
Sep 15 10:46:34.007970 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.007940    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv"]
Sep 15 10:46:34.011959 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.011901    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-83e814/auth-test-model-83e814-kserve-5bc975dbdc-wzpzv"]
Sep 15 10:46:34.086190 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.086162    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:46:34.092267 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.092238    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0eabace0-ccad-44ee-a69f-ab95ed30a1c4-maas-api-tls\") pod \"0eabace0-ccad-44ee-a69f-ab95ed30a1c4\" (UID: \"0eabace0-ccad-44ee-a69f-ab95ed30a1c4\") "
Sep 15 10:46:34.092424 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.092390    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0eabace0-ccad-44ee-a69f-ab95ed30a1c4-openshift-service-ca\") pod \"0eabace0-ccad-44ee-a69f-ab95ed30a1c4\" (UID: \"0eabace0-ccad-44ee-a69f-ab95ed30a1c4\") "
Sep 15 10:46:34.092529 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.092424    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tfxdc\" (UniqueName: \"kubernetes.io/projected/0eabace0-ccad-44ee-a69f-ab95ed30a1c4-kube-api-access-tfxdc\") pod \"0eabace0-ccad-44ee-a69f-ab95ed30a1c4\" (UID: \"0eabace0-ccad-44ee-a69f-ab95ed30a1c4\") "
Sep 15 10:46:34.092529 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.092519    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0eabace0-ccad-44ee-a69f-ab95ed30a1c4-metrics-certs\") pod \"0eabace0-ccad-44ee-a69f-ab95ed30a1c4\" (UID: \"0eabace0-ccad-44ee-a69f-ab95ed30a1c4\") "
Sep 15 10:46:34.092795 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.092765    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0eabace0-ccad-44ee-a69f-ab95ed30a1c4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0eabace0-ccad-44ee-a69f-ab95ed30a1c4" (UID: "0eabace0-ccad-44ee-a69f-ab95ed30a1c4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:46:34.094456 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.094426    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0eabace0-ccad-44ee-a69f-ab95ed30a1c4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0eabace0-ccad-44ee-a69f-ab95ed30a1c4" (UID: "0eabace0-ccad-44ee-a69f-ab95ed30a1c4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:46:34.094571 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.094545    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0eabace0-ccad-44ee-a69f-ab95ed30a1c4-kube-api-access-tfxdc" (OuterVolumeSpecName: "kube-api-access-tfxdc") pod "0eabace0-ccad-44ee-a69f-ab95ed30a1c4" (UID: "0eabace0-ccad-44ee-a69f-ab95ed30a1c4"). InnerVolumeSpecName "kube-api-access-tfxdc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:46:34.094697 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.094681    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0eabace0-ccad-44ee-a69f-ab95ed30a1c4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0eabace0-ccad-44ee-a69f-ab95ed30a1c4" (UID: "0eabace0-ccad-44ee-a69f-ab95ed30a1c4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:46:34.186138 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.186095    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:46:34.193263 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.193238    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0eabace0-ccad-44ee-a69f-ab95ed30a1c4-openshift-service-ca\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:46:34.193263 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.193262    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tfxdc\" (UniqueName: \"kubernetes.io/projected/0eabace0-ccad-44ee-a69f-ab95ed30a1c4-kube-api-access-tfxdc\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:46:34.193466 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.193275    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0eabace0-ccad-44ee-a69f-ab95ed30a1c4-metrics-certs\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:46:34.193466 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.193283    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0eabace0-ccad-44ee-a69f-ab95ed30a1c4-maas-api-tls\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:46:34.310485 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.310386    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d"]
Sep 15 10:46:34.312728 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.312705    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-ff0655-6c77d8c679-8gj5d"]
Sep 15 10:46:34.568275 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.568177    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6" podUID="20a58489-4663-478f-8a26-09468396c11e" containerName="main" containerID="cri-o://0db213345446457017eee6584e68a0df90b3430fe277877f5ac08bb81725dfba" gracePeriod=45
Sep 15 10:46:34.810782 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.810755    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6"
Sep 15 10:46:34.884415 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.884364    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:46:34.898954 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.898922    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20a58489-4663-478f-8a26-09468396c11e-tls-certs\") pod \"20a58489-4663-478f-8a26-09468396c11e\" (UID: \"20a58489-4663-478f-8a26-09468396c11e\") "
Sep 15 10:46:34.899210 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.899004    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7nkjn\" (UniqueName: \"kubernetes.io/projected/20a58489-4663-478f-8a26-09468396c11e-kube-api-access-7nkjn\") pod \"20a58489-4663-478f-8a26-09468396c11e\" (UID: \"20a58489-4663-478f-8a26-09468396c11e\") "
Sep 15 10:46:34.899210 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.899040    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20a58489-4663-478f-8a26-09468396c11e-model-cache\") pod \"20a58489-4663-478f-8a26-09468396c11e\" (UID: \"20a58489-4663-478f-8a26-09468396c11e\") "
Sep 15 10:46:34.899210 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.899058    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20a58489-4663-478f-8a26-09468396c11e-tmp-dir\") pod \"20a58489-4663-478f-8a26-09468396c11e\" (UID: \"20a58489-4663-478f-8a26-09468396c11e\") "
Sep 15 10:46:34.899210 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.899102    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20a58489-4663-478f-8a26-09468396c11e-dshm\") pod \"20a58489-4663-478f-8a26-09468396c11e\" (UID: \"20a58489-4663-478f-8a26-09468396c11e\") "
Sep 15 10:46:34.899210 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.899145    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20a58489-4663-478f-8a26-09468396c11e-home\") pod \"20a58489-4663-478f-8a26-09468396c11e\" (UID: \"20a58489-4663-478f-8a26-09468396c11e\") "
Sep 15 10:46:34.899531 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.899349    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20a58489-4663-478f-8a26-09468396c11e-model-cache" (OuterVolumeSpecName: "model-cache") pod "20a58489-4663-478f-8a26-09468396c11e" (UID: "20a58489-4663-478f-8a26-09468396c11e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:46:34.899531 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.899362    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20a58489-4663-478f-8a26-09468396c11e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "20a58489-4663-478f-8a26-09468396c11e" (UID: "20a58489-4663-478f-8a26-09468396c11e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:46:34.899531 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.899399    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20a58489-4663-478f-8a26-09468396c11e-home" (OuterVolumeSpecName: "home") pod "20a58489-4663-478f-8a26-09468396c11e" (UID: "20a58489-4663-478f-8a26-09468396c11e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:46:34.899661 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.899527    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20a58489-4663-478f-8a26-09468396c11e-model-cache\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:46:34.899661 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.899550    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20a58489-4663-478f-8a26-09468396c11e-tmp-dir\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:46:34.899661 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.899563    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20a58489-4663-478f-8a26-09468396c11e-home\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:46:34.901311 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.901292    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20a58489-4663-478f-8a26-09468396c11e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "20a58489-4663-478f-8a26-09468396c11e" (UID: "20a58489-4663-478f-8a26-09468396c11e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:46:34.901385 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.901349    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20a58489-4663-478f-8a26-09468396c11e-dshm" (OuterVolumeSpecName: "dshm") pod "20a58489-4663-478f-8a26-09468396c11e" (UID: "20a58489-4663-478f-8a26-09468396c11e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:46:34.901385 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.901362    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20a58489-4663-478f-8a26-09468396c11e-kube-api-access-7nkjn" (OuterVolumeSpecName: "kube-api-access-7nkjn") pod "20a58489-4663-478f-8a26-09468396c11e" (UID: "20a58489-4663-478f-8a26-09468396c11e"). InnerVolumeSpecName "kube-api-access-7nkjn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:46:34.995020 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.994980    2583 generic.go:356] "Generic (PLEG): container finished" podID="20a58489-4663-478f-8a26-09468396c11e" containerID="0db213345446457017eee6584e68a0df90b3430fe277877f5ac08bb81725dfba" exitCode=0
Sep 15 10:46:34.995020 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.995023    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6"
Sep 15 10:46:34.995277 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.995069    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6" event={"ID":"20a58489-4663-478f-8a26-09468396c11e","Type":"ContainerDied","Data":"0db213345446457017eee6584e68a0df90b3430fe277877f5ac08bb81725dfba"}
Sep 15 10:46:34.995277 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.995110    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6" event={"ID":"20a58489-4663-478f-8a26-09468396c11e","Type":"ContainerDied","Data":"34368fa7d700fc9848a3b9f3fd6cb394c9ed35c3d560661eaa14bb4f42c76039"}
Sep 15 10:46:34.995277 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:34.995128    2583 scope.go:117] "RemoveContainer" containerID="0db213345446457017eee6584e68a0df90b3430fe277877f5ac08bb81725dfba"
Sep 15 10:46:35.000409 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:35.000373    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20a58489-4663-478f-8a26-09468396c11e-tls-certs\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:46:35.000409 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:35.000407    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7nkjn\" (UniqueName: \"kubernetes.io/projected/20a58489-4663-478f-8a26-09468396c11e-kube-api-access-7nkjn\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:46:35.000585 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:35.000423    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20a58489-4663-478f-8a26-09468396c11e-dshm\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:46:35.004791 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:35.004773    2583 scope.go:117] "RemoveContainer" containerID="0db213345446457017eee6584e68a0df90b3430fe277877f5ac08bb81725dfba"
Sep 15 10:46:35.005079 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:46:35.005064    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0db213345446457017eee6584e68a0df90b3430fe277877f5ac08bb81725dfba\": container with ID starting with 0db213345446457017eee6584e68a0df90b3430fe277877f5ac08bb81725dfba not found: ID does not exist" containerID="0db213345446457017eee6584e68a0df90b3430fe277877f5ac08bb81725dfba"
Sep 15 10:46:35.005128 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:35.005088    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0db213345446457017eee6584e68a0df90b3430fe277877f5ac08bb81725dfba"} err="failed to get container status \"0db213345446457017eee6584e68a0df90b3430fe277877f5ac08bb81725dfba\": rpc error: code = NotFound desc = could not find container \"0db213345446457017eee6584e68a0df90b3430fe277877f5ac08bb81725dfba\": container with ID starting with 0db213345446457017eee6584e68a0df90b3430fe277877f5ac08bb81725dfba not found: ID does not exist"
Sep 15 10:46:35.019063 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:35.019031    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6"]
Sep 15 10:46:35.023106 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:35.023077    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-44134e/auth-test-model-44134e-kserve-bb574f8c5-wp7f6"]
Sep 15 10:46:35.299885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:35.299798    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0eabace0-ccad-44ee-a69f-ab95ed30a1c4" path="/var/lib/kubelet/pods/0eabace0-ccad-44ee-a69f-ab95ed30a1c4/volumes"
Sep 15 10:46:35.300243 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:35.300202    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20a58489-4663-478f-8a26-09468396c11e" path="/var/lib/kubelet/pods/20a58489-4663-478f-8a26-09468396c11e/volumes"
Sep 15 10:46:35.300572 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:35.300558    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96039a67-84c4-4287-acbf-88425dca4d3b" path="/var/lib/kubelet/pods/96039a67-84c4-4287-acbf-88425dca4d3b/volumes"
Sep 15 10:46:35.567770 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:35.567676    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:46:39.781896 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:39.781855    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:46:39.882617 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:39.882578    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:46:40.028295 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:40.028258    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:46:59.739406 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:59.739366    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:46:59.840483 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:46:59.840421    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:47:00.489526 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:47:00.489500    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:47:06.265025 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:47:06.264989    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:47:14.539305 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:47:14.539261    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:47:22.977066 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:47:22.977029    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:47:24.999914 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:47:24.999878    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:47:35.687895 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:47:35.687864    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:48:17.682780 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:17.682744    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:48:38.528415 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:38.528379    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:48:39.498464 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:39.498415    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:48:42.158294 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.158259    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs"]
Sep 15 10:48:42.158715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.158677    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3508e4ea-2813-4c8b-8007-fec91eb76f79" containerName="istio-proxy"
Sep 15 10:48:42.158715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.158689    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3508e4ea-2813-4c8b-8007-fec91eb76f79" containerName="istio-proxy"
Sep 15 10:48:42.158715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.158701    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20a58489-4663-478f-8a26-09468396c11e" containerName="main"
Sep 15 10:48:42.158715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.158707    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="20a58489-4663-478f-8a26-09468396c11e" containerName="main"
Sep 15 10:48:42.158715 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.158715    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a209f0fc-01ec-4109-a603-a82e996a8434" containerName="maas-api"
Sep 15 10:48:42.158885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.158721    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a209f0fc-01ec-4109-a603-a82e996a8434" containerName="maas-api"
Sep 15 10:48:42.158885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.158728    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48677bfc-860f-49f9-87b3-a83cce52f760" containerName="maas-api"
Sep 15 10:48:42.158885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.158733    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="48677bfc-860f-49f9-87b3-a83cce52f760" containerName="maas-api"
Sep 15 10:48:42.158885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.158739    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e854721-2229-44c8-b91e-2ab76dd1a162" containerName="istio-proxy"
Sep 15 10:48:42.158885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.158744    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e854721-2229-44c8-b91e-2ab76dd1a162" containerName="istio-proxy"
Sep 15 10:48:42.158885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.158751    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96039a67-84c4-4287-acbf-88425dca4d3b" containerName="main"
Sep 15 10:48:42.158885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.158755    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="96039a67-84c4-4287-acbf-88425dca4d3b" containerName="main"
Sep 15 10:48:42.158885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.158762    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="863cf327-3188-4b77-8ba5-90aafc821a77" containerName="maas-api"
Sep 15 10:48:42.158885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.158767    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="863cf327-3188-4b77-8ba5-90aafc821a77" containerName="maas-api"
Sep 15 10:48:42.158885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.158775    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0eabace0-ccad-44ee-a69f-ab95ed30a1c4" containerName="maas-api"
Sep 15 10:48:42.158885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.158780    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0eabace0-ccad-44ee-a69f-ab95ed30a1c4" containerName="maas-api"
Sep 15 10:48:42.158885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.158788    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6e82451-e1e6-4a82-be75-dde9c4c906b8" containerName="maas-api"
Sep 15 10:48:42.158885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.158793    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6e82451-e1e6-4a82-be75-dde9c4c906b8" containerName="maas-api"
Sep 15 10:48:42.158885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.158799    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3508e4ea-2813-4c8b-8007-fec91eb76f79" containerName="istio-proxy"
Sep 15 10:48:42.158885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.158803    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3508e4ea-2813-4c8b-8007-fec91eb76f79" containerName="istio-proxy"
Sep 15 10:48:42.158885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.158811    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39" containerName="main"
Sep 15 10:48:42.158885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.158815    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39" containerName="main"
Sep 15 10:48:42.158885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.158869    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="863cf327-3188-4b77-8ba5-90aafc821a77" containerName="maas-api"
Sep 15 10:48:42.158885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.158875    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="48677bfc-860f-49f9-87b3-a83cce52f760" containerName="maas-api"
Sep 15 10:48:42.158885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.158881    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="20a58489-4663-478f-8a26-09468396c11e" containerName="main"
Sep 15 10:48:42.158885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.158886    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3508e4ea-2813-4c8b-8007-fec91eb76f79" containerName="istio-proxy"
Sep 15 10:48:42.158885 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.158893    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3508e4ea-2813-4c8b-8007-fec91eb76f79" containerName="istio-proxy"
Sep 15 10:48:42.159554 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.158900    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="00ae72c8-ea8a-42f1-89bf-dd83c3a8cb39" containerName="main"
Sep 15 10:48:42.159554 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.158907    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e854721-2229-44c8-b91e-2ab76dd1a162" containerName="istio-proxy"
Sep 15 10:48:42.159554 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.158913    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="96039a67-84c4-4287-acbf-88425dca4d3b" containerName="main"
Sep 15 10:48:42.159554 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.158919    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="0eabace0-ccad-44ee-a69f-ab95ed30a1c4" containerName="maas-api"
Sep 15 10:48:42.159554 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.158924    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6e82451-e1e6-4a82-be75-dde9c4c906b8" containerName="maas-api"
Sep 15 10:48:42.159554 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.158930    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a209f0fc-01ec-4109-a603-a82e996a8434" containerName="maas-api"
Sep 15 10:48:42.161998 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.161977    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs"
Sep 15 10:48:42.165511 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.165481    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-83e814\"/\"openshift-service-ca.crt\""
Sep 15 10:48:42.165667 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.165481    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-83e814\"/\"kube-root-ca.crt\""
Sep 15 10:48:42.165667 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.165493    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-83e814\"/\"rate-test-model-83e814-kserve-self-signed-certs\""
Sep 15 10:48:42.165667 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.165496    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-83e814\"/\"default-dockercfg-84k6n\""
Sep 15 10:48:42.171276 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.171248    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs"]
Sep 15 10:48:42.183801 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.183774    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:48:42.242848 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.242812    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33b2566e-64c1-43c8-b130-7cce7c908a5f-tmp-dir\") pod \"rate-test-model-83e814-kserve-bddf85b59-kbhrs\" (UID: \"33b2566e-64c1-43c8-b130-7cce7c908a5f\") " pod="ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs"
Sep 15 10:48:42.243029 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.242879    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33b2566e-64c1-43c8-b130-7cce7c908a5f-home\") pod \"rate-test-model-83e814-kserve-bddf85b59-kbhrs\" (UID: \"33b2566e-64c1-43c8-b130-7cce7c908a5f\") " pod="ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs"
Sep 15 10:48:42.243029 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.242900    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33b2566e-64c1-43c8-b130-7cce7c908a5f-tls-certs\") pod \"rate-test-model-83e814-kserve-bddf85b59-kbhrs\" (UID: \"33b2566e-64c1-43c8-b130-7cce7c908a5f\") " pod="ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs"
Sep 15 10:48:42.243029 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.242921    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33b2566e-64c1-43c8-b130-7cce7c908a5f-dshm\") pod \"rate-test-model-83e814-kserve-bddf85b59-kbhrs\" (UID: \"33b2566e-64c1-43c8-b130-7cce7c908a5f\") " pod="ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs"
Sep 15 10:48:42.243029 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.242940    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33b2566e-64c1-43c8-b130-7cce7c908a5f-model-cache\") pod \"rate-test-model-83e814-kserve-bddf85b59-kbhrs\" (UID: \"33b2566e-64c1-43c8-b130-7cce7c908a5f\") " pod="ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs"
Sep 15 10:48:42.243029 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.242994    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdfxg\" (UniqueName: \"kubernetes.io/projected/33b2566e-64c1-43c8-b130-7cce7c908a5f-kube-api-access-pdfxg\") pod \"rate-test-model-83e814-kserve-bddf85b59-kbhrs\" (UID: \"33b2566e-64c1-43c8-b130-7cce7c908a5f\") " pod="ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs"
Sep 15 10:48:42.344196 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.344146    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33b2566e-64c1-43c8-b130-7cce7c908a5f-home\") pod \"rate-test-model-83e814-kserve-bddf85b59-kbhrs\" (UID: \"33b2566e-64c1-43c8-b130-7cce7c908a5f\") " pod="ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs"
Sep 15 10:48:42.344196 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.344194    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33b2566e-64c1-43c8-b130-7cce7c908a5f-tls-certs\") pod \"rate-test-model-83e814-kserve-bddf85b59-kbhrs\" (UID: \"33b2566e-64c1-43c8-b130-7cce7c908a5f\") " pod="ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs"
Sep 15 10:48:42.344508 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.344219    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33b2566e-64c1-43c8-b130-7cce7c908a5f-dshm\") pod \"rate-test-model-83e814-kserve-bddf85b59-kbhrs\" (UID: \"33b2566e-64c1-43c8-b130-7cce7c908a5f\") " pod="ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs"
Sep 15 10:48:42.344508 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.344244    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33b2566e-64c1-43c8-b130-7cce7c908a5f-model-cache\") pod \"rate-test-model-83e814-kserve-bddf85b59-kbhrs\" (UID: \"33b2566e-64c1-43c8-b130-7cce7c908a5f\") " pod="ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs"
Sep 15 10:48:42.344508 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.344271    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdfxg\" (UniqueName: \"kubernetes.io/projected/33b2566e-64c1-43c8-b130-7cce7c908a5f-kube-api-access-pdfxg\") pod \"rate-test-model-83e814-kserve-bddf85b59-kbhrs\" (UID: \"33b2566e-64c1-43c8-b130-7cce7c908a5f\") " pod="ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs"
Sep 15 10:48:42.344508 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.344323    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33b2566e-64c1-43c8-b130-7cce7c908a5f-tmp-dir\") pod \"rate-test-model-83e814-kserve-bddf85b59-kbhrs\" (UID: \"33b2566e-64c1-43c8-b130-7cce7c908a5f\") " pod="ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs"
Sep 15 10:48:42.344736 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.344713    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33b2566e-64c1-43c8-b130-7cce7c908a5f-tmp-dir\") pod \"rate-test-model-83e814-kserve-bddf85b59-kbhrs\" (UID: \"33b2566e-64c1-43c8-b130-7cce7c908a5f\") " pod="ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs"
Sep 15 10:48:42.344794 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.344760    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33b2566e-64c1-43c8-b130-7cce7c908a5f-model-cache\") pod \"rate-test-model-83e814-kserve-bddf85b59-kbhrs\" (UID: \"33b2566e-64c1-43c8-b130-7cce7c908a5f\") " pod="ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs"
Sep 15 10:48:42.344794 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.344780    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33b2566e-64c1-43c8-b130-7cce7c908a5f-home\") pod \"rate-test-model-83e814-kserve-bddf85b59-kbhrs\" (UID: \"33b2566e-64c1-43c8-b130-7cce7c908a5f\") " pod="ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs"
Sep 15 10:48:42.346685 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.346643    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33b2566e-64c1-43c8-b130-7cce7c908a5f-dshm\") pod \"rate-test-model-83e814-kserve-bddf85b59-kbhrs\" (UID: \"33b2566e-64c1-43c8-b130-7cce7c908a5f\") " pod="ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs"
Sep 15 10:48:42.346844 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.346826    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33b2566e-64c1-43c8-b130-7cce7c908a5f-tls-certs\") pod \"rate-test-model-83e814-kserve-bddf85b59-kbhrs\" (UID: \"33b2566e-64c1-43c8-b130-7cce7c908a5f\") " pod="ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs"
Sep 15 10:48:42.352780 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.352752    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdfxg\" (UniqueName: \"kubernetes.io/projected/33b2566e-64c1-43c8-b130-7cce7c908a5f-kube-api-access-pdfxg\") pod \"rate-test-model-83e814-kserve-bddf85b59-kbhrs\" (UID: \"33b2566e-64c1-43c8-b130-7cce7c908a5f\") " pod="ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs"
Sep 15 10:48:42.473492 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.473370    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs"
Sep 15 10:48:42.746530 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:42.746496    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs"]
Sep 15 10:48:42.748079 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:48:42.748053    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod33b2566e_64c1_43c8_b130_7cce7c908a5f.slice/crio-fec374c0c33385dd1aeebf70046b4a1b0d8e106fe0998ad3ed1723bd13f09860 WatchSource:0}: Error finding container fec374c0c33385dd1aeebf70046b4a1b0d8e106fe0998ad3ed1723bd13f09860: Status 404 returned error can't find the container with id fec374c0c33385dd1aeebf70046b4a1b0d8e106fe0998ad3ed1723bd13f09860
Sep 15 10:48:43.725130 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:43.725092    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs" event={"ID":"33b2566e-64c1-43c8-b130-7cce7c908a5f","Type":"ContainerStarted","Data":"83c40450ce7f2ea853f9c34fd381d9aa6bf09f5d25993377a3f2483ebedc5630"}
Sep 15 10:48:43.725616 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:43.725139    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs" event={"ID":"33b2566e-64c1-43c8-b130-7cce7c908a5f","Type":"ContainerStarted","Data":"fec374c0c33385dd1aeebf70046b4a1b0d8e106fe0998ad3ed1723bd13f09860"}
Sep 15 10:48:43.741134 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:43.741072    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs" podStartSLOduration=1.7410534229999999 podStartE2EDuration="1.741053423s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:48:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:48:43.739816844 +0000 UTC m=+1838.986783811" watchObservedRunningTime="2026-09-15 10:48:43.741053423 +0000 UTC m=+1838.988020391"
Sep 15 10:48:52.474152 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:52.474117    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs"
Sep 15 10:48:52.476575 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:52.474391    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs"
Sep 15 10:48:52.487258 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:52.487234    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs"
Sep 15 10:48:52.787279 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:48:52.787195    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs"
Sep 15 10:49:16.282393 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:49:16.282354    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:49:52.265770 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:49:52.265731    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:49:58.258703 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:49:58.258667    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs"]
Sep 15 10:49:58.276888 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:49:58.276848    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:49:59.385062 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:49:59.385029    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:49:59.848937 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:49:59.848856    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:00.145524 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:00.145497    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:00.733414 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:00.733384    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:03.233746 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:03.233712    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:03.403261 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:03.403216    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:03.500634 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:03.500539    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:03.987680 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:03.987646    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:04.763205 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:04.763162    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq"]
Sep 15 10:50:04.763827 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:04.763805    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e854721-2229-44c8-b91e-2ab76dd1a162" containerName="istio-proxy"
Sep 15 10:50:04.763899 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:04.763832    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e854721-2229-44c8-b91e-2ab76dd1a162" containerName="istio-proxy"
Sep 15 10:50:04.763981 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:04.763970    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e854721-2229-44c8-b91e-2ab76dd1a162" containerName="istio-proxy"
Sep 15 10:50:04.767538 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:04.767517    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq"
Sep 15 10:50:04.769876 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:04.769854    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-5ffc6a-serving-cert\""
Sep 15 10:50:04.776497 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:04.776473    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq"]
Sep 15 10:50:04.789754 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:04.789725    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:04.847113 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:04.847082    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:04.963314 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:04.963271    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/322e48a5-9952-406e-ac7a-c4418e8e5a33-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq\" (UID: \"322e48a5-9952-406e-ac7a-c4418e8e5a33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq"
Sep 15 10:50:04.963532 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:04.963428    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/322e48a5-9952-406e-ac7a-c4418e8e5a33-metrics-certs\") pod \"maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq\" (UID: \"322e48a5-9952-406e-ac7a-c4418e8e5a33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq"
Sep 15 10:50:04.963532 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:04.963512    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/322e48a5-9952-406e-ac7a-c4418e8e5a33-maas-api-tls\") pod \"maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq\" (UID: \"322e48a5-9952-406e-ac7a-c4418e8e5a33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq"
Sep 15 10:50:04.963634 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:04.963554    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hltjb\" (UniqueName: \"kubernetes.io/projected/322e48a5-9952-406e-ac7a-c4418e8e5a33-kube-api-access-hltjb\") pod \"maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq\" (UID: \"322e48a5-9952-406e-ac7a-c4418e8e5a33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq"
Sep 15 10:50:05.064359 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:05.064267    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/322e48a5-9952-406e-ac7a-c4418e8e5a33-metrics-certs\") pod \"maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq\" (UID: \"322e48a5-9952-406e-ac7a-c4418e8e5a33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq"
Sep 15 10:50:05.064359 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:05.064327    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/322e48a5-9952-406e-ac7a-c4418e8e5a33-maas-api-tls\") pod \"maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq\" (UID: \"322e48a5-9952-406e-ac7a-c4418e8e5a33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq"
Sep 15 10:50:05.064647 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:05.064369    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hltjb\" (UniqueName: \"kubernetes.io/projected/322e48a5-9952-406e-ac7a-c4418e8e5a33-kube-api-access-hltjb\") pod \"maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq\" (UID: \"322e48a5-9952-406e-ac7a-c4418e8e5a33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq"
Sep 15 10:50:05.064647 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:05.064409    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/322e48a5-9952-406e-ac7a-c4418e8e5a33-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq\" (UID: \"322e48a5-9952-406e-ac7a-c4418e8e5a33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq"
Sep 15 10:50:05.064647 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:50:05.064514    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-5ffc6a-serving-cert: secret "maas-api-e2e-ipp-a-5ffc6a-serving-cert" not found
Sep 15 10:50:05.064647 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:50:05.064579    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/322e48a5-9952-406e-ac7a-c4418e8e5a33-maas-api-tls podName:322e48a5-9952-406e-ac7a-c4418e8e5a33 nodeName:}" failed. No retries permitted until 2026-09-15 10:50:05.564563087 +0000 UTC m=+1920.811530034 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/322e48a5-9952-406e-ac7a-c4418e8e5a33-maas-api-tls") pod "maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq" (UID: "322e48a5-9952-406e-ac7a-c4418e8e5a33") : secret "maas-api-e2e-ipp-a-5ffc6a-serving-cert" not found
Sep 15 10:50:05.065134 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:05.065115    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/322e48a5-9952-406e-ac7a-c4418e8e5a33-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq\" (UID: \"322e48a5-9952-406e-ac7a-c4418e8e5a33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq"
Sep 15 10:50:05.066964 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:05.066937    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/322e48a5-9952-406e-ac7a-c4418e8e5a33-metrics-certs\") pod \"maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq\" (UID: \"322e48a5-9952-406e-ac7a-c4418e8e5a33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq"
Sep 15 10:50:05.072571 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:05.072551    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hltjb\" (UniqueName: \"kubernetes.io/projected/322e48a5-9952-406e-ac7a-c4418e8e5a33-kube-api-access-hltjb\") pod \"maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq\" (UID: \"322e48a5-9952-406e-ac7a-c4418e8e5a33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq"
Sep 15 10:50:05.180113 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:05.180077    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:05.302999 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:05.302964    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:05.401911 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:05.401878    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:05.570221 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:05.570187    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/322e48a5-9952-406e-ac7a-c4418e8e5a33-maas-api-tls\") pod \"maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq\" (UID: \"322e48a5-9952-406e-ac7a-c4418e8e5a33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq"
Sep 15 10:50:05.572658 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:05.572635    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/322e48a5-9952-406e-ac7a-c4418e8e5a33-maas-api-tls\") pod \"maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq\" (UID: \"322e48a5-9952-406e-ac7a-c4418e8e5a33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq"
Sep 15 10:50:05.680844 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:05.680749    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq"
Sep 15 10:50:05.953512 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:05.953485    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq"]
Sep 15 10:50:05.954385 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:50:05.954364    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod322e48a5_9952_406e_ac7a_c4418e8e5a33.slice/crio-b6a0e019489fefa4ddb93256b5372d19dc7b2a237b50bde28b0eabe9273c6999 WatchSource:0}: Error finding container b6a0e019489fefa4ddb93256b5372d19dc7b2a237b50bde28b0eabe9273c6999: Status 404 returned error can't find the container with id b6a0e019489fefa4ddb93256b5372d19dc7b2a237b50bde28b0eabe9273c6999
Sep 15 10:50:05.955723 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:05.955706    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 10:50:06.208945 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:06.208855    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq" event={"ID":"322e48a5-9952-406e-ac7a-c4418e8e5a33","Type":"ContainerStarted","Data":"b6a0e019489fefa4ddb93256b5372d19dc7b2a237b50bde28b0eabe9273c6999"}
Sep 15 10:50:08.222008 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:08.221969    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq" event={"ID":"322e48a5-9952-406e-ac7a-c4418e8e5a33","Type":"ContainerStarted","Data":"2bfe3f9144c1e7a624f12a4c91ff2ece1e075094bbf95607d25b9851b1e025d9"}
Sep 15 10:50:08.222512 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:08.222052    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq"
Sep 15 10:50:08.239746 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:08.239694    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq" podStartSLOduration=2.83294433 podStartE2EDuration="4.239675464s" totalImagesPullingTime="1.406731134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:50:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:50:08.237781146 +0000 UTC m=+1923.484748116" watchObservedRunningTime="2026-09-15 10:50:08.239675464 +0000 UTC m=+1923.486642436"
Sep 15 10:50:13.278617 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:13.278572    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs" podUID="33b2566e-64c1-43c8-b130-7cce7c908a5f" containerName="main" containerID="cri-o://83c40450ce7f2ea853f9c34fd381d9aa6bf09f5d25993377a3f2483ebedc5630" gracePeriod=45
Sep 15 10:50:13.525615 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:13.525592    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs"
Sep 15 10:50:13.538510 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:13.538438    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33b2566e-64c1-43c8-b130-7cce7c908a5f-tmp-dir\") pod \"33b2566e-64c1-43c8-b130-7cce7c908a5f\" (UID: \"33b2566e-64c1-43c8-b130-7cce7c908a5f\") "
Sep 15 10:50:13.538510 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:13.538484    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33b2566e-64c1-43c8-b130-7cce7c908a5f-model-cache\") pod \"33b2566e-64c1-43c8-b130-7cce7c908a5f\" (UID: \"33b2566e-64c1-43c8-b130-7cce7c908a5f\") "
Sep 15 10:50:13.538685 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:13.538513    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33b2566e-64c1-43c8-b130-7cce7c908a5f-dshm\") pod \"33b2566e-64c1-43c8-b130-7cce7c908a5f\" (UID: \"33b2566e-64c1-43c8-b130-7cce7c908a5f\") "
Sep 15 10:50:13.538685 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:13.538536    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33b2566e-64c1-43c8-b130-7cce7c908a5f-home\") pod \"33b2566e-64c1-43c8-b130-7cce7c908a5f\" (UID: \"33b2566e-64c1-43c8-b130-7cce7c908a5f\") "
Sep 15 10:50:13.538685 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:13.538600    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33b2566e-64c1-43c8-b130-7cce7c908a5f-tls-certs\") pod \"33b2566e-64c1-43c8-b130-7cce7c908a5f\" (UID: \"33b2566e-64c1-43c8-b130-7cce7c908a5f\") "
Sep 15 10:50:13.538685 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:13.538650    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pdfxg\" (UniqueName: \"kubernetes.io/projected/33b2566e-64c1-43c8-b130-7cce7c908a5f-kube-api-access-pdfxg\") pod \"33b2566e-64c1-43c8-b130-7cce7c908a5f\" (UID: \"33b2566e-64c1-43c8-b130-7cce7c908a5f\") "
Sep 15 10:50:13.538882 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:13.538732    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33b2566e-64c1-43c8-b130-7cce7c908a5f-model-cache" (OuterVolumeSpecName: "model-cache") pod "33b2566e-64c1-43c8-b130-7cce7c908a5f" (UID: "33b2566e-64c1-43c8-b130-7cce7c908a5f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:50:13.538882 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:13.538753    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33b2566e-64c1-43c8-b130-7cce7c908a5f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "33b2566e-64c1-43c8-b130-7cce7c908a5f" (UID: "33b2566e-64c1-43c8-b130-7cce7c908a5f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:50:13.538882 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:13.538851    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33b2566e-64c1-43c8-b130-7cce7c908a5f-home" (OuterVolumeSpecName: "home") pod "33b2566e-64c1-43c8-b130-7cce7c908a5f" (UID: "33b2566e-64c1-43c8-b130-7cce7c908a5f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:50:13.539029 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:13.539011    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33b2566e-64c1-43c8-b130-7cce7c908a5f-home\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:50:13.539084 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:13.539050    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33b2566e-64c1-43c8-b130-7cce7c908a5f-tmp-dir\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:50:13.539084 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:13.539061    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33b2566e-64c1-43c8-b130-7cce7c908a5f-model-cache\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:50:13.540745 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:13.540717    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33b2566e-64c1-43c8-b130-7cce7c908a5f-kube-api-access-pdfxg" (OuterVolumeSpecName: "kube-api-access-pdfxg") pod "33b2566e-64c1-43c8-b130-7cce7c908a5f" (UID: "33b2566e-64c1-43c8-b130-7cce7c908a5f"). InnerVolumeSpecName "kube-api-access-pdfxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:50:13.540857 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:13.540749    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33b2566e-64c1-43c8-b130-7cce7c908a5f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "33b2566e-64c1-43c8-b130-7cce7c908a5f" (UID: "33b2566e-64c1-43c8-b130-7cce7c908a5f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:50:13.540857 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:13.540792    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33b2566e-64c1-43c8-b130-7cce7c908a5f-dshm" (OuterVolumeSpecName: "dshm") pod "33b2566e-64c1-43c8-b130-7cce7c908a5f" (UID: "33b2566e-64c1-43c8-b130-7cce7c908a5f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:50:13.639858 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:13.639816    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33b2566e-64c1-43c8-b130-7cce7c908a5f-tls-certs\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:50:13.639858 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:13.639848    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pdfxg\" (UniqueName: \"kubernetes.io/projected/33b2566e-64c1-43c8-b130-7cce7c908a5f-kube-api-access-pdfxg\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:50:13.639858 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:13.639858    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33b2566e-64c1-43c8-b130-7cce7c908a5f-dshm\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:50:14.233501 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:14.233471    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq"
Sep 15 10:50:14.255401 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:14.255368    2583 generic.go:356] "Generic (PLEG): container finished" podID="33b2566e-64c1-43c8-b130-7cce7c908a5f" containerID="83c40450ce7f2ea853f9c34fd381d9aa6bf09f5d25993377a3f2483ebedc5630" exitCode=0
Sep 15 10:50:14.255617 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:14.255414    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs"
Sep 15 10:50:14.255617 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:14.255473    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs" event={"ID":"33b2566e-64c1-43c8-b130-7cce7c908a5f","Type":"ContainerDied","Data":"83c40450ce7f2ea853f9c34fd381d9aa6bf09f5d25993377a3f2483ebedc5630"}
Sep 15 10:50:14.255617 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:14.255517    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs" event={"ID":"33b2566e-64c1-43c8-b130-7cce7c908a5f","Type":"ContainerDied","Data":"fec374c0c33385dd1aeebf70046b4a1b0d8e106fe0998ad3ed1723bd13f09860"}
Sep 15 10:50:14.255617 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:14.255535    2583 scope.go:117] "RemoveContainer" containerID="83c40450ce7f2ea853f9c34fd381d9aa6bf09f5d25993377a3f2483ebedc5630"
Sep 15 10:50:14.267068 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:14.267042    2583 scope.go:117] "RemoveContainer" containerID="83c40450ce7f2ea853f9c34fd381d9aa6bf09f5d25993377a3f2483ebedc5630"
Sep 15 10:50:14.267385 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:50:14.267360    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83c40450ce7f2ea853f9c34fd381d9aa6bf09f5d25993377a3f2483ebedc5630\": container with ID starting with 83c40450ce7f2ea853f9c34fd381d9aa6bf09f5d25993377a3f2483ebedc5630 not found: ID does not exist" containerID="83c40450ce7f2ea853f9c34fd381d9aa6bf09f5d25993377a3f2483ebedc5630"
Sep 15 10:50:14.267564 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:14.267399    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83c40450ce7f2ea853f9c34fd381d9aa6bf09f5d25993377a3f2483ebedc5630"} err="failed to get container status \"83c40450ce7f2ea853f9c34fd381d9aa6bf09f5d25993377a3f2483ebedc5630\": rpc error: code = NotFound desc = could not find container \"83c40450ce7f2ea853f9c34fd381d9aa6bf09f5d25993377a3f2483ebedc5630\": container with ID starting with 83c40450ce7f2ea853f9c34fd381d9aa6bf09f5d25993377a3f2483ebedc5630 not found: ID does not exist"
Sep 15 10:50:14.282252 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:14.282225    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs"]
Sep 15 10:50:14.284831 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:14.284808    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-83e814/rate-test-model-83e814-kserve-bddf85b59-kbhrs"]
Sep 15 10:50:15.301434 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:15.301392    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33b2566e-64c1-43c8-b130-7cce7c908a5f" path="/var/lib/kubelet/pods/33b2566e-64c1-43c8-b130-7cce7c908a5f/volumes"
Sep 15 10:50:27.346974 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:27.346939    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:28.072994 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:28.072958    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:28.492368 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:28.492324    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:28.676617 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:28.676578    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:29.301389 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:29.301337    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:29.402438 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:29.402390    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:29.573129 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:29.573018    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:30.164942 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:30.164907    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:30.263533 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:30.263500    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:31.867870 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:31.867832    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:32.471261 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:32.471228    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:32.575084 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:32.575048    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:32.651003 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:32.650966    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:33.297398 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:33.297367    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:33.902870 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:33.902835    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:34.570332 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:34.570285    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:34.669872 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:34.669840    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:34.769722 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:34.769687    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:35.096675 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:35.096641    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:35.191147 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:35.191102    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:56.371693 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:56.371661    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:57.104299 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:57.104252    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:57.595043 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:57.594955    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:57.712695 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:57.712590    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:58.462995 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:58.462963    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:50:59.164408 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:50:59.164349    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:51:14.383862 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:14.383823    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:51:44.104253 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.104217    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:51:44.363841 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.363750    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"]
Sep 15 10:51:44.364312 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.364289    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33b2566e-64c1-43c8-b130-7cce7c908a5f" containerName="main"
Sep 15 10:51:44.364312 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.364312    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="33b2566e-64c1-43c8-b130-7cce7c908a5f" containerName="main"
Sep 15 10:51:44.364571 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.364429    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="33b2566e-64c1-43c8-b130-7cce7c908a5f" containerName="main"
Sep 15 10:51:44.373481 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.373429    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:51:44.376242 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.376208    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-1b5e28-openshift-default-dockercfg-wtzjg\""
Sep 15 10:51:44.380076 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.380047    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"]
Sep 15 10:51:44.385653 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.385624    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:51:44.421259 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.421218    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-istio-token\") pod \"e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") " pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:51:44.421259 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.421264    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-istiod-ca-cert\") pod \"e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") " pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:51:44.421566 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.421282    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-workload-socket\") pod \"e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") " pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:51:44.421566 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.421414    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-credential-socket\") pod \"e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") " pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:51:44.421566 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.421472    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-istio-envoy\") pod \"e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") " pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:51:44.421566 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.421495    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-istio-data\") pod \"e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") " pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:51:44.421566 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.421533    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-istio-podinfo\") pod \"e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") " pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:51:44.421566 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.421551    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-workload-certs\") pod \"e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") " pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:51:44.421842 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.421682    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpjtd\" (UniqueName: \"kubernetes.io/projected/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-kube-api-access-lpjtd\") pod \"e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") " pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:51:44.522401 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.522359    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-credential-socket\") pod \"e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") " pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:51:44.522401 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.522400    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-istio-envoy\") pod \"e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") " pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:51:44.522702 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.522418    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-istio-data\") pod \"e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") " pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:51:44.522702 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.522479    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-istio-podinfo\") pod \"e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") " pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:51:44.522702 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.522498    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-workload-certs\") pod \"e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") " pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:51:44.522702 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.522561    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpjtd\" (UniqueName: \"kubernetes.io/projected/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-kube-api-access-lpjtd\") pod \"e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") " pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:51:44.522702 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.522613    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-istio-token\") pod \"e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") " pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:51:44.522702 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.522645    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-istiod-ca-cert\") pod \"e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") " pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:51:44.522702 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.522668    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-workload-socket\") pod \"e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") " pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:51:44.523181 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.523150    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-credential-socket\") pod \"e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") " pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:51:44.523376 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.523351    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-istio-data\") pod \"e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") " pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:51:44.523476 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.523374    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-istiod-ca-cert\") pod \"e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") " pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:51:44.523476 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.523462    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-workload-certs\") pod \"e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") " pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:51:44.523616 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.523530    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-workload-socket\") pod \"e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") " pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:51:44.525096 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.525074    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-istio-envoy\") pod \"e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") " pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:51:44.525306 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.525292    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-istio-podinfo\") pod \"e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") " pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:51:44.532949 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.532915    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-istio-token\") pod \"e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") " pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:51:44.533368 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.533332    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpjtd\" (UniqueName: \"kubernetes.io/projected/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-kube-api-access-lpjtd\") pod \"e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") " pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:51:44.584146 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.584108    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:51:44.687426 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.687386    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:51:44.963912 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.963883    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"]
Sep 15 10:51:44.965697 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:51:44.965663    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffb8e359_aba1_4c2c_aca5_7634f3d23f0a.slice/crio-4c070b6ab242f19a0c1e94e7af422ed7c3c0e9e56d4c5940fff009822d73c07f WatchSource:0}: Error finding container 4c070b6ab242f19a0c1e94e7af422ed7c3c0e9e56d4c5940fff009822d73c07f: Status 404 returned error can't find the container with id 4c070b6ab242f19a0c1e94e7af422ed7c3c0e9e56d4c5940fff009822d73c07f
Sep 15 10:51:44.967916 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.967882    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:51:44.967997 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.967949    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:51:44.967997 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:44.967976    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:51:45.781745 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:45.781709    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz" event={"ID":"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a","Type":"ContainerStarted","Data":"a18dca9a14a62a225a3f597500f92551fc16596249467c41be7ac809a08bb80d"}
Sep 15 10:51:45.782245 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:45.781770    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz" event={"ID":"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a","Type":"ContainerStarted","Data":"4c070b6ab242f19a0c1e94e7af422ed7c3c0e9e56d4c5940fff009822d73c07f"}
Sep 15 10:51:45.803012 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:45.802957    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz" podStartSLOduration=1.802939649 podStartE2EDuration="1.802939649s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:51:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:51:45.80167217 +0000 UTC m=+2021.048639153" watchObservedRunningTime="2026-09-15 10:51:45.802939649 +0000 UTC m=+2021.049906618"
Sep 15 10:51:46.688287 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:46.688243    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:51:46.693610 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:46.693584    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:51:46.787347 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:46.787318    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:51:46.788267 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:46.788248    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:51:47.001307 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:47.001227    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:51:47.687506 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:47.687412    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:51:47.780314 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:47.780280    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:51:47.886795 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:47.886767    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:51:48.547182 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:48.547150    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:51:49.270392 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:49.270361    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:51:49.975701 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:49.975661    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:51:50.665098 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:50.665061    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc"]
Sep 15 10:51:50.668952 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:50.668931    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc"
Sep 15 10:51:50.671600 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:50.671570    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-1b5e28-serving-cert\""
Sep 15 10:51:50.678596 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:50.678574    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc"]
Sep 15 10:51:50.690408 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:50.690371    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:51:50.781637 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:50.781605    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68e09eab-1299-4f75-a5ba-2ed6f41bffc6-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc\" (UID: \"68e09eab-1299-4f75-a5ba-2ed6f41bffc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc"
Sep 15 10:51:50.781816 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:50.781731    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68e09eab-1299-4f75-a5ba-2ed6f41bffc6-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc\" (UID: \"68e09eab-1299-4f75-a5ba-2ed6f41bffc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc"
Sep 15 10:51:50.781816 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:50.781797    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68e09eab-1299-4f75-a5ba-2ed6f41bffc6-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc\" (UID: \"68e09eab-1299-4f75-a5ba-2ed6f41bffc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc"
Sep 15 10:51:50.781914 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:50.781863    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnnw9\" (UniqueName: \"kubernetes.io/projected/68e09eab-1299-4f75-a5ba-2ed6f41bffc6-kube-api-access-jnnw9\") pod \"maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc\" (UID: \"68e09eab-1299-4f75-a5ba-2ed6f41bffc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc"
Sep 15 10:51:50.792838 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:50.792811    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:51:50.882481 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:50.882433    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnnw9\" (UniqueName: \"kubernetes.io/projected/68e09eab-1299-4f75-a5ba-2ed6f41bffc6-kube-api-access-jnnw9\") pod \"maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc\" (UID: \"68e09eab-1299-4f75-a5ba-2ed6f41bffc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc"
Sep 15 10:51:50.882619 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:50.882517    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68e09eab-1299-4f75-a5ba-2ed6f41bffc6-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc\" (UID: \"68e09eab-1299-4f75-a5ba-2ed6f41bffc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc"
Sep 15 10:51:50.882619 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:50.882596    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68e09eab-1299-4f75-a5ba-2ed6f41bffc6-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc\" (UID: \"68e09eab-1299-4f75-a5ba-2ed6f41bffc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc"
Sep 15 10:51:50.882693 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:50.882636    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68e09eab-1299-4f75-a5ba-2ed6f41bffc6-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc\" (UID: \"68e09eab-1299-4f75-a5ba-2ed6f41bffc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc"
Sep 15 10:51:50.882693 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:51:50.882649    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b5e28-serving-cert: secret "maas-api-e2e-ipp-cleanup-1b5e28-serving-cert" not found
Sep 15 10:51:50.882773 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:51:50.882722    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68e09eab-1299-4f75-a5ba-2ed6f41bffc6-maas-api-tls podName:68e09eab-1299-4f75-a5ba-2ed6f41bffc6 nodeName:}" failed. No retries permitted until 2026-09-15 10:51:51.38270612 +0000 UTC m=+2026.629673071 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/68e09eab-1299-4f75-a5ba-2ed6f41bffc6-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc" (UID: "68e09eab-1299-4f75-a5ba-2ed6f41bffc6") : secret "maas-api-e2e-ipp-cleanup-1b5e28-serving-cert" not found
Sep 15 10:51:50.883268 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:50.883240    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68e09eab-1299-4f75-a5ba-2ed6f41bffc6-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc\" (UID: \"68e09eab-1299-4f75-a5ba-2ed6f41bffc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc"
Sep 15 10:51:50.884966 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:50.884947    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68e09eab-1299-4f75-a5ba-2ed6f41bffc6-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc\" (UID: \"68e09eab-1299-4f75-a5ba-2ed6f41bffc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc"
Sep 15 10:51:50.893247 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:50.893228    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnnw9\" (UniqueName: \"kubernetes.io/projected/68e09eab-1299-4f75-a5ba-2ed6f41bffc6-kube-api-access-jnnw9\") pod \"maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc\" (UID: \"68e09eab-1299-4f75-a5ba-2ed6f41bffc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc"
Sep 15 10:51:51.123943 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:51.123866    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:51:51.222726 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:51.222692    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:51:51.386971 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:51.386936    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68e09eab-1299-4f75-a5ba-2ed6f41bffc6-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc\" (UID: \"68e09eab-1299-4f75-a5ba-2ed6f41bffc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc"
Sep 15 10:51:51.389506 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:51.389482    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68e09eab-1299-4f75-a5ba-2ed6f41bffc6-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc\" (UID: \"68e09eab-1299-4f75-a5ba-2ed6f41bffc6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc"
Sep 15 10:51:51.434162 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:51.434132    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:51:51.581557 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:51.581525    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc"
Sep 15 10:51:51.852327 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:51.852300    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc"]
Sep 15 10:51:51.853571 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:51:51.853540    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68e09eab_1299_4f75_a5ba_2ed6f41bffc6.slice/crio-ef780ee270f35389440206dc367ef6b947cbda3993f5672e354769ba6b66f545 WatchSource:0}: Error finding container ef780ee270f35389440206dc367ef6b947cbda3993f5672e354769ba6b66f545: Status 404 returned error can't find the container with id ef780ee270f35389440206dc367ef6b947cbda3993f5672e354769ba6b66f545
Sep 15 10:51:52.825266 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:52.825231    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc" event={"ID":"68e09eab-1299-4f75-a5ba-2ed6f41bffc6","Type":"ContainerStarted","Data":"d1d561e7f66c0b7f1564b4b35c740d685adfc908292c17e1ed6411dd1bdbe2f5"}
Sep 15 10:51:52.825469 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:52.825286    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc" event={"ID":"68e09eab-1299-4f75-a5ba-2ed6f41bffc6","Type":"ContainerStarted","Data":"ef780ee270f35389440206dc367ef6b947cbda3993f5672e354769ba6b66f545"}
Sep 15 10:51:52.825469 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:52.825339    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc"
Sep 15 10:51:52.845223 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:52.845168    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc" podStartSLOduration=2.388771109 podStartE2EDuration="2.84515306s" totalImagesPullingTime="456.381951ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:51:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:51:52.842882335 +0000 UTC m=+2028.089849317" watchObservedRunningTime="2026-09-15 10:51:52.84515306 +0000 UTC m=+2028.092120029"
Sep 15 10:51:58.836022 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:51:58.835988    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc"
Sep 15 10:52:06.917108 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:06.917078    2583 generic.go:356] "Generic (PLEG): container finished" podID="ffb8e359-aba1-4c2c-aca5-7634f3d23f0a" containerID="a18dca9a14a62a225a3f597500f92551fc16596249467c41be7ac809a08bb80d" exitCode=0
Sep 15 10:52:06.917552 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:06.917149    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz" event={"ID":"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a","Type":"ContainerDied","Data":"a18dca9a14a62a225a3f597500f92551fc16596249467c41be7ac809a08bb80d"}
Sep 15 10:52:06.917605 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:06.917574    2583 scope.go:117] "RemoveContainer" containerID="a18dca9a14a62a225a3f597500f92551fc16596249467c41be7ac809a08bb80d"
Sep 15 10:52:06.918703 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:06.918679    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:52:06.918771 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:06.918741    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:52:06.918854 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:06.918771    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 15 10:52:07.925462 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:07.925410    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz" event={"ID":"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a","Type":"ContainerStarted","Data":"90fd9112e52bf27a009e916d25b0160530441b05126fae55e3303bfa15a35b8e"}
Sep 15 10:52:08.688552 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:08.688518    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:52:08.688775 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:08.688755    2583 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Sep 15 10:52:08.688817 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:08.688802    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz" podUID="ffb8e359-aba1-4c2c-aca5-7634f3d23f0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Sep 15 10:52:09.688804 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:09.688767    2583 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Sep 15 10:52:09.689199 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:09.688827    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz" podUID="ffb8e359-aba1-4c2c-aca5-7634f3d23f0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Sep 15 10:52:10.687979 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:10.687939    2583 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Sep 15 10:52:10.688149 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:10.688004    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz" podUID="ffb8e359-aba1-4c2c-aca5-7634f3d23f0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Sep 15 10:52:11.688091 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:11.688050    2583 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Sep 15 10:52:11.688501 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:11.688120    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz" podUID="ffb8e359-aba1-4c2c-aca5-7634f3d23f0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Sep 15 10:52:12.461295 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:12.461217    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:12.661557 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:12.661519    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:12.688302 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:12.688265    2583 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Sep 15 10:52:12.688714 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:12.688326    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz" podUID="ffb8e359-aba1-4c2c-aca5-7634f3d23f0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Sep 15 10:52:13.151082 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:13.151054    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:13.691889 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:13.691860    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:52:13.692284 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:13.692263    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:52:13.692934 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:13.692915    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:52:13.946414 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:13.946337    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:14.578203 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:14.578164    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:15.291211 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:15.291178    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:15.876686 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:15.876652    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:16.559067 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:16.559030    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:17.171341 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:17.171306    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:18.999659 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:18.999624    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:19.099894 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:19.099862    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:19.200509 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:19.200476    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:19.584943 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:19.584913    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:19.635903 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:19.635869    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc"]
Sep 15 10:52:19.636137 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:19.636110    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc" podUID="68e09eab-1299-4f75-a5ba-2ed6f41bffc6" containerName="maas-api" containerID="cri-o://d1d561e7f66c0b7f1564b4b35c740d685adfc908292c17e1ed6411dd1bdbe2f5" gracePeriod=30
Sep 15 10:52:19.685958 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:19.685929    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:19.786769 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:19.786734    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:19.854987 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:19.854895    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:19.886209 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:19.886181    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc"
Sep 15 10:52:19.969464 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:19.969414    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68e09eab-1299-4f75-a5ba-2ed6f41bffc6-metrics-certs\") pod \"68e09eab-1299-4f75-a5ba-2ed6f41bffc6\" (UID: \"68e09eab-1299-4f75-a5ba-2ed6f41bffc6\") "
Sep 15 10:52:19.969648 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:19.969520    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68e09eab-1299-4f75-a5ba-2ed6f41bffc6-maas-api-tls\") pod \"68e09eab-1299-4f75-a5ba-2ed6f41bffc6\" (UID: \"68e09eab-1299-4f75-a5ba-2ed6f41bffc6\") "
Sep 15 10:52:19.969648 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:19.969555    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnnw9\" (UniqueName: \"kubernetes.io/projected/68e09eab-1299-4f75-a5ba-2ed6f41bffc6-kube-api-access-jnnw9\") pod \"68e09eab-1299-4f75-a5ba-2ed6f41bffc6\" (UID: \"68e09eab-1299-4f75-a5ba-2ed6f41bffc6\") "
Sep 15 10:52:19.969648 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:19.969594    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68e09eab-1299-4f75-a5ba-2ed6f41bffc6-openshift-service-ca\") pod \"68e09eab-1299-4f75-a5ba-2ed6f41bffc6\" (UID: \"68e09eab-1299-4f75-a5ba-2ed6f41bffc6\") "
Sep 15 10:52:19.970049 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:19.970016    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68e09eab-1299-4f75-a5ba-2ed6f41bffc6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "68e09eab-1299-4f75-a5ba-2ed6f41bffc6" (UID: "68e09eab-1299-4f75-a5ba-2ed6f41bffc6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:52:19.971729 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:19.971696    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68e09eab-1299-4f75-a5ba-2ed6f41bffc6-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "68e09eab-1299-4f75-a5ba-2ed6f41bffc6" (UID: "68e09eab-1299-4f75-a5ba-2ed6f41bffc6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:52:19.971729 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:19.971712    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68e09eab-1299-4f75-a5ba-2ed6f41bffc6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "68e09eab-1299-4f75-a5ba-2ed6f41bffc6" (UID: "68e09eab-1299-4f75-a5ba-2ed6f41bffc6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:52:19.971867 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:19.971813    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68e09eab-1299-4f75-a5ba-2ed6f41bffc6-kube-api-access-jnnw9" (OuterVolumeSpecName: "kube-api-access-jnnw9") pod "68e09eab-1299-4f75-a5ba-2ed6f41bffc6" (UID: "68e09eab-1299-4f75-a5ba-2ed6f41bffc6"). InnerVolumeSpecName "kube-api-access-jnnw9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:52:19.997638 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:19.997609    2583 generic.go:356] "Generic (PLEG): container finished" podID="68e09eab-1299-4f75-a5ba-2ed6f41bffc6" containerID="d1d561e7f66c0b7f1564b4b35c740d685adfc908292c17e1ed6411dd1bdbe2f5" exitCode=0
Sep 15 10:52:19.997778 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:19.997672    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc"
Sep 15 10:52:19.997778 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:19.997681    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc" event={"ID":"68e09eab-1299-4f75-a5ba-2ed6f41bffc6","Type":"ContainerDied","Data":"d1d561e7f66c0b7f1564b4b35c740d685adfc908292c17e1ed6411dd1bdbe2f5"}
Sep 15 10:52:19.997858 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:19.997796    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc" event={"ID":"68e09eab-1299-4f75-a5ba-2ed6f41bffc6","Type":"ContainerDied","Data":"ef780ee270f35389440206dc367ef6b947cbda3993f5672e354769ba6b66f545"}
Sep 15 10:52:19.997858 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:19.997818    2583 scope.go:117] "RemoveContainer" containerID="d1d561e7f66c0b7f1564b4b35c740d685adfc908292c17e1ed6411dd1bdbe2f5"
Sep 15 10:52:20.011661 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:20.011637    2583 scope.go:117] "RemoveContainer" containerID="d1d561e7f66c0b7f1564b4b35c740d685adfc908292c17e1ed6411dd1bdbe2f5"
Sep 15 10:52:20.012625 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:52:20.012577    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1d561e7f66c0b7f1564b4b35c740d685adfc908292c17e1ed6411dd1bdbe2f5\": container with ID starting with d1d561e7f66c0b7f1564b4b35c740d685adfc908292c17e1ed6411dd1bdbe2f5 not found: ID does not exist" containerID="d1d561e7f66c0b7f1564b4b35c740d685adfc908292c17e1ed6411dd1bdbe2f5"
Sep 15 10:52:20.012752 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:20.012639    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1d561e7f66c0b7f1564b4b35c740d685adfc908292c17e1ed6411dd1bdbe2f5"} err="failed to get container status \"d1d561e7f66c0b7f1564b4b35c740d685adfc908292c17e1ed6411dd1bdbe2f5\": rpc error: code = NotFound desc = could not find container \"d1d561e7f66c0b7f1564b4b35c740d685adfc908292c17e1ed6411dd1bdbe2f5\": container with ID starting with d1d561e7f66c0b7f1564b4b35c740d685adfc908292c17e1ed6411dd1bdbe2f5 not found: ID does not exist"
Sep 15 10:52:20.022269 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:20.022235    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc"]
Sep 15 10:52:20.024927 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:20.024902    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b5e28-7c66848d58-fltsc"]
Sep 15 10:52:20.071104 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:20.071059    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68e09eab-1299-4f75-a5ba-2ed6f41bffc6-metrics-certs\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:52:20.071104 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:20.071093    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68e09eab-1299-4f75-a5ba-2ed6f41bffc6-maas-api-tls\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:52:20.071104 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:20.071103    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnnw9\" (UniqueName: \"kubernetes.io/projected/68e09eab-1299-4f75-a5ba-2ed6f41bffc6-kube-api-access-jnnw9\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:52:20.071104 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:20.071112    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68e09eab-1299-4f75-a5ba-2ed6f41bffc6-openshift-service-ca\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:52:20.192308 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:20.192273    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:20.279164 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:20.279130    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:21.077407 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:21.077375    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:21.300621 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:21.300585    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68e09eab-1299-4f75-a5ba-2ed6f41bffc6" path="/var/lib/kubelet/pods/68e09eab-1299-4f75-a5ba-2ed6f41bffc6/volumes"
Sep 15 10:52:21.682153 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:21.682116    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:22.265007 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:22.264974    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:22.991984 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:22.991955    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:25.930470 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:25.930414    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:25.988808 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:25.988780    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:26.164726 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:26.164690    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"]
Sep 15 10:52:26.164961 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:26.164939    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz" podUID="ffb8e359-aba1-4c2c-aca5-7634f3d23f0a" containerName="istio-proxy" containerID="cri-o://90fd9112e52bf27a009e916d25b0160530441b05126fae55e3303bfa15a35b8e" gracePeriod=30
Sep 15 10:52:26.185514 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:26.185413    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:28.694473 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:28.694371    2583 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 10:52:28.694851 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:28.694476    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz" podUID="ffb8e359-aba1-4c2c-aca5-7634f3d23f0a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 10:52:30.065355 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:30.065319    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:30.703858 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:30.703824    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:31.396345 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:31.396314    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:31.431138 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:31.431070    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:52:31.485972 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:31.485942    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-istio-podinfo\") pod \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") "
Sep 15 10:52:31.486172 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:31.486008    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpjtd\" (UniqueName: \"kubernetes.io/projected/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-kube-api-access-lpjtd\") pod \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") "
Sep 15 10:52:31.486172 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:31.486048    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-istio-envoy\") pod \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") "
Sep 15 10:52:31.486172 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:31.486092    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-istio-data\") pod \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") "
Sep 15 10:52:31.486172 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:31.486110    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-credential-socket\") pod \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") "
Sep 15 10:52:31.486172 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:31.486132    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-workload-certs\") pod \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") "
Sep 15 10:52:31.486172 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:31.486172    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-istiod-ca-cert\") pod \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") "
Sep 15 10:52:31.486546 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:31.486201    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-workload-socket\") pod \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") "
Sep 15 10:52:31.486546 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:31.486239    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-istio-token\") pod \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\" (UID: \"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a\") "
Sep 15 10:52:31.486633 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:31.486536    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ffb8e359-aba1-4c2c-aca5-7634f3d23f0a" (UID: "ffb8e359-aba1-4c2c-aca5-7634f3d23f0a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:52:31.486808 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:31.486784    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ffb8e359-aba1-4c2c-aca5-7634f3d23f0a" (UID: "ffb8e359-aba1-4c2c-aca5-7634f3d23f0a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:52:31.486916 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:31.486882    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ffb8e359-aba1-4c2c-aca5-7634f3d23f0a" (UID: "ffb8e359-aba1-4c2c-aca5-7634f3d23f0a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:52:31.486987 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:31.486945    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ffb8e359-aba1-4c2c-aca5-7634f3d23f0a" (UID: "ffb8e359-aba1-4c2c-aca5-7634f3d23f0a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:52:31.487048 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:31.486995    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-istio-data" (OuterVolumeSpecName: "istio-data") pod "ffb8e359-aba1-4c2c-aca5-7634f3d23f0a" (UID: "ffb8e359-aba1-4c2c-aca5-7634f3d23f0a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:52:31.488379 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:31.488354    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ffb8e359-aba1-4c2c-aca5-7634f3d23f0a" (UID: "ffb8e359-aba1-4c2c-aca5-7634f3d23f0a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 10:52:31.488379 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:31.488363    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ffb8e359-aba1-4c2c-aca5-7634f3d23f0a" (UID: "ffb8e359-aba1-4c2c-aca5-7634f3d23f0a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:52:31.488541 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:31.488437    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-istio-token" (OuterVolumeSpecName: "istio-token") pod "ffb8e359-aba1-4c2c-aca5-7634f3d23f0a" (UID: "ffb8e359-aba1-4c2c-aca5-7634f3d23f0a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:52:31.488541 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:31.488529    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-kube-api-access-lpjtd" (OuterVolumeSpecName: "kube-api-access-lpjtd") pod "ffb8e359-aba1-4c2c-aca5-7634f3d23f0a" (UID: "ffb8e359-aba1-4c2c-aca5-7634f3d23f0a"). InnerVolumeSpecName "kube-api-access-lpjtd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:52:31.587659 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:31.587613    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-istio-envoy\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:52:31.587659 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:31.587653    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-istio-data\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:52:31.587659 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:31.587669    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-credential-socket\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:52:31.587895 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:31.587683    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-workload-certs\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:52:31.587895 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:31.587696    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-istiod-ca-cert\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:52:31.587895 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:31.587709    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-workload-socket\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:52:31.587895 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:31.587722    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-istio-token\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:52:31.587895 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:31.587735    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-istio-podinfo\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:52:31.587895 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:31.587748    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpjtd\" (UniqueName: \"kubernetes.io/projected/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a-kube-api-access-lpjtd\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:52:31.999129 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:31.999090    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:32.075643 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:32.075602    2583 generic.go:356] "Generic (PLEG): container finished" podID="ffb8e359-aba1-4c2c-aca5-7634f3d23f0a" containerID="90fd9112e52bf27a009e916d25b0160530441b05126fae55e3303bfa15a35b8e" exitCode=0
Sep 15 10:52:32.075845 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:32.075673    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"
Sep 15 10:52:32.075845 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:32.075683    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz" event={"ID":"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a","Type":"ContainerDied","Data":"90fd9112e52bf27a009e916d25b0160530441b05126fae55e3303bfa15a35b8e"}
Sep 15 10:52:32.075845 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:32.075777    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz" event={"ID":"ffb8e359-aba1-4c2c-aca5-7634f3d23f0a","Type":"ContainerDied","Data":"4c070b6ab242f19a0c1e94e7af422ed7c3c0e9e56d4c5940fff009822d73c07f"}
Sep 15 10:52:32.075845 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:32.075809    2583 scope.go:117] "RemoveContainer" containerID="90fd9112e52bf27a009e916d25b0160530441b05126fae55e3303bfa15a35b8e"
Sep 15 10:52:32.086094 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:32.086069    2583 scope.go:117] "RemoveContainer" containerID="a18dca9a14a62a225a3f597500f92551fc16596249467c41be7ac809a08bb80d"
Sep 15 10:52:32.096167 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:32.096132    2583 scope.go:117] "RemoveContainer" containerID="90fd9112e52bf27a009e916d25b0160530441b05126fae55e3303bfa15a35b8e"
Sep 15 10:52:32.096652 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:52:32.096626    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90fd9112e52bf27a009e916d25b0160530441b05126fae55e3303bfa15a35b8e\": container with ID starting with 90fd9112e52bf27a009e916d25b0160530441b05126fae55e3303bfa15a35b8e not found: ID does not exist" containerID="90fd9112e52bf27a009e916d25b0160530441b05126fae55e3303bfa15a35b8e"
Sep 15 10:52:32.096738 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:32.096661    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90fd9112e52bf27a009e916d25b0160530441b05126fae55e3303bfa15a35b8e"} err="failed to get container status \"90fd9112e52bf27a009e916d25b0160530441b05126fae55e3303bfa15a35b8e\": rpc error: code = NotFound desc = could not find container \"90fd9112e52bf27a009e916d25b0160530441b05126fae55e3303bfa15a35b8e\": container with ID starting with 90fd9112e52bf27a009e916d25b0160530441b05126fae55e3303bfa15a35b8e not found: ID does not exist"
Sep 15 10:52:32.096738 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:32.096681    2583 scope.go:117] "RemoveContainer" containerID="a18dca9a14a62a225a3f597500f92551fc16596249467c41be7ac809a08bb80d"
Sep 15 10:52:32.096992 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:52:32.096972    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a18dca9a14a62a225a3f597500f92551fc16596249467c41be7ac809a08bb80d\": container with ID starting with a18dca9a14a62a225a3f597500f92551fc16596249467c41be7ac809a08bb80d not found: ID does not exist" containerID="a18dca9a14a62a225a3f597500f92551fc16596249467c41be7ac809a08bb80d"
Sep 15 10:52:32.097056 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:32.097000    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a18dca9a14a62a225a3f597500f92551fc16596249467c41be7ac809a08bb80d"} err="failed to get container status \"a18dca9a14a62a225a3f597500f92551fc16596249467c41be7ac809a08bb80d\": rpc error: code = NotFound desc = could not find container \"a18dca9a14a62a225a3f597500f92551fc16596249467c41be7ac809a08bb80d\": container with ID starting with a18dca9a14a62a225a3f597500f92551fc16596249467c41be7ac809a08bb80d not found: ID does not exist"
Sep 15 10:52:32.097748 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:32.097726    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"]
Sep 15 10:52:32.101510 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:32.101478    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-1b5e28-openshift-default-f76b8d6cd-rbtzz"]
Sep 15 10:52:32.640273 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:32.640244    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:33.300663 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:33.300628    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffb8e359-aba1-4c2c-aca5-7634f3d23f0a" path="/var/lib/kubelet/pods/ffb8e359-aba1-4c2c-aca5-7634f3d23f0a/volumes"
Sep 15 10:52:33.486361 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:33.486328    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:45.113114 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:45.113073    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq"]
Sep 15 10:52:45.113711 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:45.113392    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq" podUID="322e48a5-9952-406e-ac7a-c4418e8e5a33" containerName="maas-api" containerID="cri-o://2bfe3f9144c1e7a624f12a4c91ff2ece1e075094bbf95607d25b9851b1e025d9" gracePeriod=30
Sep 15 10:52:45.138768 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:45.138734    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:45.201149 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:45.201080    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:45.258757 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:45.258724    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:45.317322 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:45.317290    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:45.371838 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:45.371773    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq"
Sep 15 10:52:45.530008 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:45.529978    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/322e48a5-9952-406e-ac7a-c4418e8e5a33-openshift-service-ca\") pod \"322e48a5-9952-406e-ac7a-c4418e8e5a33\" (UID: \"322e48a5-9952-406e-ac7a-c4418e8e5a33\") "
Sep 15 10:52:45.530183 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:45.530020    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hltjb\" (UniqueName: \"kubernetes.io/projected/322e48a5-9952-406e-ac7a-c4418e8e5a33-kube-api-access-hltjb\") pod \"322e48a5-9952-406e-ac7a-c4418e8e5a33\" (UID: \"322e48a5-9952-406e-ac7a-c4418e8e5a33\") "
Sep 15 10:52:45.530183 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:45.530068    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/322e48a5-9952-406e-ac7a-c4418e8e5a33-metrics-certs\") pod \"322e48a5-9952-406e-ac7a-c4418e8e5a33\" (UID: \"322e48a5-9952-406e-ac7a-c4418e8e5a33\") "
Sep 15 10:52:45.530183 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:45.530118    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/322e48a5-9952-406e-ac7a-c4418e8e5a33-maas-api-tls\") pod \"322e48a5-9952-406e-ac7a-c4418e8e5a33\" (UID: \"322e48a5-9952-406e-ac7a-c4418e8e5a33\") "
Sep 15 10:52:45.530426 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:45.530395    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/322e48a5-9952-406e-ac7a-c4418e8e5a33-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "322e48a5-9952-406e-ac7a-c4418e8e5a33" (UID: "322e48a5-9952-406e-ac7a-c4418e8e5a33"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:52:45.530600 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:45.530576    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/322e48a5-9952-406e-ac7a-c4418e8e5a33-openshift-service-ca\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:52:45.532327 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:45.532295    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/322e48a5-9952-406e-ac7a-c4418e8e5a33-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "322e48a5-9952-406e-ac7a-c4418e8e5a33" (UID: "322e48a5-9952-406e-ac7a-c4418e8e5a33"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:52:45.532428 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:45.532350    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/322e48a5-9952-406e-ac7a-c4418e8e5a33-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "322e48a5-9952-406e-ac7a-c4418e8e5a33" (UID: "322e48a5-9952-406e-ac7a-c4418e8e5a33"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:52:45.532428 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:45.532365    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/322e48a5-9952-406e-ac7a-c4418e8e5a33-kube-api-access-hltjb" (OuterVolumeSpecName: "kube-api-access-hltjb") pod "322e48a5-9952-406e-ac7a-c4418e8e5a33" (UID: "322e48a5-9952-406e-ac7a-c4418e8e5a33"). InnerVolumeSpecName "kube-api-access-hltjb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:52:45.632146 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:45.632055    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/322e48a5-9952-406e-ac7a-c4418e8e5a33-maas-api-tls\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:52:45.632146 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:45.632083    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hltjb\" (UniqueName: \"kubernetes.io/projected/322e48a5-9952-406e-ac7a-c4418e8e5a33-kube-api-access-hltjb\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:52:45.632146 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:45.632093    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/322e48a5-9952-406e-ac7a-c4418e8e5a33-metrics-certs\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:52:45.681099 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:45.681065    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:45.781823 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:45.781792    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:46.166315 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:46.166279    2583 generic.go:356] "Generic (PLEG): container finished" podID="322e48a5-9952-406e-ac7a-c4418e8e5a33" containerID="2bfe3f9144c1e7a624f12a4c91ff2ece1e075094bbf95607d25b9851b1e025d9" exitCode=0
Sep 15 10:52:46.166751 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:46.166328    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq"
Sep 15 10:52:46.166751 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:46.166372    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq" event={"ID":"322e48a5-9952-406e-ac7a-c4418e8e5a33","Type":"ContainerDied","Data":"2bfe3f9144c1e7a624f12a4c91ff2ece1e075094bbf95607d25b9851b1e025d9"}
Sep 15 10:52:46.166751 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:46.166424    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq" event={"ID":"322e48a5-9952-406e-ac7a-c4418e8e5a33","Type":"ContainerDied","Data":"b6a0e019489fefa4ddb93256b5372d19dc7b2a237b50bde28b0eabe9273c6999"}
Sep 15 10:52:46.166751 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:46.166479    2583 scope.go:117] "RemoveContainer" containerID="2bfe3f9144c1e7a624f12a4c91ff2ece1e075094bbf95607d25b9851b1e025d9"
Sep 15 10:52:46.176926 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:46.176903    2583 scope.go:117] "RemoveContainer" containerID="2bfe3f9144c1e7a624f12a4c91ff2ece1e075094bbf95607d25b9851b1e025d9"
Sep 15 10:52:46.177249 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:52:46.177229    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bfe3f9144c1e7a624f12a4c91ff2ece1e075094bbf95607d25b9851b1e025d9\": container with ID starting with 2bfe3f9144c1e7a624f12a4c91ff2ece1e075094bbf95607d25b9851b1e025d9 not found: ID does not exist" containerID="2bfe3f9144c1e7a624f12a4c91ff2ece1e075094bbf95607d25b9851b1e025d9"
Sep 15 10:52:46.177316 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:46.177261    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bfe3f9144c1e7a624f12a4c91ff2ece1e075094bbf95607d25b9851b1e025d9"} err="failed to get container status \"2bfe3f9144c1e7a624f12a4c91ff2ece1e075094bbf95607d25b9851b1e025d9\": rpc error: code = NotFound desc = could not find container \"2bfe3f9144c1e7a624f12a4c91ff2ece1e075094bbf95607d25b9851b1e025d9\": container with ID starting with 2bfe3f9144c1e7a624f12a4c91ff2ece1e075094bbf95607d25b9851b1e025d9 not found: ID does not exist"
Sep 15 10:52:46.188353 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:46.188324    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq"]
Sep 15 10:52:46.190663 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:46.190638    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-5ffc6a-7769bb5dd7-pvzvq"]
Sep 15 10:52:46.489525 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:46.489407    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:47.099207 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:47.099171    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:47.301042 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:47.301004    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="322e48a5-9952-406e-ac7a-c4418e8e5a33" path="/var/lib/kubelet/pods/322e48a5-9952-406e-ac7a-c4418e8e5a33/volumes"
Sep 15 10:52:47.692432 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:47.692400    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:52:50.707318 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:52:50.707282    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:53:27.167057 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:53:27.167025    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:53:28.343815 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:53:28.343773    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:53:33.228929 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:53:33.228895    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:53:33.332559 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:53:33.332525    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:53:33.488830 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:53:33.488732    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:53:34.156853 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:53:34.156818    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:53:34.873189 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:53:34.873154    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:53:35.466600 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:53:35.466566    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:53:36.071126 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:53:36.071083    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:53:36.687783 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:53:36.687750    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:53:44.237595 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:53:44.237557    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:53:44.304296 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:53:44.304264    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:53:44.405195 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:53:44.405165    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:53:44.460820 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:53:44.460785    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:53:44.776423 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:53:44.776387    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:53:44.832702 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:53:44.832668    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:53:45.687160 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:53:45.687125    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:53:46.293774 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:53:46.293744    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:53:50.564327 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:53:50.564281    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:53:50.665080 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:53:50.665042    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:53:50.814759 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:53:50.814676    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:53:52.005134 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:53:52.005102    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:53:52.767924 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:53:52.767894    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:53:53.381179 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:53:53.381143    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:54:01.683174 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:01.683096    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9"]
Sep 15 10:54:01.683743 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:01.683358    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9" podUID="861885eb-eaf6-4765-b1e4-6820797dce8d" containerName="maas-api" containerID="cri-o://6e6bc17e93e675a2c41ea32baafaaef908156dd681752e87e1d7dfe043a59b37" gracePeriod=30
Sep 15 10:54:01.701093 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:01.701061    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:54:01.799024 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:01.798992    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:54:01.877854 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:01.877819    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:54:01.932481 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:01.932438    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9"
Sep 15 10:54:01.940383 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:01.940308    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:54:02.081598 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:02.081567    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4x24\" (UniqueName: \"kubernetes.io/projected/861885eb-eaf6-4765-b1e4-6820797dce8d-kube-api-access-p4x24\") pod \"861885eb-eaf6-4765-b1e4-6820797dce8d\" (UID: \"861885eb-eaf6-4765-b1e4-6820797dce8d\") "
Sep 15 10:54:02.081809 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:02.081641    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/861885eb-eaf6-4765-b1e4-6820797dce8d-metrics-certs\") pod \"861885eb-eaf6-4765-b1e4-6820797dce8d\" (UID: \"861885eb-eaf6-4765-b1e4-6820797dce8d\") "
Sep 15 10:54:02.081809 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:02.081681    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/861885eb-eaf6-4765-b1e4-6820797dce8d-maas-api-tls\") pod \"861885eb-eaf6-4765-b1e4-6820797dce8d\" (UID: \"861885eb-eaf6-4765-b1e4-6820797dce8d\") "
Sep 15 10:54:02.081809 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:02.081745    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/861885eb-eaf6-4765-b1e4-6820797dce8d-openshift-service-ca\") pod \"861885eb-eaf6-4765-b1e4-6820797dce8d\" (UID: \"861885eb-eaf6-4765-b1e4-6820797dce8d\") "
Sep 15 10:54:02.082183 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:02.082151    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/861885eb-eaf6-4765-b1e4-6820797dce8d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "861885eb-eaf6-4765-b1e4-6820797dce8d" (UID: "861885eb-eaf6-4765-b1e4-6820797dce8d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:54:02.083766 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:02.083739    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/861885eb-eaf6-4765-b1e4-6820797dce8d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "861885eb-eaf6-4765-b1e4-6820797dce8d" (UID: "861885eb-eaf6-4765-b1e4-6820797dce8d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:54:02.083882 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:02.083840    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/861885eb-eaf6-4765-b1e4-6820797dce8d-kube-api-access-p4x24" (OuterVolumeSpecName: "kube-api-access-p4x24") pod "861885eb-eaf6-4765-b1e4-6820797dce8d" (UID: "861885eb-eaf6-4765-b1e4-6820797dce8d"). InnerVolumeSpecName "kube-api-access-p4x24". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:54:02.083882 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:02.083858    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/861885eb-eaf6-4765-b1e4-6820797dce8d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "861885eb-eaf6-4765-b1e4-6820797dce8d" (UID: "861885eb-eaf6-4765-b1e4-6820797dce8d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 10:54:02.183039 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:02.182996    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4x24\" (UniqueName: \"kubernetes.io/projected/861885eb-eaf6-4765-b1e4-6820797dce8d-kube-api-access-p4x24\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:54:02.183039 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:02.183031    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/861885eb-eaf6-4765-b1e4-6820797dce8d-metrics-certs\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:54:02.183039 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:02.183042    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/861885eb-eaf6-4765-b1e4-6820797dce8d-maas-api-tls\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:54:02.183039 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:02.183050    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/861885eb-eaf6-4765-b1e4-6820797dce8d-openshift-service-ca\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:54:02.235119 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:02.235030    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:54:02.335311 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:02.335276    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:54:02.607175 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:02.607089    2583 generic.go:356] "Generic (PLEG): container finished" podID="861885eb-eaf6-4765-b1e4-6820797dce8d" containerID="6e6bc17e93e675a2c41ea32baafaaef908156dd681752e87e1d7dfe043a59b37" exitCode=0
Sep 15 10:54:02.607175 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:02.607137    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9"
Sep 15 10:54:02.607362 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:02.607172    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9" event={"ID":"861885eb-eaf6-4765-b1e4-6820797dce8d","Type":"ContainerDied","Data":"6e6bc17e93e675a2c41ea32baafaaef908156dd681752e87e1d7dfe043a59b37"}
Sep 15 10:54:02.607362 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:02.607230    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9" event={"ID":"861885eb-eaf6-4765-b1e4-6820797dce8d","Type":"ContainerDied","Data":"0780624a3680bbfe96e0d5b8bd40460398374c507af52d53860620731ab0ae69"}
Sep 15 10:54:02.607362 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:02.607259    2583 scope.go:117] "RemoveContainer" containerID="6e6bc17e93e675a2c41ea32baafaaef908156dd681752e87e1d7dfe043a59b37"
Sep 15 10:54:02.617653 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:02.617633    2583 scope.go:117] "RemoveContainer" containerID="6e6bc17e93e675a2c41ea32baafaaef908156dd681752e87e1d7dfe043a59b37"
Sep 15 10:54:02.617952 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:54:02.617933    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e6bc17e93e675a2c41ea32baafaaef908156dd681752e87e1d7dfe043a59b37\": container with ID starting with 6e6bc17e93e675a2c41ea32baafaaef908156dd681752e87e1d7dfe043a59b37 not found: ID does not exist" containerID="6e6bc17e93e675a2c41ea32baafaaef908156dd681752e87e1d7dfe043a59b37"
Sep 15 10:54:02.618009 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:02.617961    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e6bc17e93e675a2c41ea32baafaaef908156dd681752e87e1d7dfe043a59b37"} err="failed to get container status \"6e6bc17e93e675a2c41ea32baafaaef908156dd681752e87e1d7dfe043a59b37\": rpc error: code = NotFound desc = could not find container \"6e6bc17e93e675a2c41ea32baafaaef908156dd681752e87e1d7dfe043a59b37\": container with ID starting with 6e6bc17e93e675a2c41ea32baafaaef908156dd681752e87e1d7dfe043a59b37 not found: ID does not exist"
Sep 15 10:54:02.629569 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:02.629532    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9"]
Sep 15 10:54:02.633280 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:02.633252    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-83e814-6fb98f6d98-xnqw9"]
Sep 15 10:54:02.901985 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:02.901955    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:54:03.300281 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:03.300199    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="861885eb-eaf6-4765-b1e4-6820797dce8d" path="/var/lib/kubelet/pods/861885eb-eaf6-4765-b1e4-6820797dce8d/volumes"
Sep 15 10:54:08.288431 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:08.288393    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:54:08.413149 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:08.413108    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:54:08.551896 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:08.551792    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"]
Sep 15 10:54:08.552084 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:08.552061    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5" podUID="f66d9141-89c8-40cf-a691-d8d19d2fd610" containerName="istio-proxy" containerID="cri-o://f4b8edc426824e00fafd06d72a745586292378c7bed432e91298ef7130013fd4" gracePeriod=30
Sep 15 10:54:08.567494 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:08.567426    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:54:09.184212 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:09.184182    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:54:09.325482 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:09.325427    2583 patch_prober.go:28] interesting pod/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 15 10:54:09.325846 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:09.325512    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5" podUID="f66d9141-89c8-40cf-a691-d8d19d2fd610" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 15 10:54:09.994060 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:09.994029    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:54:10.603203 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:10.603167    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:54:13.803276 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:13.803242    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:54:13.891807 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:13.891774    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f66d9141-89c8-40cf-a691-d8d19d2fd610-workload-certs\") pod \"f66d9141-89c8-40cf-a691-d8d19d2fd610\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") "
Sep 15 10:54:13.891974 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:13.891819    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f66d9141-89c8-40cf-a691-d8d19d2fd610-istio-data\") pod \"f66d9141-89c8-40cf-a691-d8d19d2fd610\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") "
Sep 15 10:54:13.891974 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:13.891874    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f66d9141-89c8-40cf-a691-d8d19d2fd610-istio-podinfo\") pod \"f66d9141-89c8-40cf-a691-d8d19d2fd610\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") "
Sep 15 10:54:13.891974 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:13.891919    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f66d9141-89c8-40cf-a691-d8d19d2fd610-istiod-ca-cert\") pod \"f66d9141-89c8-40cf-a691-d8d19d2fd610\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") "
Sep 15 10:54:13.891974 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:13.891963    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f66d9141-89c8-40cf-a691-d8d19d2fd610-credential-socket\") pod \"f66d9141-89c8-40cf-a691-d8d19d2fd610\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") "
Sep 15 10:54:13.892195 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:13.891995    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f66d9141-89c8-40cf-a691-d8d19d2fd610-istio-token\") pod \"f66d9141-89c8-40cf-a691-d8d19d2fd610\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") "
Sep 15 10:54:13.892195 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:13.892028    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f66d9141-89c8-40cf-a691-d8d19d2fd610-istio-envoy\") pod \"f66d9141-89c8-40cf-a691-d8d19d2fd610\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") "
Sep 15 10:54:13.892195 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:13.892059    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qnxjm\" (UniqueName: \"kubernetes.io/projected/f66d9141-89c8-40cf-a691-d8d19d2fd610-kube-api-access-qnxjm\") pod \"f66d9141-89c8-40cf-a691-d8d19d2fd610\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") "
Sep 15 10:54:13.892195 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:13.892086    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f66d9141-89c8-40cf-a691-d8d19d2fd610-workload-socket\") pod \"f66d9141-89c8-40cf-a691-d8d19d2fd610\" (UID: \"f66d9141-89c8-40cf-a691-d8d19d2fd610\") "
Sep 15 10:54:13.892474 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:13.892075    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f66d9141-89c8-40cf-a691-d8d19d2fd610-istio-data" (OuterVolumeSpecName: "istio-data") pod "f66d9141-89c8-40cf-a691-d8d19d2fd610" (UID: "f66d9141-89c8-40cf-a691-d8d19d2fd610"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:54:13.892608 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:13.892332    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f66d9141-89c8-40cf-a691-d8d19d2fd610-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f66d9141-89c8-40cf-a691-d8d19d2fd610" (UID: "f66d9141-89c8-40cf-a691-d8d19d2fd610"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:54:13.892608 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:13.892362    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f66d9141-89c8-40cf-a691-d8d19d2fd610-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f66d9141-89c8-40cf-a691-d8d19d2fd610" (UID: "f66d9141-89c8-40cf-a691-d8d19d2fd610"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 10:54:13.892608 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:13.892492    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f66d9141-89c8-40cf-a691-d8d19d2fd610-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f66d9141-89c8-40cf-a691-d8d19d2fd610" (UID: "f66d9141-89c8-40cf-a691-d8d19d2fd610"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:54:13.892608 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:13.892571    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f66d9141-89c8-40cf-a691-d8d19d2fd610-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f66d9141-89c8-40cf-a691-d8d19d2fd610" (UID: "f66d9141-89c8-40cf-a691-d8d19d2fd610"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:54:13.894241 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:13.894217    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f66d9141-89c8-40cf-a691-d8d19d2fd610-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f66d9141-89c8-40cf-a691-d8d19d2fd610" (UID: "f66d9141-89c8-40cf-a691-d8d19d2fd610"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 15 10:54:13.894328 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:13.894225    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f66d9141-89c8-40cf-a691-d8d19d2fd610-kube-api-access-qnxjm" (OuterVolumeSpecName: "kube-api-access-qnxjm") pod "f66d9141-89c8-40cf-a691-d8d19d2fd610" (UID: "f66d9141-89c8-40cf-a691-d8d19d2fd610"). InnerVolumeSpecName "kube-api-access-qnxjm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:54:13.894626 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:13.894605    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f66d9141-89c8-40cf-a691-d8d19d2fd610-istio-token" (OuterVolumeSpecName: "istio-token") pod "f66d9141-89c8-40cf-a691-d8d19d2fd610" (UID: "f66d9141-89c8-40cf-a691-d8d19d2fd610"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 10:54:13.894626 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:13.894608    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f66d9141-89c8-40cf-a691-d8d19d2fd610-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f66d9141-89c8-40cf-a691-d8d19d2fd610" (UID: "f66d9141-89c8-40cf-a691-d8d19d2fd610"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 15 10:54:13.993471 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:13.993404    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f66d9141-89c8-40cf-a691-d8d19d2fd610-istiod-ca-cert\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:54:13.993471 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:13.993437    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f66d9141-89c8-40cf-a691-d8d19d2fd610-credential-socket\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:54:13.993681 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:13.993480    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f66d9141-89c8-40cf-a691-d8d19d2fd610-istio-token\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:54:13.993681 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:13.993496    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f66d9141-89c8-40cf-a691-d8d19d2fd610-istio-envoy\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:54:13.993681 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:13.993509    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnxjm\" (UniqueName: \"kubernetes.io/projected/f66d9141-89c8-40cf-a691-d8d19d2fd610-kube-api-access-qnxjm\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:54:13.993681 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:13.993522    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f66d9141-89c8-40cf-a691-d8d19d2fd610-workload-socket\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:54:13.993681 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:13.993532    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f66d9141-89c8-40cf-a691-d8d19d2fd610-workload-certs\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:54:13.993681 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:13.993540    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f66d9141-89c8-40cf-a691-d8d19d2fd610-istio-data\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:54:13.993681 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:13.993547    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f66d9141-89c8-40cf-a691-d8d19d2fd610-istio-podinfo\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 10:54:14.673167 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:14.673128    2583 generic.go:356] "Generic (PLEG): container finished" podID="f66d9141-89c8-40cf-a691-d8d19d2fd610" containerID="f4b8edc426824e00fafd06d72a745586292378c7bed432e91298ef7130013fd4" exitCode=0
Sep 15 10:54:14.673361 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:14.673184    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"
Sep 15 10:54:14.673361 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:14.673221    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5" event={"ID":"f66d9141-89c8-40cf-a691-d8d19d2fd610","Type":"ContainerDied","Data":"f4b8edc426824e00fafd06d72a745586292378c7bed432e91298ef7130013fd4"}
Sep 15 10:54:14.673361 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:14.673280    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5" event={"ID":"f66d9141-89c8-40cf-a691-d8d19d2fd610","Type":"ContainerDied","Data":"72ad55c07fb65c5948d012882a2efe123bfa52f09a66b61f7f9f82fa810da442"}
Sep 15 10:54:14.673361 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:14.673303    2583 scope.go:117] "RemoveContainer" containerID="f4b8edc426824e00fafd06d72a745586292378c7bed432e91298ef7130013fd4"
Sep 15 10:54:14.684952 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:14.684932    2583 scope.go:117] "RemoveContainer" containerID="d33a3316bcfdd1aca57469f83ba9e4a5049a25578b8e905e562a8d98d8b54e6a"
Sep 15 10:54:14.693694 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:14.693673    2583 scope.go:117] "RemoveContainer" containerID="f4b8edc426824e00fafd06d72a745586292378c7bed432e91298ef7130013fd4"
Sep 15 10:54:14.694007 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:54:14.693988    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4b8edc426824e00fafd06d72a745586292378c7bed432e91298ef7130013fd4\": container with ID starting with f4b8edc426824e00fafd06d72a745586292378c7bed432e91298ef7130013fd4 not found: ID does not exist" containerID="f4b8edc426824e00fafd06d72a745586292378c7bed432e91298ef7130013fd4"
Sep 15 10:54:14.694049 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:14.694019    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4b8edc426824e00fafd06d72a745586292378c7bed432e91298ef7130013fd4"} err="failed to get container status \"f4b8edc426824e00fafd06d72a745586292378c7bed432e91298ef7130013fd4\": rpc error: code = NotFound desc = could not find container \"f4b8edc426824e00fafd06d72a745586292378c7bed432e91298ef7130013fd4\": container with ID starting with f4b8edc426824e00fafd06d72a745586292378c7bed432e91298ef7130013fd4 not found: ID does not exist"
Sep 15 10:54:14.694049 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:14.694037    2583 scope.go:117] "RemoveContainer" containerID="d33a3316bcfdd1aca57469f83ba9e4a5049a25578b8e905e562a8d98d8b54e6a"
Sep 15 10:54:14.694283 ip-10-0-141-235 kubenswrapper[2583]: E0915 10:54:14.694268    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d33a3316bcfdd1aca57469f83ba9e4a5049a25578b8e905e562a8d98d8b54e6a\": container with ID starting with d33a3316bcfdd1aca57469f83ba9e4a5049a25578b8e905e562a8d98d8b54e6a not found: ID does not exist" containerID="d33a3316bcfdd1aca57469f83ba9e4a5049a25578b8e905e562a8d98d8b54e6a"
Sep 15 10:54:14.694321 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:14.694290    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d33a3316bcfdd1aca57469f83ba9e4a5049a25578b8e905e562a8d98d8b54e6a"} err="failed to get container status \"d33a3316bcfdd1aca57469f83ba9e4a5049a25578b8e905e562a8d98d8b54e6a\": rpc error: code = NotFound desc = could not find container \"d33a3316bcfdd1aca57469f83ba9e4a5049a25578b8e905e562a8d98d8b54e6a\": container with ID starting with d33a3316bcfdd1aca57469f83ba9e4a5049a25578b8e905e562a8d98d8b54e6a not found: ID does not exist"
Sep 15 10:54:14.699525 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:14.699494    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"]
Sep 15 10:54:14.704611 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:14.704585    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-83e814-openshift-default-5bcd85d8f8-57mt5"]
Sep 15 10:54:15.300305 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:15.300262    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f66d9141-89c8-40cf-a691-d8d19d2fd610" path="/var/lib/kubelet/pods/f66d9141-89c8-40cf-a691-d8d19d2fd610/volumes"
Sep 15 10:54:19.442329 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:19.442296    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:54:19.543406 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:19.543375    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:54:19.613267 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:19.613210    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:54:19.692593 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:19.692505    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:54:19.978982 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:19.978892    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:54:20.079851 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:20.079817    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:54:21.369432 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:21.369398    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:54:21.470262 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:21.470232    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:54:22.260392 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:22.260364    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:54:25.802556 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:25.802516    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:54:25.881961 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:25.881927    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:54:26.063417 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:26.063323    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:54:54.940639 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:54.940604    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:54:55.041578 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:54:55.041539    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:55:53.699178 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:55:53.699131    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:55:55.242265 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:55:55.242229    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:57:17.665774 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:57:17.665739    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:58:10.242785 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:58:10.242750    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:58:52.004289 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:58:52.004257    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:58:59.499178 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:58:59.499137    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:59:25.003125 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:25.003086    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:59:26.760740 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:26.760710    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 10:59:42.975149 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:42.975111    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7dbbd6f894-4284z"]
Sep 15 10:59:42.975616 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:42.975521    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffb8e359-aba1-4c2c-aca5-7634f3d23f0a" containerName="istio-proxy"
Sep 15 10:59:42.975616 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:42.975535    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffb8e359-aba1-4c2c-aca5-7634f3d23f0a" containerName="istio-proxy"
Sep 15 10:59:42.975616 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:42.975549    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68e09eab-1299-4f75-a5ba-2ed6f41bffc6" containerName="maas-api"
Sep 15 10:59:42.975616 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:42.975555    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="68e09eab-1299-4f75-a5ba-2ed6f41bffc6" containerName="maas-api"
Sep 15 10:59:42.975616 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:42.975564    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f66d9141-89c8-40cf-a691-d8d19d2fd610" containerName="istio-proxy"
Sep 15 10:59:42.975616 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:42.975570    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f66d9141-89c8-40cf-a691-d8d19d2fd610" containerName="istio-proxy"
Sep 15 10:59:42.975616 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:42.975579    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="861885eb-eaf6-4765-b1e4-6820797dce8d" containerName="maas-api"
Sep 15 10:59:42.975616 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:42.975584    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="861885eb-eaf6-4765-b1e4-6820797dce8d" containerName="maas-api"
Sep 15 10:59:42.975616 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:42.975592    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f66d9141-89c8-40cf-a691-d8d19d2fd610" containerName="istio-proxy"
Sep 15 10:59:42.975616 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:42.975596    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f66d9141-89c8-40cf-a691-d8d19d2fd610" containerName="istio-proxy"
Sep 15 10:59:42.975616 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:42.975610    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="322e48a5-9952-406e-ac7a-c4418e8e5a33" containerName="maas-api"
Sep 15 10:59:42.975616 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:42.975616    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="322e48a5-9952-406e-ac7a-c4418e8e5a33" containerName="maas-api"
Sep 15 10:59:42.975989 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:42.975666    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffb8e359-aba1-4c2c-aca5-7634f3d23f0a" containerName="istio-proxy"
Sep 15 10:59:42.975989 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:42.975674    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f66d9141-89c8-40cf-a691-d8d19d2fd610" containerName="istio-proxy"
Sep 15 10:59:42.975989 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:42.975681    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffb8e359-aba1-4c2c-aca5-7634f3d23f0a" containerName="istio-proxy"
Sep 15 10:59:42.975989 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:42.975689    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="68e09eab-1299-4f75-a5ba-2ed6f41bffc6" containerName="maas-api"
Sep 15 10:59:42.975989 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:42.975694    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f66d9141-89c8-40cf-a691-d8d19d2fd610" containerName="istio-proxy"
Sep 15 10:59:42.975989 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:42.975701    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="322e48a5-9952-406e-ac7a-c4418e8e5a33" containerName="maas-api"
Sep 15 10:59:42.975989 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:42.975706    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="861885eb-eaf6-4765-b1e4-6820797dce8d" containerName="maas-api"
Sep 15 10:59:42.978651 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:42.978634    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7dbbd6f894-4284z"
Sep 15 10:59:42.989031 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:42.989003    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7dbbd6f894-4284z"]
Sep 15 10:59:43.078201 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:43.078158    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f410bd43-3338-4994-840a-05867d1d7e43-maas-api-tls\") pod \"maas-api-7dbbd6f894-4284z\" (UID: \"f410bd43-3338-4994-840a-05867d1d7e43\") " pod="odh-ai-gateway-infra/maas-api-7dbbd6f894-4284z"
Sep 15 10:59:43.078201 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:43.078205    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txz8w\" (UniqueName: \"kubernetes.io/projected/f410bd43-3338-4994-840a-05867d1d7e43-kube-api-access-txz8w\") pod \"maas-api-7dbbd6f894-4284z\" (UID: \"f410bd43-3338-4994-840a-05867d1d7e43\") " pod="odh-ai-gateway-infra/maas-api-7dbbd6f894-4284z"
Sep 15 10:59:43.078503 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:43.078271    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f410bd43-3338-4994-840a-05867d1d7e43-metrics-certs\") pod \"maas-api-7dbbd6f894-4284z\" (UID: \"f410bd43-3338-4994-840a-05867d1d7e43\") " pod="odh-ai-gateway-infra/maas-api-7dbbd6f894-4284z"
Sep 15 10:59:43.078503 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:43.078374    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f410bd43-3338-4994-840a-05867d1d7e43-openshift-service-ca\") pod \"maas-api-7dbbd6f894-4284z\" (UID: \"f410bd43-3338-4994-840a-05867d1d7e43\") " pod="odh-ai-gateway-infra/maas-api-7dbbd6f894-4284z"
Sep 15 10:59:43.179131 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:43.179085    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f410bd43-3338-4994-840a-05867d1d7e43-maas-api-tls\") pod \"maas-api-7dbbd6f894-4284z\" (UID: \"f410bd43-3338-4994-840a-05867d1d7e43\") " pod="odh-ai-gateway-infra/maas-api-7dbbd6f894-4284z"
Sep 15 10:59:43.179131 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:43.179136    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txz8w\" (UniqueName: \"kubernetes.io/projected/f410bd43-3338-4994-840a-05867d1d7e43-kube-api-access-txz8w\") pod \"maas-api-7dbbd6f894-4284z\" (UID: \"f410bd43-3338-4994-840a-05867d1d7e43\") " pod="odh-ai-gateway-infra/maas-api-7dbbd6f894-4284z"
Sep 15 10:59:43.179408 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:43.179169    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f410bd43-3338-4994-840a-05867d1d7e43-metrics-certs\") pod \"maas-api-7dbbd6f894-4284z\" (UID: \"f410bd43-3338-4994-840a-05867d1d7e43\") " pod="odh-ai-gateway-infra/maas-api-7dbbd6f894-4284z"
Sep 15 10:59:43.179408 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:43.179246    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f410bd43-3338-4994-840a-05867d1d7e43-openshift-service-ca\") pod \"maas-api-7dbbd6f894-4284z\" (UID: \"f410bd43-3338-4994-840a-05867d1d7e43\") " pod="odh-ai-gateway-infra/maas-api-7dbbd6f894-4284z"
Sep 15 10:59:43.179952 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:43.179923    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f410bd43-3338-4994-840a-05867d1d7e43-openshift-service-ca\") pod \"maas-api-7dbbd6f894-4284z\" (UID: \"f410bd43-3338-4994-840a-05867d1d7e43\") " pod="odh-ai-gateway-infra/maas-api-7dbbd6f894-4284z"
Sep 15 10:59:43.181610 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:43.181592    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f410bd43-3338-4994-840a-05867d1d7e43-maas-api-tls\") pod \"maas-api-7dbbd6f894-4284z\" (UID: \"f410bd43-3338-4994-840a-05867d1d7e43\") " pod="odh-ai-gateway-infra/maas-api-7dbbd6f894-4284z"
Sep 15 10:59:43.181706 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:43.181690    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f410bd43-3338-4994-840a-05867d1d7e43-metrics-certs\") pod \"maas-api-7dbbd6f894-4284z\" (UID: \"f410bd43-3338-4994-840a-05867d1d7e43\") " pod="odh-ai-gateway-infra/maas-api-7dbbd6f894-4284z"
Sep 15 10:59:43.186651 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:43.186628    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txz8w\" (UniqueName: \"kubernetes.io/projected/f410bd43-3338-4994-840a-05867d1d7e43-kube-api-access-txz8w\") pod \"maas-api-7dbbd6f894-4284z\" (UID: \"f410bd43-3338-4994-840a-05867d1d7e43\") " pod="odh-ai-gateway-infra/maas-api-7dbbd6f894-4284z"
Sep 15 10:59:43.289783 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:43.289689    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7dbbd6f894-4284z"
Sep 15 10:59:43.564295 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:43.564262    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7dbbd6f894-4284z"]
Sep 15 10:59:43.565436 ip-10-0-141-235 kubenswrapper[2583]: W0915 10:59:43.565411    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf410bd43_3338_4994_840a_05867d1d7e43.slice/crio-6bf344586eb401950f90a9e0dd53182eedd1b6220f01393dd73a85d59844a48e WatchSource:0}: Error finding container 6bf344586eb401950f90a9e0dd53182eedd1b6220f01393dd73a85d59844a48e: Status 404 returned error can't find the container with id 6bf344586eb401950f90a9e0dd53182eedd1b6220f01393dd73a85d59844a48e
Sep 15 10:59:43.566770 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:43.566750    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 15 10:59:44.498043 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:44.497998    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7dbbd6f894-4284z" event={"ID":"f410bd43-3338-4994-840a-05867d1d7e43","Type":"ContainerStarted","Data":"9d7c01c7dd04149e1387e15416fbba76aac8b5621be69e942f89308853b244a3"}
Sep 15 10:59:44.498480 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:44.498066    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7dbbd6f894-4284z" event={"ID":"f410bd43-3338-4994-840a-05867d1d7e43","Type":"ContainerStarted","Data":"6bf344586eb401950f90a9e0dd53182eedd1b6220f01393dd73a85d59844a48e"}
Sep 15 10:59:44.498480 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:44.498095    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7dbbd6f894-4284z"
Sep 15 10:59:44.517793 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:44.517736    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7dbbd6f894-4284z" podStartSLOduration=2.101998327 podStartE2EDuration="2.517717611s" totalImagesPullingTime="415.719284ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-15 10:59:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 10:59:44.516747497 +0000 UTC m=+2499.763714467" watchObservedRunningTime="2026-09-15 10:59:44.517717611 +0000 UTC m=+2499.764684618"
Sep 15 10:59:50.510089 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:50.510059    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7dbbd6f894-4284z"
Sep 15 10:59:58.649378 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:58.649285    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5b77bbc99d-756d5"]
Sep 15 10:59:58.649803 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:58.649595    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5b77bbc99d-756d5" podUID="29bb1f2b-9b06-4d79-935a-c7a99dc43dd7" containerName="maas-api" containerID="cri-o://ffecff1f56a64bb97f95b46decbf7932abbdd91e79457d41724122b788ee719a" gracePeriod=30
Sep 15 10:59:59.892087 ip-10-0-141-235 kubenswrapper[2583]: I0915 10:59:59.892056    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5b77bbc99d-756d5"
Sep 15 11:00:00.038191 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:00.038096    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6s9f\" (UniqueName: \"kubernetes.io/projected/29bb1f2b-9b06-4d79-935a-c7a99dc43dd7-kube-api-access-c6s9f\") pod \"29bb1f2b-9b06-4d79-935a-c7a99dc43dd7\" (UID: \"29bb1f2b-9b06-4d79-935a-c7a99dc43dd7\") "
Sep 15 11:00:00.038357 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:00.038197    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29bb1f2b-9b06-4d79-935a-c7a99dc43dd7-metrics-certs\") pod \"29bb1f2b-9b06-4d79-935a-c7a99dc43dd7\" (UID: \"29bb1f2b-9b06-4d79-935a-c7a99dc43dd7\") "
Sep 15 11:00:00.038357 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:00.038221    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29bb1f2b-9b06-4d79-935a-c7a99dc43dd7-openshift-service-ca\") pod \"29bb1f2b-9b06-4d79-935a-c7a99dc43dd7\" (UID: \"29bb1f2b-9b06-4d79-935a-c7a99dc43dd7\") "
Sep 15 11:00:00.038357 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:00.038264    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29bb1f2b-9b06-4d79-935a-c7a99dc43dd7-maas-api-tls\") pod \"29bb1f2b-9b06-4d79-935a-c7a99dc43dd7\" (UID: \"29bb1f2b-9b06-4d79-935a-c7a99dc43dd7\") "
Sep 15 11:00:00.038670 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:00.038640    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29bb1f2b-9b06-4d79-935a-c7a99dc43dd7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "29bb1f2b-9b06-4d79-935a-c7a99dc43dd7" (UID: "29bb1f2b-9b06-4d79-935a-c7a99dc43dd7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 15 11:00:00.040345 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:00.040319    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29bb1f2b-9b06-4d79-935a-c7a99dc43dd7-kube-api-access-c6s9f" (OuterVolumeSpecName: "kube-api-access-c6s9f") pod "29bb1f2b-9b06-4d79-935a-c7a99dc43dd7" (UID: "29bb1f2b-9b06-4d79-935a-c7a99dc43dd7"). InnerVolumeSpecName "kube-api-access-c6s9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 15 11:00:00.040570 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:00.040342    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29bb1f2b-9b06-4d79-935a-c7a99dc43dd7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "29bb1f2b-9b06-4d79-935a-c7a99dc43dd7" (UID: "29bb1f2b-9b06-4d79-935a-c7a99dc43dd7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 11:00:00.040570 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:00.040324    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29bb1f2b-9b06-4d79-935a-c7a99dc43dd7-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "29bb1f2b-9b06-4d79-935a-c7a99dc43dd7" (UID: "29bb1f2b-9b06-4d79-935a-c7a99dc43dd7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 15 11:00:00.139588 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:00.139554    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29bb1f2b-9b06-4d79-935a-c7a99dc43dd7-maas-api-tls\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 11:00:00.139588 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:00.139583    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6s9f\" (UniqueName: \"kubernetes.io/projected/29bb1f2b-9b06-4d79-935a-c7a99dc43dd7-kube-api-access-c6s9f\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 11:00:00.139588 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:00.139593    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29bb1f2b-9b06-4d79-935a-c7a99dc43dd7-metrics-certs\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 11:00:00.139828 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:00.139602    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29bb1f2b-9b06-4d79-935a-c7a99dc43dd7-openshift-service-ca\") on node \"ip-10-0-141-235.ec2.internal\" DevicePath \"\""
Sep 15 11:00:00.584830 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:00.584798    2583 generic.go:356] "Generic (PLEG): container finished" podID="29bb1f2b-9b06-4d79-935a-c7a99dc43dd7" containerID="ffecff1f56a64bb97f95b46decbf7932abbdd91e79457d41724122b788ee719a" exitCode=0
Sep 15 11:00:00.585039 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:00.584858    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5b77bbc99d-756d5"
Sep 15 11:00:00.585039 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:00.584888    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5b77bbc99d-756d5" event={"ID":"29bb1f2b-9b06-4d79-935a-c7a99dc43dd7","Type":"ContainerDied","Data":"ffecff1f56a64bb97f95b46decbf7932abbdd91e79457d41724122b788ee719a"}
Sep 15 11:00:00.585039 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:00.584952    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5b77bbc99d-756d5" event={"ID":"29bb1f2b-9b06-4d79-935a-c7a99dc43dd7","Type":"ContainerDied","Data":"e6ee94d1d954c79f34f32808ebcfcbfdb4966dd82b0897c4d6601e25db88bc0f"}
Sep 15 11:00:00.585039 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:00.584981    2583 scope.go:117] "RemoveContainer" containerID="ffecff1f56a64bb97f95b46decbf7932abbdd91e79457d41724122b788ee719a"
Sep 15 11:00:00.601548 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:00.601522    2583 scope.go:117] "RemoveContainer" containerID="ffecff1f56a64bb97f95b46decbf7932abbdd91e79457d41724122b788ee719a"
Sep 15 11:00:00.601890 ip-10-0-141-235 kubenswrapper[2583]: E0915 11:00:00.601870    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffecff1f56a64bb97f95b46decbf7932abbdd91e79457d41724122b788ee719a\": container with ID starting with ffecff1f56a64bb97f95b46decbf7932abbdd91e79457d41724122b788ee719a not found: ID does not exist" containerID="ffecff1f56a64bb97f95b46decbf7932abbdd91e79457d41724122b788ee719a"
Sep 15 11:00:00.601957 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:00.601903    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffecff1f56a64bb97f95b46decbf7932abbdd91e79457d41724122b788ee719a"} err="failed to get container status \"ffecff1f56a64bb97f95b46decbf7932abbdd91e79457d41724122b788ee719a\": rpc error: code = NotFound desc = could not find container \"ffecff1f56a64bb97f95b46decbf7932abbdd91e79457d41724122b788ee719a\": container with ID starting with ffecff1f56a64bb97f95b46decbf7932abbdd91e79457d41724122b788ee719a not found: ID does not exist"
Sep 15 11:00:00.609832 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:00.609803    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5b77bbc99d-756d5"]
Sep 15 11:00:00.614977 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:00.614949    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5b77bbc99d-756d5"]
Sep 15 11:00:01.300727 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:01.300687    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29bb1f2b-9b06-4d79-935a-c7a99dc43dd7" path="/var/lib/kubelet/pods/29bb1f2b-9b06-4d79-935a-c7a99dc43dd7/volumes"
Sep 15 11:00:11.822615 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:11.822574    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 11:00:18.198278 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:18.198247    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-xvnnk_71e3decd-7333-4ed5-82cf-e82de3601c18/server/0.log"
Sep 15 11:00:18.322082 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:18.322046    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-bbfhr_e3b4daf0-82f9-4aef-801d-a9ab121f984c/manager/0.log"
Sep 15 11:00:18.660685 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:18.660657    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6699d7bd9c-bnltb_105dfa5e-4c1b-4913-bf15-7cda5d01f89b/copy-manifests/0.log"
Sep 15 11:00:18.666950 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:18.666929    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6699d7bd9c-bnltb_105dfa5e-4c1b-4913-bf15-7cda5d01f89b/manager/0.log"
Sep 15 11:00:18.986820 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:18.986739    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7dbbd6f894-4284z_f410bd43-3338-4994-840a-05867d1d7e43/maas-api/0.log"
Sep 15 11:00:19.475123 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:19.475083    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 11:00:19.525079 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:19.525056    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 11:00:21.229245 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:21.229218    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cp69vw_585f0d63-e037-4387-b8dd-e4530c18c319/istio-proxy/0.log"
Sep 15 11:00:21.344102 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:21.344069    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-24mmv_8106e39c-4f35-4890-91eb-1204ca4ccc4b/discovery/0.log"
Sep 15 11:00:22.410458 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:22.410419    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-wttp4_0622f16d-0989-453a-bb5c-0e7ed6669318/main/0.log"
Sep 15 11:00:22.518352 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:22.518321    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-gx6vt_cf199338-8a03-45c6-be1c-aca427903dcc/main/0.log"
Sep 15 11:00:22.855056 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:22.854977    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-8ptwc_0ceb4a31-6489-4363-b35e-f5aa3adacd43/main/0.log"
Sep 15 11:00:28.442483 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:28.442424    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 11:00:28.542031 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:28.542000    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log" limit=1024
Sep 15 11:00:31.261894 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:31.261858    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qjcwx_d9306059-36e0-4eae-8dc8-3ce075dc9b21/global-pull-secret-syncer/0.log"
Sep 15 11:00:31.342707 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:31.342671    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8hkxj_17509e5d-d9dc-4f53-a111-4929185b0a3c/konnectivity-agent/0.log"
Sep 15 11:00:31.468898 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:31.468869    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-235.ec2.internal_2b3d6833f74b07abf73dd136dadede80/haproxy/0.log"
Sep 15 11:00:37.791883 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:37.791851    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e09a6ac0-aeb1-4dae-97c1-08fcaee53de4/alertmanager/0.log"
Sep 15 11:00:37.813298 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:37.813261    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e09a6ac0-aeb1-4dae-97c1-08fcaee53de4/config-reloader/0.log"
Sep 15 11:00:37.833691 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:37.833661    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e09a6ac0-aeb1-4dae-97c1-08fcaee53de4/kube-rbac-proxy-web/0.log"
Sep 15 11:00:37.863315 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:37.863285    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e09a6ac0-aeb1-4dae-97c1-08fcaee53de4/kube-rbac-proxy/0.log"
Sep 15 11:00:37.884183 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:37.884154    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e09a6ac0-aeb1-4dae-97c1-08fcaee53de4/kube-rbac-proxy-metric/0.log"
Sep 15 11:00:37.905392 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:37.905367    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e09a6ac0-aeb1-4dae-97c1-08fcaee53de4/prom-label-proxy/0.log"
Sep 15 11:00:37.933277 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:37.933219    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e09a6ac0-aeb1-4dae-97c1-08fcaee53de4/init-config-reloader/0.log"
Sep 15 11:00:38.103875 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:38.103798    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5d85778b9d-c2gkm_344ccc05-67d1-4e88-8750-288583c42a5b/monitoring-plugin/0.log"
Sep 15 11:00:38.225722 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:38.225696    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ftgzh_22a055ff-e4bf-49e1-aaaf-81b643287419/node-exporter/0.log"
Sep 15 11:00:38.245947 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:38.245920    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ftgzh_22a055ff-e4bf-49e1-aaaf-81b643287419/kube-rbac-proxy/0.log"
Sep 15 11:00:38.267291 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:38.267260    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ftgzh_22a055ff-e4bf-49e1-aaaf-81b643287419/init-textfile/0.log"
Sep 15 11:00:38.676730 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:38.676699    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-wkxvg_6124cb0d-6bc8-4be2-beaf-a215f84856cd/prometheus-operator/0.log"
Sep 15 11:00:38.694628 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:38.694598    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-wkxvg_6124cb0d-6bc8-4be2-beaf-a215f84856cd/kube-rbac-proxy/0.log"
Sep 15 11:00:38.718419 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:38.718329    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-tqvwm_a8ede461-d783-4166-a83f-22110b18dd42/prometheus-operator-admission-webhook/0.log"
Sep 15 11:00:38.760725 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:38.760695    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7d8f48c558-95vcf_dfa404fd-cf1f-498c-8767-14639fb2bb4d/telemeter-client/0.log"
Sep 15 11:00:38.780836 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:38.780807    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7d8f48c558-95vcf_dfa404fd-cf1f-498c-8767-14639fb2bb4d/reload/0.log"
Sep 15 11:00:38.801106 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:38.801080    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7d8f48c558-95vcf_dfa404fd-cf1f-498c-8767-14639fb2bb4d/kube-rbac-proxy/0.log"
Sep 15 11:00:38.830895 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:38.830868    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78bc77696b-9bk68_24e0b95f-40c1-44b4-9d78-63961811f3cf/thanos-query/0.log"
Sep 15 11:00:38.852062 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:38.852032    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78bc77696b-9bk68_24e0b95f-40c1-44b4-9d78-63961811f3cf/kube-rbac-proxy-web/0.log"
Sep 15 11:00:38.876129 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:38.876099    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78bc77696b-9bk68_24e0b95f-40c1-44b4-9d78-63961811f3cf/kube-rbac-proxy/0.log"
Sep 15 11:00:38.898468 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:38.898421    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78bc77696b-9bk68_24e0b95f-40c1-44b4-9d78-63961811f3cf/prom-label-proxy/0.log"
Sep 15 11:00:38.918302 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:38.918278    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78bc77696b-9bk68_24e0b95f-40c1-44b4-9d78-63961811f3cf/kube-rbac-proxy-rules/0.log"
Sep 15 11:00:38.941162 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:38.941137    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78bc77696b-9bk68_24e0b95f-40c1-44b4-9d78-63961811f3cf/kube-rbac-proxy-metrics/0.log"
Sep 15 11:00:41.629385 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:41.629349    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6d6995cf76-f9jf2_7fa13ed5-9d37-40e8-b621-62c60369ba9a/console/0.log"
Sep 15 11:00:42.273085 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:42.273045    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-2wphs/perf-node-gather-daemonset-m4mrw"]
Sep 15 11:00:42.273640 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:42.273616    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29bb1f2b-9b06-4d79-935a-c7a99dc43dd7" containerName="maas-api"
Sep 15 11:00:42.273788 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:42.273664    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="29bb1f2b-9b06-4d79-935a-c7a99dc43dd7" containerName="maas-api"
Sep 15 11:00:42.273788 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:42.273678    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffb8e359-aba1-4c2c-aca5-7634f3d23f0a" containerName="istio-proxy"
Sep 15 11:00:42.273788 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:42.273686    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffb8e359-aba1-4c2c-aca5-7634f3d23f0a" containerName="istio-proxy"
Sep 15 11:00:42.273962 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:42.273795    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="29bb1f2b-9b06-4d79-935a-c7a99dc43dd7" containerName="maas-api"
Sep 15 11:00:42.277330 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:42.277290    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2wphs/perf-node-gather-daemonset-m4mrw"
Sep 15 11:00:42.279818 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:42.279794    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2wphs\"/\"kube-root-ca.crt\""
Sep 15 11:00:42.280607 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:42.280580    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-2wphs\"/\"default-dockercfg-4b7wp\""
Sep 15 11:00:42.280719 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:42.280623    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2wphs\"/\"openshift-service-ca.crt\""
Sep 15 11:00:42.285156 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:42.285129    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2wphs/perf-node-gather-daemonset-m4mrw"]
Sep 15 11:00:42.427877 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:42.427834    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5308f236-7d4a-4d41-a7ab-a50fcdd0b5af-sys\") pod \"perf-node-gather-daemonset-m4mrw\" (UID: \"5308f236-7d4a-4d41-a7ab-a50fcdd0b5af\") " pod="openshift-must-gather-2wphs/perf-node-gather-daemonset-m4mrw"
Sep 15 11:00:42.427877 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:42.427875    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rz5tx\" (UniqueName: \"kubernetes.io/projected/5308f236-7d4a-4d41-a7ab-a50fcdd0b5af-kube-api-access-rz5tx\") pod \"perf-node-gather-daemonset-m4mrw\" (UID: \"5308f236-7d4a-4d41-a7ab-a50fcdd0b5af\") " pod="openshift-must-gather-2wphs/perf-node-gather-daemonset-m4mrw"
Sep 15 11:00:42.428116 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:42.427900    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5308f236-7d4a-4d41-a7ab-a50fcdd0b5af-proc\") pod \"perf-node-gather-daemonset-m4mrw\" (UID: \"5308f236-7d4a-4d41-a7ab-a50fcdd0b5af\") " pod="openshift-must-gather-2wphs/perf-node-gather-daemonset-m4mrw"
Sep 15 11:00:42.428116 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:42.427987    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5308f236-7d4a-4d41-a7ab-a50fcdd0b5af-lib-modules\") pod \"perf-node-gather-daemonset-m4mrw\" (UID: \"5308f236-7d4a-4d41-a7ab-a50fcdd0b5af\") " pod="openshift-must-gather-2wphs/perf-node-gather-daemonset-m4mrw"
Sep 15 11:00:42.428116 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:42.428083    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5308f236-7d4a-4d41-a7ab-a50fcdd0b5af-podres\") pod \"perf-node-gather-daemonset-m4mrw\" (UID: \"5308f236-7d4a-4d41-a7ab-a50fcdd0b5af\") " pod="openshift-must-gather-2wphs/perf-node-gather-daemonset-m4mrw"
Sep 15 11:00:42.529410 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:42.529296    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5308f236-7d4a-4d41-a7ab-a50fcdd0b5af-podres\") pod \"perf-node-gather-daemonset-m4mrw\" (UID: \"5308f236-7d4a-4d41-a7ab-a50fcdd0b5af\") " pod="openshift-must-gather-2wphs/perf-node-gather-daemonset-m4mrw"
Sep 15 11:00:42.529410 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:42.529387    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5308f236-7d4a-4d41-a7ab-a50fcdd0b5af-sys\") pod \"perf-node-gather-daemonset-m4mrw\" (UID: \"5308f236-7d4a-4d41-a7ab-a50fcdd0b5af\") " pod="openshift-must-gather-2wphs/perf-node-gather-daemonset-m4mrw"
Sep 15 11:00:42.529410 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:42.529407    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rz5tx\" (UniqueName: \"kubernetes.io/projected/5308f236-7d4a-4d41-a7ab-a50fcdd0b5af-kube-api-access-rz5tx\") pod \"perf-node-gather-daemonset-m4mrw\" (UID: \"5308f236-7d4a-4d41-a7ab-a50fcdd0b5af\") " pod="openshift-must-gather-2wphs/perf-node-gather-daemonset-m4mrw"
Sep 15 11:00:42.529684 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:42.529428    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5308f236-7d4a-4d41-a7ab-a50fcdd0b5af-proc\") pod \"perf-node-gather-daemonset-m4mrw\" (UID: \"5308f236-7d4a-4d41-a7ab-a50fcdd0b5af\") " pod="openshift-must-gather-2wphs/perf-node-gather-daemonset-m4mrw"
Sep 15 11:00:42.529684 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:42.529484    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5308f236-7d4a-4d41-a7ab-a50fcdd0b5af-lib-modules\") pod \"perf-node-gather-daemonset-m4mrw\" (UID: \"5308f236-7d4a-4d41-a7ab-a50fcdd0b5af\") " pod="openshift-must-gather-2wphs/perf-node-gather-daemonset-m4mrw"
Sep 15 11:00:42.529684 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:42.529496    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5308f236-7d4a-4d41-a7ab-a50fcdd0b5af-podres\") pod \"perf-node-gather-daemonset-m4mrw\" (UID: \"5308f236-7d4a-4d41-a7ab-a50fcdd0b5af\") " pod="openshift-must-gather-2wphs/perf-node-gather-daemonset-m4mrw"
Sep 15 11:00:42.529684 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:42.529515    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5308f236-7d4a-4d41-a7ab-a50fcdd0b5af-sys\") pod \"perf-node-gather-daemonset-m4mrw\" (UID: \"5308f236-7d4a-4d41-a7ab-a50fcdd0b5af\") " pod="openshift-must-gather-2wphs/perf-node-gather-daemonset-m4mrw"
Sep 15 11:00:42.529684 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:42.529595    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5308f236-7d4a-4d41-a7ab-a50fcdd0b5af-proc\") pod \"perf-node-gather-daemonset-m4mrw\" (UID: \"5308f236-7d4a-4d41-a7ab-a50fcdd0b5af\") " pod="openshift-must-gather-2wphs/perf-node-gather-daemonset-m4mrw"
Sep 15 11:00:42.529870 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:42.529686    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5308f236-7d4a-4d41-a7ab-a50fcdd0b5af-lib-modules\") pod \"perf-node-gather-daemonset-m4mrw\" (UID: \"5308f236-7d4a-4d41-a7ab-a50fcdd0b5af\") " pod="openshift-must-gather-2wphs/perf-node-gather-daemonset-m4mrw"
Sep 15 11:00:42.537715 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:42.537685    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rz5tx\" (UniqueName: \"kubernetes.io/projected/5308f236-7d4a-4d41-a7ab-a50fcdd0b5af-kube-api-access-rz5tx\") pod \"perf-node-gather-daemonset-m4mrw\" (UID: \"5308f236-7d4a-4d41-a7ab-a50fcdd0b5af\") " pod="openshift-must-gather-2wphs/perf-node-gather-daemonset-m4mrw"
Sep 15 11:00:42.589759 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:42.589710    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2wphs/perf-node-gather-daemonset-m4mrw"
Sep 15 11:00:42.862121 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:42.862087    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2wphs/perf-node-gather-daemonset-m4mrw"]
Sep 15 11:00:42.863690 ip-10-0-141-235 kubenswrapper[2583]: W0915 11:00:42.863661    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5308f236_7d4a_4d41_a7ab_a50fcdd0b5af.slice/crio-67f6496d169a2b7a9346154ed1308d2d974b43e26a38143ffde180ec843ec2ec WatchSource:0}: Error finding container 67f6496d169a2b7a9346154ed1308d2d974b43e26a38143ffde180ec843ec2ec: Status 404 returned error can't find the container with id 67f6496d169a2b7a9346154ed1308d2d974b43e26a38143ffde180ec843ec2ec
Sep 15 11:00:43.802720 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:43.802686    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-k2zv8_c724eb6e-3079-4433-993e-02a515bb2132/dns/0.log"
Sep 15 11:00:43.822595 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:43.822559    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2wphs/perf-node-gather-daemonset-m4mrw" event={"ID":"5308f236-7d4a-4d41-a7ab-a50fcdd0b5af","Type":"ContainerStarted","Data":"b2b1b33650201cda46e907430c61c8f4f76628bc5b273935e8ee5c45ca702f60"}
Sep 15 11:00:43.822787 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:43.822622    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2wphs/perf-node-gather-daemonset-m4mrw" event={"ID":"5308f236-7d4a-4d41-a7ab-a50fcdd0b5af","Type":"ContainerStarted","Data":"67f6496d169a2b7a9346154ed1308d2d974b43e26a38143ffde180ec843ec2ec"}
Sep 15 11:00:43.822787 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:43.822731    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-2wphs/perf-node-gather-daemonset-m4mrw"
Sep 15 11:00:43.822934 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:43.822915    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-k2zv8_c724eb6e-3079-4433-993e-02a515bb2132/kube-rbac-proxy/0.log"
Sep 15 11:00:43.838620 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:43.838571    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-2wphs/perf-node-gather-daemonset-m4mrw" podStartSLOduration=1.8385540219999998 podStartE2EDuration="1.838554022s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-15 11:00:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-15 11:00:43.836363293 +0000 UTC m=+2559.083330262" watchObservedRunningTime="2026-09-15 11:00:43.838554022 +0000 UTC m=+2559.085520991"
Sep 15 11:00:43.849390 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:43.849365    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mw292_88f8318b-0faa-4708-afc6-de7f7ae3cee3/dns-node-resolver/0.log"
Sep 15 11:00:44.474771 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:44.474740    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5gw48_91c14855-f9ab-4882-8185-8289a2bd8269/node-ca/0.log"
Sep 15 11:00:45.486240 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:45.486211    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cp69vw_585f0d63-e037-4387-b8dd-e4530c18c319/istio-proxy/0.log"
Sep 15 11:00:45.950989 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:45.950953    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-24mmv_8106e39c-4f35-4890-91eb-1204ca4ccc4b/discovery/0.log"
Sep 15 11:00:47.072615 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:47.072573    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lr47g_55758f39-d31a-4b0c-bb35-ef1f3be7d25b/serve-healthcheck-canary/0.log"
Sep 15 11:00:47.561802 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:47.561763    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2cl9v_6a655087-f02c-416c-87d2-7aba0811f5c0/kube-rbac-proxy/0.log"
Sep 15 11:00:47.586764 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:47.586732    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2cl9v_6a655087-f02c-416c-87d2-7aba0811f5c0/exporter/0.log"
Sep 15 11:00:47.608130 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:47.608102    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2cl9v_6a655087-f02c-416c-87d2-7aba0811f5c0/extractor/0.log"
Sep 15 11:00:49.839203 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:49.839169    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-2wphs/perf-node-gather-daemonset-m4mrw"
Sep 15 11:00:50.210644 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:50.210616    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-xvnnk_71e3decd-7333-4ed5-82cf-e82de3601c18/server/0.log"
Sep 15 11:00:50.296147 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:50.296112    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-bbfhr_e3b4daf0-82f9-4aef-801d-a9ab121f984c/manager/0.log"
Sep 15 11:00:50.452815 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:50.452789    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6699d7bd9c-bnltb_105dfa5e-4c1b-4913-bf15-7cda5d01f89b/manager/0.log"
Sep 15 11:00:50.471610 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:50.471522    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6699d7bd9c-bnltb_105dfa5e-4c1b-4913-bf15-7cda5d01f89b/copy-manifests/0.log"
Sep 15 11:00:51.946479 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:51.946389    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-85d6978c87-9pkz6_eb75d5b6-1850-4fc3-9220-aa6d5d107380/manager/0.log"
Sep 15 11:00:51.973480 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:51.973435    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-g77dt_6d971aa8-dd09-44d1-a61e-d89311b49f2f/openshift-lws-operator/0.log"
Sep 15 11:00:52.298979 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:52.298905    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cp69vw_585f0d63-e037-4387-b8dd-e4530c18c319/istio-proxy/0.log"
Sep 15 11:00:52.759691 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:52.759660    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-24mmv_8106e39c-4f35-4890-91eb-1204ca4ccc4b/discovery/0.log"
Sep 15 11:00:54.432144 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:54.432112    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cp69vw_585f0d63-e037-4387-b8dd-e4530c18c319/istio-proxy/0.log"
Sep 15 11:00:54.878629 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:54.878537    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-24mmv_8106e39c-4f35-4890-91eb-1204ca4ccc4b/discovery/0.log"
Sep 15 11:00:58.544458 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:58.544413    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-58dk4_b64b5ead-1ece-4aff-8975-5c59302a42d6/kube-multus-additional-cni-plugins/0.log"
Sep 15 11:00:58.565809 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:58.565686    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-58dk4_b64b5ead-1ece-4aff-8975-5c59302a42d6/egress-router-binary-copy/0.log"
Sep 15 11:00:58.588405 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:58.588380    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-58dk4_b64b5ead-1ece-4aff-8975-5c59302a42d6/cni-plugins/0.log"
Sep 15 11:00:58.610341 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:58.610307    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-58dk4_b64b5ead-1ece-4aff-8975-5c59302a42d6/bond-cni-plugin/0.log"
Sep 15 11:00:58.630718 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:58.630690    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-58dk4_b64b5ead-1ece-4aff-8975-5c59302a42d6/routeoverride-cni/0.log"
Sep 15 11:00:58.652269 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:58.652223    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-58dk4_b64b5ead-1ece-4aff-8975-5c59302a42d6/whereabouts-cni-bincopy/0.log"
Sep 15 11:00:58.672303 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:58.672275    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-58dk4_b64b5ead-1ece-4aff-8975-5c59302a42d6/whereabouts-cni/0.log"
Sep 15 11:00:59.068861 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:59.068822    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-n5sdr_572e0352-78c0-4269-a50f-4d6e86d06f46/kube-multus/0.log"
Sep 15 11:00:59.171452 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:59.171412    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-gsxvv_c5d3dc08-8291-4828-87d1-f0da01fd24c8/network-metrics-daemon/0.log"
Sep 15 11:00:59.190093 ip-10-0-141-235 kubenswrapper[2583]: I0915 11:00:59.190065    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-gsxvv_c5d3dc08-8291-4828-87d1-f0da01fd24c8/kube-rbac-proxy/0.log"