Jul 29 13:59:18.987899 ip-10-0-131-187 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 29 13:59:18.987910 ip-10-0-131-187 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 29 13:59:18.987917 ip-10-0-131-187 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 29 13:59:18.988238 ip-10-0-131-187 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 29 13:59:29.219980 ip-10-0-131-187 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 29 13:59:29.219998 ip-10-0-131-187 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot dcb24302752c49c0ad140b3ea7f96e4b --
Jul 29 14:01:08.419972 ip-10-0-131-187 systemd[1]: Starting Kubernetes Kubelet...
Jul 29 14:01:08.879573 ip-10-0-131-187 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 14:01:08.879573 ip-10-0-131-187 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 29 14:01:08.879573 ip-10-0-131-187 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 14:01:08.879573 ip-10-0-131-187 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 29 14:01:08.879573 ip-10-0-131-187 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 14:01:08.880253 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.879639    2559 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 29 14:01:08.884347 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.883940    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 14:01:08.884347 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.883964    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 14:01:08.884347 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.883976    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 14:01:08.884347 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.883981    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 14:01:08.884347 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.883985    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 14:01:08.884347 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.883990    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 14:01:08.884347 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.883995    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 14:01:08.884347 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.883999    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 14:01:08.884347 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884004    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 14:01:08.884347 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884008    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 14:01:08.884347 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884013    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 14:01:08.884347 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884017    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 14:01:08.884347 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884021    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 14:01:08.884347 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884026    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 14:01:08.884347 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884035    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 14:01:08.884347 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884041    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 14:01:08.884347 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884045    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 14:01:08.884347 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884050    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 14:01:08.884347 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884055    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 14:01:08.884347 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884059    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 14:01:08.884958 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884064    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 14:01:08.884958 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884075    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 14:01:08.884958 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884080    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 14:01:08.884958 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884084    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 14:01:08.884958 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884088    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 14:01:08.884958 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884093    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 14:01:08.884958 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884097    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 14:01:08.884958 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884106    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 14:01:08.884958 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884110    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 14:01:08.884958 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884114    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 14:01:08.884958 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884118    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 14:01:08.884958 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884125    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 14:01:08.884958 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884131    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 14:01:08.884958 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884138    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 14:01:08.884958 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884143    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 14:01:08.884958 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884148    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 14:01:08.884958 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884153    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 14:01:08.884958 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884158    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 14:01:08.884958 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884163    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 14:01:08.884958 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884173    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 14:01:08.885439 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884177    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 14:01:08.885439 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884182    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 14:01:08.885439 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884186    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 14:01:08.885439 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884191    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 14:01:08.885439 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884195    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 14:01:08.885439 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884199    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 14:01:08.885439 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884204    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 14:01:08.885439 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884208    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 14:01:08.885439 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884213    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 14:01:08.885439 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884218    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 14:01:08.885439 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884226    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 14:01:08.885439 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884235    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 14:01:08.885439 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884240    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 14:01:08.885439 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884244    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 14:01:08.885439 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884248    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 14:01:08.885439 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884252    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 14:01:08.885439 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884257    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 14:01:08.885439 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884262    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 14:01:08.885439 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884267    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 14:01:08.885963 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884271    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 14:01:08.885963 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884275    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 14:01:08.885963 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884280    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 14:01:08.885963 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884284    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 14:01:08.885963 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884288    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 14:01:08.885963 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884297    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 14:01:08.885963 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884302    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 14:01:08.885963 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884307    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 14:01:08.885963 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884311    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 14:01:08.885963 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884316    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 14:01:08.885963 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884321    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 14:01:08.885963 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884325    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 14:01:08.885963 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884329    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 14:01:08.885963 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884334    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 14:01:08.885963 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884338    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 14:01:08.885963 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884342    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 14:01:08.885963 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884347    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 14:01:08.885963 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884394    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 14:01:08.885963 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884422    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 14:01:08.885963 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884429    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 14:01:08.886436 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884448    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 14:01:08.886436 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884453    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 14:01:08.886436 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884458    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 14:01:08.886436 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884470    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 14:01:08.886436 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884480    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 14:01:08.886436 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884486    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 14:01:08.886436 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884505    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 14:01:08.886436 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884508    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 14:01:08.886436 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884616    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 14:01:08.886436 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884621    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 14:01:08.886436 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884624    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 14:01:08.886436 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884628    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 14:01:08.886436 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884631    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 14:01:08.886436 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884634    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 14:01:08.886436 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884637    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 14:01:08.886436 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884640    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 14:01:08.886436 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884643    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 14:01:08.886436 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884647    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 14:01:08.886436 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884650    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 14:01:08.886907 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.884653    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 14:01:08.886907 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885404    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 14:01:08.886907 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885412    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 14:01:08.886907 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885416    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 14:01:08.886907 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885419    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 14:01:08.886907 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885421    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 14:01:08.886907 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885424    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 14:01:08.886907 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885427    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 14:01:08.886907 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885430    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 14:01:08.886907 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885433    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 14:01:08.886907 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885435    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 14:01:08.886907 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885438    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 14:01:08.886907 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885441    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 14:01:08.886907 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885444    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 14:01:08.886907 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885447    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 14:01:08.886907 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885449    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 14:01:08.886907 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885452    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 14:01:08.886907 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885455    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 14:01:08.886907 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885457    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 14:01:08.886907 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885460    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 14:01:08.887399 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885463    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 14:01:08.887399 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885465    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 14:01:08.887399 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885468    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 14:01:08.887399 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885470    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 14:01:08.887399 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885473    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 14:01:08.887399 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885476    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 14:01:08.887399 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885478    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 14:01:08.887399 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885481    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 14:01:08.887399 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885483    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 14:01:08.887399 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885486    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 14:01:08.887399 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885489    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 14:01:08.887399 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885491    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 14:01:08.887399 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885494    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 14:01:08.887399 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885498    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 14:01:08.887399 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885503    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 14:01:08.887399 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885507    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 14:01:08.887399 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885511    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 14:01:08.887399 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885514    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 14:01:08.887399 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885517    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 14:01:08.887399 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885519    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 14:01:08.887898 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885522    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 14:01:08.887898 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885525    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 14:01:08.887898 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885527    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 14:01:08.887898 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885530    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 14:01:08.887898 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885532    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 14:01:08.887898 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885535    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 14:01:08.887898 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885537    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 14:01:08.887898 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885540    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 14:01:08.887898 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885542    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 14:01:08.887898 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885545    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 14:01:08.887898 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885547    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 14:01:08.887898 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885550    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 14:01:08.887898 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885552    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 14:01:08.887898 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885554    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 14:01:08.887898 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885557    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 14:01:08.887898 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885560    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 14:01:08.887898 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885562    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 14:01:08.887898 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885565    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 14:01:08.887898 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885568    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 14:01:08.887898 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885570    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 14:01:08.888391 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885573    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 14:01:08.888391 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885576    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 14:01:08.888391 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885579    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 14:01:08.888391 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885582    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 14:01:08.888391 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885585    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 14:01:08.888391 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885588    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 14:01:08.888391 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885591    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 14:01:08.888391 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885593    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 14:01:08.888391 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885595    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 14:01:08.888391 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885598    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 14:01:08.888391 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885602    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 14:01:08.888391 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885605    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 14:01:08.888391 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885608    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 14:01:08.888391 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885611    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 14:01:08.888391 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885613    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 14:01:08.888391 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885616    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 14:01:08.888391 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885619    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 14:01:08.888391 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885621    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 14:01:08.888391 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885624    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 14:01:08.888871 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885626    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 14:01:08.888871 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885629    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 14:01:08.888871 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885631    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 14:01:08.888871 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885634    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 14:01:08.888871 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885642    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 14:01:08.888871 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885645    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 14:01:08.888871 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885648    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 14:01:08.888871 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885650    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 14:01:08.888871 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885653    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 14:01:08.888871 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885657    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 14:01:08.888871 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885661    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 14:01:08.888871 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885664    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 14:01:08.888871 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885666    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 14:01:08.888871 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885669    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 14:01:08.888871 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885671    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 14:01:08.888871 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885674    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 14:01:08.888871 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885677    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 14:01:08.888871 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885679    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 14:01:08.888871 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885682    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 14:01:08.889320 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885705    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 14:01:08.889320 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.885709    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 14:01:08.889320 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886094    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 14:01:08.889320 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886099    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 14:01:08.889320 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886102    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 14:01:08.889320 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886105    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 14:01:08.889320 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886108    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 14:01:08.889320 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886110    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 14:01:08.889320 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886113    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 14:01:08.889320 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886116    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 14:01:08.889320 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886119    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 14:01:08.889320 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886121    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 14:01:08.889320 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886124    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 14:01:08.889320 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886127    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 14:01:08.889320 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886130    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 14:01:08.889320 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886132    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 14:01:08.889320 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886135    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 14:01:08.889320 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886137    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 14:01:08.889320 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886141    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 14:01:08.889320 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886144    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 14:01:08.889818 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886146    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 14:01:08.889818 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886149    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 14:01:08.889818 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886152    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 14:01:08.889818 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886154    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 14:01:08.889818 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886158    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 14:01:08.889818 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886162    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 14:01:08.889818 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886165    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 14:01:08.889818 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886169    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 14:01:08.889818 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886172    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 14:01:08.889818 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886175    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 14:01:08.889818 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886178    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 14:01:08.889818 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886183    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 14:01:08.889818 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886186    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 14:01:08.889818 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886188    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 14:01:08.889818 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886191    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 14:01:08.889818 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886194    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 14:01:08.889818 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886196    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 14:01:08.889818 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886199    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 14:01:08.889818 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886202    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 14:01:08.890332 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886204    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 14:01:08.890332 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886206    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 14:01:08.890332 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886209    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 14:01:08.890332 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886212    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 14:01:08.890332 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886214    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 14:01:08.890332 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886217    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 14:01:08.890332 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886219    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 14:01:08.890332 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886222    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 14:01:08.890332 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886225    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 14:01:08.890332 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886227    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 14:01:08.890332 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886230    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 14:01:08.890332 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886233    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 14:01:08.890332 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886236    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 14:01:08.890332 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886238    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 14:01:08.890332 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886241    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 14:01:08.890332 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886243    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 14:01:08.890332 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886246    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 14:01:08.890332 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886248    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 14:01:08.890332 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886251    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 14:01:08.890332 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886253    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 14:01:08.890845 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886256    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 14:01:08.890845 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886259    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 14:01:08.890845 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886261    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 14:01:08.890845 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886264    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 14:01:08.890845 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886268    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 14:01:08.890845 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886270    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 14:01:08.890845 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886273    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 14:01:08.890845 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886275    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 14:01:08.890845 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886278    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 14:01:08.890845 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886280    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 14:01:08.890845 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886283    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 14:01:08.890845 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886285    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 14:01:08.890845 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886288    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 14:01:08.890845 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886290    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 14:01:08.890845 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886293    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 14:01:08.890845 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886295    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 14:01:08.890845 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886297    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 14:01:08.890845 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886301    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 14:01:08.890845 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886303    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 14:01:08.890845 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886305    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 14:01:08.891321 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886308    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 14:01:08.891321 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886310    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 14:01:08.891321 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886313    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 14:01:08.891321 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886315    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 14:01:08.891321 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886317    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 14:01:08.891321 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886320    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 14:01:08.891321 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886323    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 14:01:08.891321 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886325    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 14:01:08.891321 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886327    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 14:01:08.891321 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886330    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 14:01:08.891321 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886332    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 14:01:08.891321 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886335    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 14:01:08.891321 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886337    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 14:01:08.891321 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886339    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 14:01:08.891321 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886344    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 14:01:08.891321 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886346    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 14:01:08.891321 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886351    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 14:01:08.891321 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886355    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 14:01:08.891321 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886357    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 14:01:08.891321 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886360    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 14:01:08.891805 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886362    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 14:01:08.891805 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.886365    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 14:01:08.891805 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886434    2559 flags.go:64] FLAG: --address="0.0.0.0"
Jul 29 14:01:08.891805 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886445    2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 29 14:01:08.891805 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886451    2559 flags.go:64] FLAG: --anonymous-auth="true"
Jul 29 14:01:08.891805 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886455    2559 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 29 14:01:08.891805 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886459    2559 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 29 14:01:08.891805 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886463    2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 29 14:01:08.891805 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886467    2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 29 14:01:08.891805 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886471    2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 29 14:01:08.891805 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886474    2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 29 14:01:08.891805 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886477    2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 29 14:01:08.891805 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886480    2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 29 14:01:08.891805 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886483    2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 29 14:01:08.891805 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886486    2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 29 14:01:08.891805 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886489    2559 flags.go:64] FLAG: --cgroup-root=""
Jul 29 14:01:08.891805 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886492    2559 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 29 14:01:08.891805 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886495    2559 flags.go:64] FLAG: --client-ca-file=""
Jul 29 14:01:08.891805 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886499    2559 flags.go:64] FLAG: --cloud-provider="external"
Jul 29 14:01:08.891805 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886502    2559 flags.go:64] FLAG: --cluster-dns="[]"
Jul 29 14:01:08.891805 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886507    2559 flags.go:64] FLAG: --cluster-domain=""
Jul 29 14:01:08.891805 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886510    2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 29 14:01:08.891805 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886513    2559 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 29 14:01:08.891805 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886516    2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 29 14:01:08.892369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886520    2559 flags.go:64] FLAG: --container-log-max-files="5"
Jul 29 14:01:08.892369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886523    2559 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 29 14:01:08.892369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886527    2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 29 14:01:08.892369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886530    2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 29 14:01:08.892369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886536    2559 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 29 14:01:08.892369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886539    2559 flags.go:64] FLAG: --contention-profiling="false"
Jul 29 14:01:08.892369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886542    2559 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 29 14:01:08.892369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886545    2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 29 14:01:08.892369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886548    2559 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 29 14:01:08.892369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886551    2559 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 29 14:01:08.892369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886555    2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 29 14:01:08.892369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886558    2559 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 29 14:01:08.892369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886561    2559 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 29 14:01:08.892369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886564    2559 flags.go:64] FLAG: --enable-load-reader="false"
Jul 29 14:01:08.892369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886567    2559 flags.go:64] FLAG: --enable-server="true"
Jul 29 14:01:08.892369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886570    2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 29 14:01:08.892369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886574    2559 flags.go:64] FLAG: --event-burst="100"
Jul 29 14:01:08.892369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886577    2559 flags.go:64] FLAG: --event-qps="50"
Jul 29 14:01:08.892369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886580    2559 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 29 14:01:08.892369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886583    2559 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 29 14:01:08.892369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886586    2559 flags.go:64] FLAG: --eviction-hard=""
Jul 29 14:01:08.892369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886589    2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 29 14:01:08.892369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886592    2559 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 29 14:01:08.892369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886595    2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 29 14:01:08.892369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886598    2559 flags.go:64] FLAG: --eviction-soft=""
Jul 29 14:01:08.893079 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886601    2559 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 29 14:01:08.893079 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886603    2559 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 29 14:01:08.893079 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886607    2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 29 14:01:08.893079 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886610    2559 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 29 14:01:08.893079 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886612    2559 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 29 14:01:08.893079 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886615    2559 flags.go:64] FLAG: --fail-swap-on="true"
Jul 29 14:01:08.893079 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886618    2559 flags.go:64] FLAG: --feature-gates=""
Jul 29 14:01:08.893079 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886621    2559 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 29 14:01:08.893079 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886624    2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 29 14:01:08.893079 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886628    2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 29 14:01:08.893079 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886631    2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 29 14:01:08.893079 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886637    2559 flags.go:64] FLAG: --healthz-port="10248"
Jul 29 14:01:08.893079 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886640    2559 flags.go:64] FLAG: --help="false"
Jul 29 14:01:08.893079 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886643    2559 flags.go:64] FLAG: --hostname-override="ip-10-0-131-187.ec2.internal"
Jul 29 14:01:08.893079 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886646    2559 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 29 14:01:08.893079 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886649    2559 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 29 14:01:08.893079 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886652    2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 29 14:01:08.893079 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886655    2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 29 14:01:08.893079 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886659    2559 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 29 14:01:08.893079 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886662    2559 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 29 14:01:08.893079 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886664    2559 flags.go:64] FLAG: --image-service-endpoint=""
Jul 29 14:01:08.893079 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886667    2559 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 29 14:01:08.893079 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886670    2559 flags.go:64] FLAG: --kube-api-burst="100"
Jul 29 14:01:08.893663 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886673    2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 29 14:01:08.893663 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886676    2559 flags.go:64] FLAG: --kube-api-qps="50"
Jul 29 14:01:08.893663 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886679    2559 flags.go:64] FLAG: --kube-reserved=""
Jul 29 14:01:08.893663 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886682    2559 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 29 14:01:08.893663 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886698    2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 29 14:01:08.893663 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886703    2559 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 29 14:01:08.893663 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886708    2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 29 14:01:08.893663 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886711    2559 flags.go:64] FLAG: --lock-file=""
Jul 29 14:01:08.893663 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886714    2559 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 29 14:01:08.893663 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886717    2559 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 29 14:01:08.893663 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886720    2559 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 29 14:01:08.893663 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886726    2559 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 29 14:01:08.893663 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886730    2559 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 29 14:01:08.893663 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886733    2559 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 29 14:01:08.893663 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886736    2559 flags.go:64] FLAG: --logging-format="text"
Jul 29 14:01:08.893663 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886739    2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 29 14:01:08.893663 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886742    2559 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 29 14:01:08.893663 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886745    2559 flags.go:64] FLAG: --manifest-url=""
Jul 29 14:01:08.893663 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886749    2559 flags.go:64] FLAG: --manifest-url-header=""
Jul 29 14:01:08.893663 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886753    2559 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 29 14:01:08.893663 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886758    2559 flags.go:64] FLAG: --max-open-files="1000000"
Jul 29 14:01:08.893663 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886762    2559 flags.go:64] FLAG: --max-pods="110"
Jul 29 14:01:08.893663 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886765    2559 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 29 14:01:08.893663 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886768    2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 29 14:01:08.893663 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886771    2559 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 29 14:01:08.894315 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886774    2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 29 14:01:08.894315 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886777    2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 29 14:01:08.894315 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886780    2559 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 29 14:01:08.894315 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886783    2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 29 14:01:08.894315 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886790    2559 flags.go:64] FLAG: --node-status-max-images="50"
Jul 29 14:01:08.894315 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886793    2559 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 29 14:01:08.894315 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886796    2559 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 29 14:01:08.894315 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886799    2559 flags.go:64] FLAG: --pod-cidr=""
Jul 29 14:01:08.894315 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886801    2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 29 14:01:08.894315 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886806    2559 flags.go:64] FLAG: --pod-manifest-path=""
Jul 29 14:01:08.894315 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886809    2559 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 29 14:01:08.894315 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886812    2559 flags.go:64] FLAG: --pods-per-core="0"
Jul 29 14:01:08.894315 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886815    2559 flags.go:64] FLAG: --port="10250"
Jul 29 14:01:08.894315 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886818    2559 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 29 14:01:08.894315 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886821    2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02ca744fe20db5f5b"
Jul 29 14:01:08.894315 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886824    2559 flags.go:64] FLAG: --qos-reserved=""
Jul 29 14:01:08.894315 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886827    2559 flags.go:64] FLAG: --read-only-port="10255"
Jul 29 14:01:08.894315 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886830    2559 flags.go:64] FLAG: --register-node="true"
Jul 29 14:01:08.894315 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886833    2559 flags.go:64] FLAG: --register-with-taints=""
Jul 29 14:01:08.894315 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886836    2559 flags.go:64] FLAG: --registry-burst="10"
Jul 29 14:01:08.894315 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886840    2559 flags.go:64] FLAG: --registry-qps="5"
Jul 29 14:01:08.894315 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886843    2559 flags.go:64] FLAG: --reserved-cpus=""
Jul 29 14:01:08.894315 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886846    2559 flags.go:64] FLAG: --reserved-memory=""
Jul 29 14:01:08.894315 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886849    2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 29 14:01:08.894315 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886852    2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 29 14:01:08.894938 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886855    2559 flags.go:64] FLAG: --rotate-certificates="false"
Jul 29 14:01:08.894938 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886859    2559 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 29 14:01:08.894938 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886863    2559 flags.go:64] FLAG: --runonce="false"
Jul 29 14:01:08.894938 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886866    2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 29 14:01:08.894938 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886869    2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 29 14:01:08.894938 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886872    2559 flags.go:64] FLAG: --seccomp-default="false"
Jul 29 14:01:08.894938 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886875    2559 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 29 14:01:08.894938 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886878    2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 29 14:01:08.894938 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886881    2559 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 29 14:01:08.894938 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886884    2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 29 14:01:08.894938 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886887    2559 flags.go:64] FLAG: --storage-driver-password="root"
Jul 29 14:01:08.894938 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886890    2559 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 29 14:01:08.894938 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886893    2559 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 29 14:01:08.894938 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886896    2559 flags.go:64] FLAG: --storage-driver-user="root"
Jul 29 14:01:08.894938 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886898    2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 29 14:01:08.894938 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886901    2559 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 29 14:01:08.894938 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886904    2559 flags.go:64] FLAG: --system-cgroups=""
Jul 29 14:01:08.894938 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886907    2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 29 14:01:08.894938 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886912    2559 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 29 14:01:08.894938 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886915    2559 flags.go:64] FLAG: --tls-cert-file=""
Jul 29 14:01:08.894938 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886918    2559 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 29 14:01:08.894938 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886923    2559 flags.go:64] FLAG: --tls-min-version=""
Jul 29 14:01:08.894938 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886926    2559 flags.go:64] FLAG: --tls-private-key-file=""
Jul 29 14:01:08.894938 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886929    2559 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 29 14:01:08.894938 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886931    2559 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 29 14:01:08.895537 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886934    2559 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 29 14:01:08.895537 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886937    2559 flags.go:64] FLAG: --v="2"
Jul 29 14:01:08.895537 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886943    2559 flags.go:64] FLAG: --version="false"
Jul 29 14:01:08.895537 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886948    2559 flags.go:64] FLAG: --vmodule=""
Jul 29 14:01:08.895537 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886952    2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 29 14:01:08.895537 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.886955    2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 29 14:01:08.895537 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887064    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 14:01:08.895537 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887067    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 14:01:08.895537 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887072    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 14:01:08.895537 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887076    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 14:01:08.895537 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887078    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 14:01:08.895537 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887081    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 14:01:08.895537 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887084    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 14:01:08.895537 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887086    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 14:01:08.895537 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887089    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 14:01:08.895537 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887091    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 14:01:08.895537 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887094    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 14:01:08.895537 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887096    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 14:01:08.895537 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887099    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 14:01:08.895537 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887101    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 14:01:08.895537 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887106    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 14:01:08.896177 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887110    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 14:01:08.896177 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887113    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 14:01:08.896177 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887116    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 14:01:08.896177 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887120    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 14:01:08.896177 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887124    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 14:01:08.896177 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887127    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 14:01:08.896177 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887130    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 14:01:08.896177 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887133    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 14:01:08.896177 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887135    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 14:01:08.896177 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887138    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 14:01:08.896177 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887140    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 14:01:08.896177 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887143    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 14:01:08.896177 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887145    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 14:01:08.896177 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887148    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 14:01:08.896177 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887150    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 14:01:08.896177 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887160    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 14:01:08.896177 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887162    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 14:01:08.896177 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887165    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 14:01:08.896177 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887167    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 14:01:08.896638 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887171    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 14:01:08.896638 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887175    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 14:01:08.896638 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887178    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 14:01:08.896638 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887180    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 14:01:08.896638 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887183    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 14:01:08.896638 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887186    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 14:01:08.896638 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887188    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 14:01:08.896638 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887191    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 14:01:08.896638 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887193    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 14:01:08.896638 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887196    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 14:01:08.896638 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887198    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 14:01:08.896638 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887201    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 14:01:08.896638 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887203    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 14:01:08.896638 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887207    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 14:01:08.896638 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887210    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 14:01:08.896638 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887212    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 14:01:08.896638 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887214    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 14:01:08.896638 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887217    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 14:01:08.896638 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887220    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 14:01:08.896638 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887222    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 14:01:08.897125 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887225    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 14:01:08.897125 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887227    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 14:01:08.897125 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887229    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 14:01:08.897125 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887232    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 14:01:08.897125 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887234    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 14:01:08.897125 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887237    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 14:01:08.897125 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887240    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 14:01:08.897125 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887242    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 14:01:08.897125 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887245    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 14:01:08.897125 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887248    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 14:01:08.897125 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887250    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 14:01:08.897125 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887252    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 14:01:08.897125 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887256    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 14:01:08.897125 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887261    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 14:01:08.897125 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887263    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 14:01:08.897125 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887266    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 14:01:08.897125 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887269    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 14:01:08.897125 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887271    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 14:01:08.897125 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887274    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 14:01:08.897125 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887276    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 14:01:08.897610 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887279    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 14:01:08.897610 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887281    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 14:01:08.897610 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887284    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 14:01:08.897610 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887286    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 14:01:08.897610 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887289    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 14:01:08.897610 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887291    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 14:01:08.897610 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887294    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 14:01:08.897610 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887297    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 14:01:08.897610 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887299    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 14:01:08.897610 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887302    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 14:01:08.897610 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887305    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 14:01:08.897610 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887307    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 14:01:08.897610 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887310    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 14:01:08.897610 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887312    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 14:01:08.897610 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887315    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 14:01:08.897610 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887317    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 14:01:08.897610 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887320    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 14:01:08.897610 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887322    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 14:01:08.897610 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887324    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 14:01:08.897610 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887327    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 14:01:08.898127 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887329    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 14:01:08.898127 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887333    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 14:01:08.898127 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887337    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 14:01:08.898127 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887340    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 14:01:08.898127 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.887344    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 14:01:08.898127 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.887353    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 14:01:08.898127 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.894578    2559 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 29 14:01:08.898127 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.894593    2559 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 29 14:01:08.898127 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.894607    2559 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 29 14:01:08.898127 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.894613    2559 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 29 14:01:08.898127 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894666    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 14:01:08.898127 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894671    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 14:01:08.898127 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894674    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 14:01:08.898127 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894677    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 14:01:08.898473 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894679    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 14:01:08.898473 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894682    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 14:01:08.898473 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894700    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 14:01:08.898473 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894703    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 14:01:08.898473 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894706    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 14:01:08.898473 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894709    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 14:01:08.898473 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894711    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 14:01:08.898473 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894714    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 14:01:08.898473 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894717    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 14:01:08.898473 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894720    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 14:01:08.898473 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894722    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 14:01:08.898473 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894725    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 14:01:08.898473 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894727    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 14:01:08.898473 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894730    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 14:01:08.898473 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894732    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 14:01:08.898473 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894735    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 14:01:08.898473 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894737    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 14:01:08.898473 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894740    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 14:01:08.898473 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894742    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 14:01:08.898473 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894745    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 14:01:08.898968 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894748    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 14:01:08.898968 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894750    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 14:01:08.898968 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894752    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 14:01:08.898968 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894755    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 14:01:08.898968 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894758    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 14:01:08.898968 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894762    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 14:01:08.898968 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894765    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 14:01:08.898968 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894768    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 14:01:08.898968 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894771    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 14:01:08.898968 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894774    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 14:01:08.898968 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894776    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 14:01:08.898968 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894779    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 14:01:08.898968 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894782    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 14:01:08.898968 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894784    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 14:01:08.898968 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894787    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 14:01:08.898968 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894790    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 14:01:08.898968 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894792    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 14:01:08.898968 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894795    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 14:01:08.898968 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894797    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 14:01:08.898968 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894800    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 14:01:08.899497 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894802    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 14:01:08.899497 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894805    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 14:01:08.899497 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894807    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 14:01:08.899497 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894810    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 14:01:08.899497 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894813    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 14:01:08.899497 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894815    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 14:01:08.899497 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894818    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 14:01:08.899497 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894820    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 14:01:08.899497 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894824    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 14:01:08.899497 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894828    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 14:01:08.899497 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894831    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 14:01:08.899497 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894834    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 14:01:08.899497 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894837    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 14:01:08.899497 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894839    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 14:01:08.899497 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894842    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 14:01:08.899497 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894844    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 14:01:08.899497 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894847    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 14:01:08.899497 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894851    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 14:01:08.899497 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894854    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 14:01:08.900011 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894857    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 14:01:08.900011 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894860    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 14:01:08.900011 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894862    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 14:01:08.900011 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894865    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 14:01:08.900011 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894867    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 14:01:08.900011 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894870    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 14:01:08.900011 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894872    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 14:01:08.900011 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894875    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 14:01:08.900011 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894878    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 14:01:08.900011 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894880    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 14:01:08.900011 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894883    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 14:01:08.900011 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894885    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 14:01:08.900011 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894888    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 14:01:08.900011 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894892    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 14:01:08.900011 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894895    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 14:01:08.900011 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894898    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 14:01:08.900011 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894900    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 14:01:08.900011 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894903    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 14:01:08.900011 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894906    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 14:01:08.900011 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894908    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 14:01:08.900507 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894911    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 14:01:08.900507 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894913    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 14:01:08.900507 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894917    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 14:01:08.900507 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894921    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 14:01:08.900507 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894923    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 14:01:08.900507 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894926    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 14:01:08.900507 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894929    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 14:01:08.900507 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894931    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 14:01:08.900507 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894934    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 14:01:08.900507 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894936    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 14:01:08.900507 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894939    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 14:01:08.900507 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894942    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 14:01:08.900507 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894945    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 14:01:08.900507 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894948    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 14:01:08.900507 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894951    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 14:01:08.900507 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.894954    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 14:01:08.900900 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.894959    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 14:01:08.900900 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895063    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 14:01:08.900900 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895067    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 14:01:08.900900 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895070    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 14:01:08.900900 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895073    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 14:01:08.900900 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895075    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 14:01:08.900900 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895078    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 14:01:08.900900 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895080    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 14:01:08.900900 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895083    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 14:01:08.900900 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895086    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 14:01:08.900900 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895088    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 14:01:08.900900 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895091    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 14:01:08.900900 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895093    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 14:01:08.900900 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895095    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 14:01:08.900900 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895098    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 14:01:08.901275 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895101    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 14:01:08.901275 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895103    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 14:01:08.901275 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895106    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 14:01:08.901275 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895109    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 14:01:08.901275 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895112    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 14:01:08.901275 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895114    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 14:01:08.901275 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895116    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 14:01:08.901275 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895119    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 14:01:08.901275 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895121    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 14:01:08.901275 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895124    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 14:01:08.901275 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895126    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 14:01:08.901275 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895129    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 14:01:08.901275 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895132    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 14:01:08.901275 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895134    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 14:01:08.901275 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895137    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 14:01:08.901275 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895140    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 14:01:08.901275 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895143    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 14:01:08.901275 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895147    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 14:01:08.901275 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895149    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 14:01:08.901749 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895152    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 14:01:08.901749 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895155    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 14:01:08.901749 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895158    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 14:01:08.901749 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895160    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 14:01:08.901749 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895163    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 14:01:08.901749 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895165    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 14:01:08.901749 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895168    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 14:01:08.901749 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895170    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 14:01:08.901749 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895172    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 14:01:08.901749 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895175    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 14:01:08.901749 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895177    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 14:01:08.901749 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895180    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 14:01:08.901749 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895182    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 14:01:08.901749 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895185    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 14:01:08.901749 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895187    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 14:01:08.901749 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895190    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 14:01:08.901749 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895193    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 14:01:08.901749 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895195    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 14:01:08.901749 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895198    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 14:01:08.901749 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895201    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 14:01:08.902250 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895203    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 14:01:08.902250 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895205    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 14:01:08.902250 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895208    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 14:01:08.902250 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895210    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 14:01:08.902250 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895213    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 14:01:08.902250 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895216    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 14:01:08.902250 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895219    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 14:01:08.902250 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895221    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 14:01:08.902250 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895224    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 14:01:08.902250 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895226    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 14:01:08.902250 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895229    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 14:01:08.902250 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895231    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 14:01:08.902250 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895234    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 14:01:08.902250 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895236    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 14:01:08.902250 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895239    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 14:01:08.902250 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895241    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 14:01:08.902250 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895244    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 14:01:08.902250 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895246    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 14:01:08.902250 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895249    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 14:01:08.902250 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895251    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 14:01:08.902762 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895254    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 14:01:08.902762 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895257    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 14:01:08.902762 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895261    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 14:01:08.902762 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895263    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 14:01:08.902762 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895267    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 14:01:08.902762 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895269    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 14:01:08.902762 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895272    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 14:01:08.902762 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895275    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 14:01:08.902762 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895277    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 14:01:08.902762 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895280    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 14:01:08.902762 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895282    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 14:01:08.902762 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895285    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 14:01:08.902762 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895287    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 14:01:08.902762 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895290    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 14:01:08.902762 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895292    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 14:01:08.902762 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895294    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 14:01:08.902762 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895297    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 14:01:08.902762 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895299    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 14:01:08.902762 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895302    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 14:01:08.903239 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895305    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 14:01:08.903239 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895308    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 14:01:08.903239 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895311    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 14:01:08.903239 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895314    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 14:01:08.903239 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895317    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 14:01:08.903239 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895319    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 14:01:08.903239 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:08.895322    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 14:01:08.903239 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.895326    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 14:01:08.903239 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.895423    2559 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 29 14:01:08.903239 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.897640    2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 29 14:01:08.903239 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.899317    2559 server.go:1019] "Starting client certificate rotation"
Jul 29 14:01:08.903239 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.899416    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 14:01:08.903239 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.899445    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 14:01:08.903554 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.902178    2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 14:01:08.903626 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.903608    2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 14:01:08.916792 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.916774    2559 log.go:25] "Validated CRI v1 runtime API"
Jul 29 14:01:08.919233 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.919217    2559 log.go:25] "Validated CRI v1 image API"
Jul 29 14:01:08.920104 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.920087    2559 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 29 14:01:08.926447 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.926427    2559 fs.go:135] Filesystem UUIDs: map[3769301e-2420-4184-85e9-704b2fa82453:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 fd163e0e-3c65-4db5-9168-7fa3a53d1260:/dev/nvme0n1p4]
Jul 29 14:01:08.926519 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.926447    2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 29 14:01:08.930291 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.930274    2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 14:01:08.931472 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.931367    2559 manager.go:216] Machine: {Timestamp:2026-07-29 14:01:08.930038197 +0000 UTC m=+0.381079217 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097972 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23645f3ca245baf0958e7a8329f7aa SystemUUID:ec23645f-3ca2-45ba-f095-8e7a8329f7aa BootID:dcb24302-752c-49c0-ad14-0b3ea7f96e4b Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2f:00:6b:23:1f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2f:00:6b:23:1f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:92:19:e3:9b:f7:96 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 29 14:01:08.931472 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.931467    2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 29 14:01:08.931575 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.931544    2559 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 29 14:01:08.934766 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.934741    2559 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 29 14:01:08.934896 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.934769    2559 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-187.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 29 14:01:08.935595 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.935585    2559 topology_manager.go:138] "Creating topology manager with none policy"
Jul 29 14:01:08.935633 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.935598    2559 container_manager_linux.go:309] "Creating device plugin manager"
Jul 29 14:01:08.935633 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.935605    2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 14:01:08.935633 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.935631    2559 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 14:01:08.936887 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.936876    2559 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 29 14:01:08.938611 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.938600    2559 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 14:01:08.938754 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.938744    2559 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 29 14:01:08.938788 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.938776    2559 kubelet.go:486] "Attempting to sync node with API server"
Jul 29 14:01:08.938822 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.938790    2559 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 29 14:01:08.938822 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.938810    2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 29 14:01:08.938876 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.938824    2559 kubelet.go:398] "Adding apiserver pod source"
Jul 29 14:01:08.938876 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.938835    2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 29 14:01:08.939236 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.939225    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 14:01:08.939333 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.939245    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 14:01:08.941802 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.941786    2559 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 29 14:01:08.944128 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.944114    2559 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 29 14:01:08.944172 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.944145    2559 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 29 14:01:08.946246 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.946229    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 29 14:01:08.946246 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.946246    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 29 14:01:08.946246 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.946253    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 29 14:01:08.946440 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.946260    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 29 14:01:08.946440 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.946273    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 29 14:01:08.946440 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.946280    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 29 14:01:08.946440 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.946288    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 29 14:01:08.946440 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.946293    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 29 14:01:08.946440 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.946301    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 29 14:01:08.946440 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.946306    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 29 14:01:08.946440 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.946314    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 29 14:01:08.946440 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.946319    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 29 14:01:08.946440 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.946342    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 29 14:01:08.946440 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.946350    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 29 14:01:08.947769 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.947754    2559 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-187.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 29 14:01:08.947835 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:08.947778    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-187.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 29 14:01:08.947888 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:08.947843    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 29 14:01:08.949751 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.949738    2559 server.go:1268] "Started kubelet"
Jul 29 14:01:08.949850 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.949824    2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 29 14:01:08.950619 ip-10-0-131-187 systemd[1]: Started Kubernetes Kubelet.
Jul 29 14:01:08.955504 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.950716    2559 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 29 14:01:08.955641 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.955628    2559 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 29 14:01:08.956984 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.956955    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-z5j6s"
Jul 29 14:01:08.957652 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.957632    2559 server.go:310] "Adding debug handlers to kubelet server"
Jul 29 14:01:08.958426 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.958392    2559 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 29 14:01:08.959613 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:08.959593    2559 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 29 14:01:08.959872 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:08.958812    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-187.ec2.internal.18c6c78347482548  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-187.ec2.internal,UID:ip-10-0-131-187.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-187.ec2.internal,},FirstTimestamp:2026-07-29 14:01:08.949534024 +0000 UTC m=+0.400575069,LastTimestamp:2026-07-29 14:01:08.949534024 +0000 UTC m=+0.400575069,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-187.ec2.internal,}"
Jul 29 14:01:08.964417 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.964401    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-z5j6s"
Jul 29 14:01:08.966369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.966349    2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 29 14:01:08.966369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.966359    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 29 14:01:08.966524 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:08.966441    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-187.ec2.internal\" not found"
Jul 29 14:01:08.967186 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.967169    2559 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 29 14:01:08.967186 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.967184    2559 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 29 14:01:08.967323 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.967188    2559 factory.go:55] Registering systemd factory
Jul 29 14:01:08.967323 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.967187    2559 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 29 14:01:08.967323 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.967204    2559 factory.go:223] Registration of the systemd container factory successfully
Jul 29 14:01:08.967323 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.967272    2559 reconstruct.go:97] "Volume reconstruction finished"
Jul 29 14:01:08.967323 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.967281    2559 reconciler.go:29] "Reconciler: start to sync state"
Jul 29 14:01:08.967535 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.967409    2559 factory.go:153] Registering CRI-O factory
Jul 29 14:01:08.967535 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.967424    2559 factory.go:223] Registration of the crio container factory successfully
Jul 29 14:01:08.967535 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.967475    2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 29 14:01:08.967535 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.967498    2559 factory.go:103] Registering Raw factory
Jul 29 14:01:08.967535 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.967514    2559 manager.go:1196] Started watching for new ooms in manager
Jul 29 14:01:08.967966 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.967951    2559 manager.go:349] Starting recovery of all containers
Jul 29 14:01:08.976646 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.976628    2559 manager.go:354] Recovery completed
Jul 29 14:01:08.977356 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.977334    2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 14:01:08.980006 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:08.979991    2559 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-131-187.ec2.internal\" not found" node="ip-10-0-131-187.ec2.internal"
Jul 29 14:01:08.980936 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.980924    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 14:01:08.982965 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.982949    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-187.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 14:01:08.983035 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.982976    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-187.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 14:01:08.983035 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.982986    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-187.ec2.internal" event="NodeHasSufficientPID"
Jul 29 14:01:08.983584 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.983571    2559 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 29 14:01:08.983584 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.983582    2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 29 14:01:08.983726 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.983600    2559 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 14:01:08.983726 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.983615    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 14:01:08.984360 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.984347    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-187.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 14:01:08.984412 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.984371    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-187.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 14:01:08.984412 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.984380    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-187.ec2.internal" event="NodeHasSufficientPID"
Jul 29 14:01:08.985250 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.985238    2559 policy_none.go:49] "None policy: Start"
Jul 29 14:01:08.985308 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.985254    2559 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 29 14:01:08.985308 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.985264    2559 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 29 14:01:08.986118 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:08.986108    2559 policy_none.go:47] "Start"
Jul 29 14:01:09.009359 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.009343    2559 manager.go:337] "Starting Device Plugin manager"
Jul 29 14:01:09.009437 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:09.009382    2559 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 29 14:01:09.009437 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.009393    2559 server.go:87] "Starting device plugin registration server"
Jul 29 14:01:09.009597 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.009586    2559 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 29 14:01:09.009627 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.009601    2559 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 29 14:01:09.011494 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.011478    2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 29 14:01:09.011573 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.011564    2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 29 14:01:09.011618 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.011573    2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 29 14:01:09.012163 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:09.012146    2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 29 14:01:09.012219 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:09.012189    2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-187.ec2.internal\" not found"
Jul 29 14:01:09.068337 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.068307    2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 29 14:01:09.069724 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.069701    2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 29 14:01:09.069804 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.069744    2559 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 29 14:01:09.069804 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.069767    2559 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 29 14:01:09.069892 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:09.069805    2559 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 29 14:01:09.072225 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.072209    2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 14:01:09.110196 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.110167    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 14:01:09.111334 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.111316    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-187.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 14:01:09.111408 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.111349    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-187.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 14:01:09.111408 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.111360    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-187.ec2.internal" event="NodeHasSufficientPID"
Jul 29 14:01:09.111408 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.111381    2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-187.ec2.internal"
Jul 29 14:01:09.122429 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.122414    2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-187.ec2.internal"
Jul 29 14:01:09.122478 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:09.122434    2559 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-187.ec2.internal\": node \"ip-10-0-131-187.ec2.internal\" not found"
Jul 29 14:01:09.138093 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:09.138044    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-187.ec2.internal\" not found"
Jul 29 14:01:09.170527 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.170380    2559 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-187.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-187.ec2.internal"]
Jul 29 14:01:09.170611 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.170568    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 14:01:09.171921 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.171903    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-187.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 14:01:09.171966 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.171930    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-187.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 14:01:09.171966 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.171942    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-187.ec2.internal" event="NodeHasSufficientPID"
Jul 29 14:01:09.172933 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.172922    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 14:01:09.173039 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.173024    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-187.ec2.internal"
Jul 29 14:01:09.173080 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.173053    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 14:01:09.173616 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.173589    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-187.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 14:01:09.173705 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.173622    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-187.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 14:01:09.173705 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.173634    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-187.ec2.internal" event="NodeHasSufficientPID"
Jul 29 14:01:09.173705 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.173593    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-187.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 14:01:09.173705 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.173705    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-187.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 14:01:09.173833 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.173723    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-187.ec2.internal" event="NodeHasSufficientPID"
Jul 29 14:01:09.174611 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.174598    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-187.ec2.internal"
Jul 29 14:01:09.174649 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.174620    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 14:01:09.178446 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.178431    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-187.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 14:01:09.178547 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.178460    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-187.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 14:01:09.178547 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.178474    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-187.ec2.internal" event="NodeHasSufficientPID"
Jul 29 14:01:09.196395 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:09.196372    2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-187.ec2.internal\" not found" node="ip-10-0-131-187.ec2.internal"
Jul 29 14:01:09.200911 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:09.200897    2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-187.ec2.internal\" not found" node="ip-10-0-131-187.ec2.internal"
Jul 29 14:01:09.238130 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:09.238103    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-187.ec2.internal\" not found"
Jul 29 14:01:09.269175 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.269145    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/770d58cac971d94e189e1619f4f30236-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-187.ec2.internal\" (UID: \"770d58cac971d94e189e1619f4f30236\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-187.ec2.internal"
Jul 29 14:01:09.269322 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.269181    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4dad8c770ea44aa6fdaf6d1c2a53cc88-config\") pod \"kube-apiserver-proxy-ip-10-0-131-187.ec2.internal\" (UID: \"4dad8c770ea44aa6fdaf6d1c2a53cc88\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-187.ec2.internal"
Jul 29 14:01:09.269322 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.269205    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/770d58cac971d94e189e1619f4f30236-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-187.ec2.internal\" (UID: \"770d58cac971d94e189e1619f4f30236\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-187.ec2.internal"
Jul 29 14:01:09.338761 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:09.338733    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-187.ec2.internal\" not found"
Jul 29 14:01:09.370153 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.370130    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/770d58cac971d94e189e1619f4f30236-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-187.ec2.internal\" (UID: \"770d58cac971d94e189e1619f4f30236\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-187.ec2.internal"
Jul 29 14:01:09.370238 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.370156    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4dad8c770ea44aa6fdaf6d1c2a53cc88-config\") pod \"kube-apiserver-proxy-ip-10-0-131-187.ec2.internal\" (UID: \"4dad8c770ea44aa6fdaf6d1c2a53cc88\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-187.ec2.internal"
Jul 29 14:01:09.370238 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.370179    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/770d58cac971d94e189e1619f4f30236-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-187.ec2.internal\" (UID: \"770d58cac971d94e189e1619f4f30236\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-187.ec2.internal"
Jul 29 14:01:09.370238 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.370228    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/770d58cac971d94e189e1619f4f30236-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-187.ec2.internal\" (UID: \"770d58cac971d94e189e1619f4f30236\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-187.ec2.internal"
Jul 29 14:01:09.370357 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.370259    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4dad8c770ea44aa6fdaf6d1c2a53cc88-config\") pod \"kube-apiserver-proxy-ip-10-0-131-187.ec2.internal\" (UID: \"4dad8c770ea44aa6fdaf6d1c2a53cc88\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-187.ec2.internal"
Jul 29 14:01:09.370357 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.370263    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/770d58cac971d94e189e1619f4f30236-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-187.ec2.internal\" (UID: \"770d58cac971d94e189e1619f4f30236\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-187.ec2.internal"
Jul 29 14:01:09.439362 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:09.439333    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-187.ec2.internal\" not found"
Jul 29 14:01:09.498906 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.498878    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-187.ec2.internal"
Jul 29 14:01:09.503580 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:09.503560    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-187.ec2.internal"
Jul 29 14:01:09.540311 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:09.540281    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-187.ec2.internal\" not found"
Jul 29 14:01:11.201184 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:11.201156    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-187.ec2.internal\" not found"
Jul 29 14:01:11.301779 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:11.301686    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-187.ec2.internal\" not found"
Jul 29 14:01:11.402351 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:11.402318    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-187.ec2.internal\" not found"
Jul 29 14:01:11.453517 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.453493    2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 14:01:11.458309 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.458289    2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 29 14:01:11.458421 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.458386    2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 14:01:11.458467 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.458429    2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 14:01:11.458467 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.458430    2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 14:01:11.499599 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.499577    2559 apiserver.go:52] "Watching apiserver"
Jul 29 14:01:11.519236 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.519215    2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 14:01:11.520701 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.520675    2559 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 29 14:01:11.521716 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.521692    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-84fpq","kube-system/konnectivity-agent-7xpvn","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlk7p","openshift-dns/node-resolver-d9ncc","openshift-multus/network-metrics-daemon-cwxl9","openshift-network-diagnostics/network-check-target-gbj56","openshift-network-operator/iptables-alerter-mz4xm","openshift-ovn-kubernetes/ovnkube-node-hdw8l","openshift-cluster-node-tuning-operator/tuned-szq55","openshift-image-registry/node-ca-nl5mz","openshift-multus/multus-additional-cni-plugins-bcfwk","openshift-multus/multus-hbw9p"]
Jul 29 14:01:11.522955 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.522936    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:01:11.523044 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:11.523013    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84fpq" podUID="b0c6da59-c35d-4d20-8346-3dd5afc3a8f3"
Jul 29 14:01:11.523942 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.523924    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7xpvn"
Jul 29 14:01:11.524946 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.524930    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlk7p"
Jul 29 14:01:11.525874 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.525859    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-d9ncc"
Jul 29 14:01:11.526693 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.526671    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 29 14:01:11.526788 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.526733    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4dnrb\""
Jul 29 14:01:11.526854 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.526829    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 29 14:01:11.526906 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.526859    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 29 14:01:11.526959 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.526924    2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-187.ec2.internal"
Jul 29 14:01:11.527232 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.527218    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwxl9"
Jul 29 14:01:11.527306 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:11.527288    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwxl9" podUID="93be82f5-6d56-4bd3-9822-bef13d5738e1"
Jul 29 14:01:11.527442 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.527427    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-w7hfg\""
Jul 29 14:01:11.528330 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.528315    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbj56"
Jul 29 14:01:11.529673 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.529107    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 29 14:01:11.529673 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.529121    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 29 14:01:11.529673 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.529138    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 29 14:01:11.529673 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:11.528379    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbj56" podUID="6d24b8b8-c850-495d-b0f7-ca7ea53a8a57"
Jul 29 14:01:11.529673 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.529465    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 29 14:01:11.530647 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.530043    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 29 14:01:11.530861 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.530841    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mz4xm"
Jul 29 14:01:11.530921 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.530882    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-sgq4z\""
Jul 29 14:01:11.531230 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.531188    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-28 13:56:10 +0000 UTC" deadline="2028-01-23 04:42:47.781093246 +0000 UTC"
Jul 29 14:01:11.531308 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.531230    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13022h41m36.249866558s"
Jul 29 14:01:11.532159 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.532141    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.533417 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.533400    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.533771 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.533750    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 29 14:01:11.533771 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.533760    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 29 14:01:11.533944 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.533755    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/253b40f9-e444-40f9-a59c-1cd0f39c4dbe-registration-dir\") pod \"aws-ebs-csi-driver-node-jlk7p\" (UID: \"253b40f9-e444-40f9-a59c-1cd0f39c4dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlk7p"
Jul 29 14:01:11.533944 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.533843    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 29 14:01:11.533944 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.533817    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/253b40f9-e444-40f9-a59c-1cd0f39c4dbe-etc-selinux\") pod \"aws-ebs-csi-driver-node-jlk7p\" (UID: \"253b40f9-e444-40f9-a59c-1cd0f39c4dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlk7p"
Jul 29 14:01:11.533944 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.533883    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-872pb\" (UniqueName: \"kubernetes.io/projected/253b40f9-e444-40f9-a59c-1cd0f39c4dbe-kube-api-access-872pb\") pod \"aws-ebs-csi-driver-node-jlk7p\" (UID: \"253b40f9-e444-40f9-a59c-1cd0f39c4dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlk7p"
Jul 29 14:01:11.533944 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.533938    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtmq4\" (UniqueName: \"kubernetes.io/projected/22c32d1e-2781-499f-a092-2898013fb37e-kube-api-access-jtmq4\") pod \"node-resolver-d9ncc\" (UID: \"22c32d1e-2781-499f-a092-2898013fb37e\") " pod="openshift-dns/node-resolver-d9ncc"
Jul 29 14:01:11.534154 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.533972    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/253b40f9-e444-40f9-a59c-1cd0f39c4dbe-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jlk7p\" (UID: \"253b40f9-e444-40f9-a59c-1cd0f39c4dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlk7p"
Jul 29 14:01:11.534154 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.534008    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22c32d1e-2781-499f-a092-2898013fb37e-tmp-dir\") pod \"node-resolver-d9ncc\" (UID: \"22c32d1e-2781-499f-a092-2898013fb37e\") " pod="openshift-dns/node-resolver-d9ncc"
Jul 29 14:01:11.534154 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.534030    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-78lgq\""
Jul 29 14:01:11.534154 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.534100    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66d5t\" (UniqueName: \"kubernetes.io/projected/93be82f5-6d56-4bd3-9822-bef13d5738e1-kube-api-access-66d5t\") pod \"network-metrics-daemon-cwxl9\" (UID: \"93be82f5-6d56-4bd3-9822-bef13d5738e1\") " pod="openshift-multus/network-metrics-daemon-cwxl9"
Jul 29 14:01:11.534154 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.534127    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnwlt\" (UniqueName: \"kubernetes.io/projected/6d24b8b8-c850-495d-b0f7-ca7ea53a8a57-kube-api-access-tnwlt\") pod \"network-check-target-gbj56\" (UID: \"6d24b8b8-c850-495d-b0f7-ca7ea53a8a57\") " pod="openshift-network-diagnostics/network-check-target-gbj56"
Jul 29 14:01:11.534313 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.534167    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b0c6da59-c35d-4d20-8346-3dd5afc3a8f3-original-pull-secret\") pod \"global-pull-secret-syncer-84fpq\" (UID: \"b0c6da59-c35d-4d20-8346-3dd5afc3a8f3\") " pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:01:11.534313 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.534185    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/62eed460-c3d4-4d6e-a7ff-cf1290a96c53-konnectivity-ca\") pod \"konnectivity-agent-7xpvn\" (UID: \"62eed460-c3d4-4d6e-a7ff-cf1290a96c53\") " pod="kube-system/konnectivity-agent-7xpvn"
Jul 29 14:01:11.534313 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.534206    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/253b40f9-e444-40f9-a59c-1cd0f39c4dbe-device-dir\") pod \"aws-ebs-csi-driver-node-jlk7p\" (UID: \"253b40f9-e444-40f9-a59c-1cd0f39c4dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlk7p"
Jul 29 14:01:11.534313 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.534228    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/22c32d1e-2781-499f-a092-2898013fb37e-hosts-file\") pod \"node-resolver-d9ncc\" (UID: \"22c32d1e-2781-499f-a092-2898013fb37e\") " pod="openshift-dns/node-resolver-d9ncc"
Jul 29 14:01:11.534313 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.534251    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/253b40f9-e444-40f9-a59c-1cd0f39c4dbe-sys-fs\") pod \"aws-ebs-csi-driver-node-jlk7p\" (UID: \"253b40f9-e444-40f9-a59c-1cd0f39c4dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlk7p"
Jul 29 14:01:11.534313 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.534267    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93be82f5-6d56-4bd3-9822-bef13d5738e1-metrics-certs\") pod \"network-metrics-daemon-cwxl9\" (UID: \"93be82f5-6d56-4bd3-9822-bef13d5738e1\") " pod="openshift-multus/network-metrics-daemon-cwxl9"
Jul 29 14:01:11.534313 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.534285    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b0c6da59-c35d-4d20-8346-3dd5afc3a8f3-kubelet-config\") pod \"global-pull-secret-syncer-84fpq\" (UID: \"b0c6da59-c35d-4d20-8346-3dd5afc3a8f3\") " pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:01:11.534535 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.534317    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b0c6da59-c35d-4d20-8346-3dd5afc3a8f3-dbus\") pod \"global-pull-secret-syncer-84fpq\" (UID: \"b0c6da59-c35d-4d20-8346-3dd5afc3a8f3\") " pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:01:11.534535 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.534337    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/62eed460-c3d4-4d6e-a7ff-cf1290a96c53-agent-certs\") pod \"konnectivity-agent-7xpvn\" (UID: \"62eed460-c3d4-4d6e-a7ff-cf1290a96c53\") " pod="kube-system/konnectivity-agent-7xpvn"
Jul 29 14:01:11.534535 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.534350    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/253b40f9-e444-40f9-a59c-1cd0f39c4dbe-socket-dir\") pod \"aws-ebs-csi-driver-node-jlk7p\" (UID: \"253b40f9-e444-40f9-a59c-1cd0f39c4dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlk7p"
Jul 29 14:01:11.534881 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.534866    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 29 14:01:11.534993 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.534977    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nl5mz"
Jul 29 14:01:11.535483 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.535466    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 29 14:01:11.535578 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.535509    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xgc9t\""
Jul 29 14:01:11.535578 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.535526    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 29 14:01:11.536306 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.536289    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bcfwk"
Jul 29 14:01:11.536773 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.536753    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 29 14:01:11.536857 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.536814    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 29 14:01:11.536857 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.536846    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 29 14:01:11.537024 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.536954    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 29 14:01:11.537117 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.537102    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-bpbqm\""
Jul 29 14:01:11.537162 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.537140    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 29 14:01:11.537557 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.537538    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.537668 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.537542    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 29 14:01:11.537668 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.537656    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 29 14:01:11.537835 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.537817    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 29 14:01:11.537903 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.537883    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qck8x\""
Jul 29 14:01:11.538746 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.538727    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 29 14:01:11.539307 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.539288    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 29 14:01:11.539469 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.539454    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 29 14:01:11.539800 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.539767    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-ccv6q\""
Jul 29 14:01:11.539800 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.539777    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 14:01:11.539939 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.539811    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 29 14:01:11.539939 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.539818    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 29 14:01:11.539939 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.539859    2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-187.ec2.internal"
Jul 29 14:01:11.540051 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.540020    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-187.ec2.internal"]
Jul 29 14:01:11.540886 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.540874    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 29 14:01:11.540924 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.540898    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-kwvxs\""
Jul 29 14:01:11.545672 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.545655    2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 14:01:11.556525 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.556479    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 14:01:11.556696 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.556680    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-131-187.ec2.internal"]
Jul 29 14:01:11.560757 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.560740    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-vw5dd"
Jul 29 14:01:11.569937 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.569918    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vw5dd"
Jul 29 14:01:11.573052 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:11.573028    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dad8c770ea44aa6fdaf6d1c2a53cc88.slice/crio-c4e0277849e0bd6633cb8b76c3c284e0317174752684889c3c26ee1b9d5abfc1 WatchSource:0}: Error finding container c4e0277849e0bd6633cb8b76c3c284e0317174752684889c3c26ee1b9d5abfc1: Status 404 returned error can't find the container with id c4e0277849e0bd6633cb8b76c3c284e0317174752684889c3c26ee1b9d5abfc1
Jul 29 14:01:11.573256 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:11.573230    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod770d58cac971d94e189e1619f4f30236.slice/crio-57b590765e46a7f3c5d09e1fcb1219161810ccc6f4cbfb0dc0135b777e340405 WatchSource:0}: Error finding container 57b590765e46a7f3c5d09e1fcb1219161810ccc6f4cbfb0dc0135b777e340405: Status 404 returned error can't find the container with id 57b590765e46a7f3c5d09e1fcb1219161810ccc6f4cbfb0dc0135b777e340405
Jul 29 14:01:11.579930 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.579916    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 14:01:11.628294 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.628274    2559 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 29 14:01:11.632702 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.632662    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-187.ec2.internal" event={"ID":"4dad8c770ea44aa6fdaf6d1c2a53cc88","Type":"ContainerStarted","Data":"c4e0277849e0bd6633cb8b76c3c284e0317174752684889c3c26ee1b9d5abfc1"}
Jul 29 14:01:11.633635 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.633616    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-187.ec2.internal" event={"ID":"770d58cac971d94e189e1619f4f30236","Type":"ContainerStarted","Data":"57b590765e46a7f3c5d09e1fcb1219161810ccc6f4cbfb0dc0135b777e340405"}
Jul 29 14:01:11.634845 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.634829    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93be82f5-6d56-4bd3-9822-bef13d5738e1-metrics-certs\") pod \"network-metrics-daemon-cwxl9\" (UID: \"93be82f5-6d56-4bd3-9822-bef13d5738e1\") " pod="openshift-multus/network-metrics-daemon-cwxl9"
Jul 29 14:01:11.634887 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.634854    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b0c6da59-c35d-4d20-8346-3dd5afc3a8f3-dbus\") pod \"global-pull-secret-syncer-84fpq\" (UID: \"b0c6da59-c35d-4d20-8346-3dd5afc3a8f3\") " pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:01:11.634887 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.634871    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-system-cni-dir\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.634972 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.634891    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-host-cni-bin\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.634972 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.634914    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d6541ac6-8d3b-4175-9881-3e5e82d72429-etc-sysconfig\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.635080 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.634968    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6541ac6-8d3b-4175-9881-3e5e82d72429-etc-kubernetes\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.635080 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:11.634985    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 14:01:11.635080 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635014    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d6541ac6-8d3b-4175-9881-3e5e82d72429-etc-systemd\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.635080 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:11.635037    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93be82f5-6d56-4bd3-9822-bef13d5738e1-metrics-certs podName:93be82f5-6d56-4bd3-9822-bef13d5738e1 nodeName:}" failed. No retries permitted until 2026-07-29 14:01:12.135021259 +0000 UTC m=+2.025692602 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93be82f5-6d56-4bd3-9822-bef13d5738e1-metrics-certs") pod "network-metrics-daemon-cwxl9" (UID: "93be82f5-6d56-4bd3-9822-bef13d5738e1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 14:01:11.635080 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635070    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b0c6da59-c35d-4d20-8346-3dd5afc3a8f3-dbus\") pod \"global-pull-secret-syncer-84fpq\" (UID: \"b0c6da59-c35d-4d20-8346-3dd5afc3a8f3\") " pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:01:11.635311 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635086    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/62eed460-c3d4-4d6e-a7ff-cf1290a96c53-agent-certs\") pod \"konnectivity-agent-7xpvn\" (UID: \"62eed460-c3d4-4d6e-a7ff-cf1290a96c53\") " pod="kube-system/konnectivity-agent-7xpvn"
Jul 29 14:01:11.635311 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635103    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/253b40f9-e444-40f9-a59c-1cd0f39c4dbe-socket-dir\") pod \"aws-ebs-csi-driver-node-jlk7p\" (UID: \"253b40f9-e444-40f9-a59c-1cd0f39c4dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlk7p"
Jul 29 14:01:11.635311 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635117    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/253b40f9-e444-40f9-a59c-1cd0f39c4dbe-registration-dir\") pod \"aws-ebs-csi-driver-node-jlk7p\" (UID: \"253b40f9-e444-40f9-a59c-1cd0f39c4dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlk7p"
Jul 29 14:01:11.635311 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635133    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-node-log\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.635311 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635147    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6541ac6-8d3b-4175-9881-3e5e82d72429-var-lib-kubelet\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.635311 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635172    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d6541ac6-8d3b-4175-9881-3e5e82d72429-host\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.635311 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635189    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/253b40f9-e444-40f9-a59c-1cd0f39c4dbe-registration-dir\") pod \"aws-ebs-csi-driver-node-jlk7p\" (UID: \"253b40f9-e444-40f9-a59c-1cd0f39c4dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlk7p"
Jul 29 14:01:11.635311 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635208    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-cni-binary-copy\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.635311 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635237    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/253b40f9-e444-40f9-a59c-1cd0f39c4dbe-socket-dir\") pod \"aws-ebs-csi-driver-node-jlk7p\" (UID: \"253b40f9-e444-40f9-a59c-1cd0f39c4dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlk7p"
Jul 29 14:01:11.635311 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635244    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-etc-kubernetes\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.635311 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635272    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-run-systemd\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.635311 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635312    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-etc-openvswitch\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.635764 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635339    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2317c90b-86b0-4ce3-a455-a973befb1b11-serviceca\") pod \"node-ca-nl5mz\" (UID: \"2317c90b-86b0-4ce3-a455-a973befb1b11\") " pod="openshift-image-registry/node-ca-nl5mz"
Jul 29 14:01:11.635764 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635352    2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 29 14:01:11.635764 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635365    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbctv\" (UniqueName: \"kubernetes.io/projected/2317c90b-86b0-4ce3-a455-a973befb1b11-kube-api-access-wbctv\") pod \"node-ca-nl5mz\" (UID: \"2317c90b-86b0-4ce3-a455-a973befb1b11\") " pod="openshift-image-registry/node-ca-nl5mz"
Jul 29 14:01:11.635764 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635401    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/253b40f9-e444-40f9-a59c-1cd0f39c4dbe-sys-fs\") pod \"aws-ebs-csi-driver-node-jlk7p\" (UID: \"253b40f9-e444-40f9-a59c-1cd0f39c4dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlk7p"
Jul 29 14:01:11.635764 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635424    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-multus-socket-dir-parent\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.635764 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635445    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-systemd-units\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.635764 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635465    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/253b40f9-e444-40f9-a59c-1cd0f39c4dbe-sys-fs\") pod \"aws-ebs-csi-driver-node-jlk7p\" (UID: \"253b40f9-e444-40f9-a59c-1cd0f39c4dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlk7p"
Jul 29 14:01:11.635764 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635467    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d8rw\" (UniqueName: \"kubernetes.io/projected/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-kube-api-access-4d8rw\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.635764 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635490    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d6541ac6-8d3b-4175-9881-3e5e82d72429-run\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.635764 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635505    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/24dbef56-b5b2-49e3-b2d6-e3ea78eeb731-cni-binary-copy\") pod \"multus-additional-cni-plugins-bcfwk\" (UID: \"24dbef56-b5b2-49e3-b2d6-e3ea78eeb731\") " pod="openshift-multus/multus-additional-cni-plugins-bcfwk"
Jul 29 14:01:11.635764 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635521    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6n5tp\" (UniqueName: \"kubernetes.io/projected/24dbef56-b5b2-49e3-b2d6-e3ea78eeb731-kube-api-access-6n5tp\") pod \"multus-additional-cni-plugins-bcfwk\" (UID: \"24dbef56-b5b2-49e3-b2d6-e3ea78eeb731\") " pod="openshift-multus/multus-additional-cni-plugins-bcfwk"
Jul 29 14:01:11.635764 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635538    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/24dbef56-b5b2-49e3-b2d6-e3ea78eeb731-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bcfwk\" (UID: \"24dbef56-b5b2-49e3-b2d6-e3ea78eeb731\") " pod="openshift-multus/multus-additional-cni-plugins-bcfwk"
Jul 29 14:01:11.635764 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635553    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-log-socket\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.635764 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635568    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d6541ac6-8d3b-4175-9881-3e5e82d72429-etc-tuned\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.635764 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635590    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/253b40f9-e444-40f9-a59c-1cd0f39c4dbe-device-dir\") pod \"aws-ebs-csi-driver-node-jlk7p\" (UID: \"253b40f9-e444-40f9-a59c-1cd0f39c4dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlk7p"
Jul 29 14:01:11.635764 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635605    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-host-var-lib-cni-multus\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.635764 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635619    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dh7l\" (UniqueName: \"kubernetes.io/projected/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-kube-api-access-7dh7l\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.636475 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635634    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-host-kubelet\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.636475 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635657    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-host-run-ovn-kubernetes\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.636475 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635687    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/253b40f9-e444-40f9-a59c-1cd0f39c4dbe-device-dir\") pod \"aws-ebs-csi-driver-node-jlk7p\" (UID: \"253b40f9-e444-40f9-a59c-1cd0f39c4dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlk7p"
Jul 29 14:01:11.636475 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635687    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d6541ac6-8d3b-4175-9881-3e5e82d72429-etc-sysctl-conf\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.636475 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635717    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b0c6da59-c35d-4d20-8346-3dd5afc3a8f3-kubelet-config\") pod \"global-pull-secret-syncer-84fpq\" (UID: \"b0c6da59-c35d-4d20-8346-3dd5afc3a8f3\") " pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:01:11.636475 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635755    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-ovn-node-metrics-cert\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.636475 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635763    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b0c6da59-c35d-4d20-8346-3dd5afc3a8f3-kubelet-config\") pod \"global-pull-secret-syncer-84fpq\" (UID: \"b0c6da59-c35d-4d20-8346-3dd5afc3a8f3\") " pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:01:11.636475 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635780    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/253b40f9-e444-40f9-a59c-1cd0f39c4dbe-etc-selinux\") pod \"aws-ebs-csi-driver-node-jlk7p\" (UID: \"253b40f9-e444-40f9-a59c-1cd0f39c4dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlk7p"
Jul 29 14:01:11.636475 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635809    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-872pb\" (UniqueName: \"kubernetes.io/projected/253b40f9-e444-40f9-a59c-1cd0f39c4dbe-kube-api-access-872pb\") pod \"aws-ebs-csi-driver-node-jlk7p\" (UID: \"253b40f9-e444-40f9-a59c-1cd0f39c4dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlk7p"
Jul 29 14:01:11.636475 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635834    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtmq4\" (UniqueName: \"kubernetes.io/projected/22c32d1e-2781-499f-a092-2898013fb37e-kube-api-access-jtmq4\") pod \"node-resolver-d9ncc\" (UID: \"22c32d1e-2781-499f-a092-2898013fb37e\") " pod="openshift-dns/node-resolver-d9ncc"
Jul 29 14:01:11.636475 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635843    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/253b40f9-e444-40f9-a59c-1cd0f39c4dbe-etc-selinux\") pod \"aws-ebs-csi-driver-node-jlk7p\" (UID: \"253b40f9-e444-40f9-a59c-1cd0f39c4dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlk7p"
Jul 29 14:01:11.636475 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635859    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/24dbef56-b5b2-49e3-b2d6-e3ea78eeb731-cnibin\") pod \"multus-additional-cni-plugins-bcfwk\" (UID: \"24dbef56-b5b2-49e3-b2d6-e3ea78eeb731\") " pod="openshift-multus/multus-additional-cni-plugins-bcfwk"
Jul 29 14:01:11.636475 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635884    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/045bee14-dd3a-4494-9a05-ae72fcf00f6d-iptables-alerter-script\") pod \"iptables-alerter-mz4xm\" (UID: \"045bee14-dd3a-4494-9a05-ae72fcf00f6d\") " pod="openshift-network-operator/iptables-alerter-mz4xm"
Jul 29 14:01:11.636475 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635910    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-host-var-lib-kubelet\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.636475 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635934    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-multus-daemon-config\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.636475 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635962    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66d5t\" (UniqueName: \"kubernetes.io/projected/93be82f5-6d56-4bd3-9822-bef13d5738e1-kube-api-access-66d5t\") pod \"network-metrics-daemon-cwxl9\" (UID: \"93be82f5-6d56-4bd3-9822-bef13d5738e1\") " pod="openshift-multus/network-metrics-daemon-cwxl9"
Jul 29 14:01:11.637236 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.635985    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b0c6da59-c35d-4d20-8346-3dd5afc3a8f3-original-pull-secret\") pod \"global-pull-secret-syncer-84fpq\" (UID: \"b0c6da59-c35d-4d20-8346-3dd5afc3a8f3\") " pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:01:11.637236 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.636009    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/24dbef56-b5b2-49e3-b2d6-e3ea78eeb731-os-release\") pod \"multus-additional-cni-plugins-bcfwk\" (UID: \"24dbef56-b5b2-49e3-b2d6-e3ea78eeb731\") " pod="openshift-multus/multus-additional-cni-plugins-bcfwk"
Jul 29 14:01:11.637236 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.636032    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/24dbef56-b5b2-49e3-b2d6-e3ea78eeb731-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bcfwk\" (UID: \"24dbef56-b5b2-49e3-b2d6-e3ea78eeb731\") " pod="openshift-multus/multus-additional-cni-plugins-bcfwk"
Jul 29 14:01:11.637236 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.636084    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-multus-cni-dir\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.637236 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:11.636098    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 14:01:11.637236 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:11.636132    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0c6da59-c35d-4d20-8346-3dd5afc3a8f3-original-pull-secret podName:b0c6da59-c35d-4d20-8346-3dd5afc3a8f3 nodeName:}" failed. No retries permitted until 2026-07-29 14:01:12.136120598 +0000 UTC m=+2.026791914 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b0c6da59-c35d-4d20-8346-3dd5afc3a8f3-original-pull-secret") pod "global-pull-secret-syncer-84fpq" (UID: "b0c6da59-c35d-4d20-8346-3dd5afc3a8f3") : object "kube-system"/"original-pull-secret" not registered
Jul 29 14:01:11.637236 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.636108    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-os-release\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.637236 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.636182    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-host-var-lib-cni-bin\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.637236 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.636224    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-host-slash\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.637236 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.636244    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-run-ovn\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.637236 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.636261    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d6541ac6-8d3b-4175-9881-3e5e82d72429-etc-modprobe-d\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.637236 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.636295    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4p7r\" (UniqueName: \"kubernetes.io/projected/d6541ac6-8d3b-4175-9881-3e5e82d72429-kube-api-access-m4p7r\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.637236 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.636326    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/045bee14-dd3a-4494-9a05-ae72fcf00f6d-host-slash\") pod \"iptables-alerter-mz4xm\" (UID: \"045bee14-dd3a-4494-9a05-ae72fcf00f6d\") " pod="openshift-network-operator/iptables-alerter-mz4xm"
Jul 29 14:01:11.637236 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.636353    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-hostroot\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.637236 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.636381    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-host-run-netns\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.637236 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.636405    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-run-openvswitch\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.637686 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.636435    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-host-cni-netd\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.637686 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.636471    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.637686 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.636497    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-env-overrides\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.637686 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.636521    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-ovnkube-script-lib\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.637686 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.636549    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnwlt\" (UniqueName: \"kubernetes.io/projected/6d24b8b8-c850-495d-b0f7-ca7ea53a8a57-kube-api-access-tnwlt\") pod \"network-check-target-gbj56\" (UID: \"6d24b8b8-c850-495d-b0f7-ca7ea53a8a57\") " pod="openshift-network-diagnostics/network-check-target-gbj56"
Jul 29 14:01:11.637686 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.636662    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-host-run-k8s-cni-cncf-io\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.637686 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.636691    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-multus-conf-dir\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.637686 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.636714    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6541ac6-8d3b-4175-9881-3e5e82d72429-tmp\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.637686 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.636735    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2317c90b-86b0-4ce3-a455-a973befb1b11-host\") pod \"node-ca-nl5mz\" (UID: \"2317c90b-86b0-4ce3-a455-a973befb1b11\") " pod="openshift-image-registry/node-ca-nl5mz"
Jul 29 14:01:11.637686 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.636758    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/253b40f9-e444-40f9-a59c-1cd0f39c4dbe-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jlk7p\" (UID: \"253b40f9-e444-40f9-a59c-1cd0f39c4dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlk7p"
Jul 29 14:01:11.637686 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.636791    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22c32d1e-2781-499f-a092-2898013fb37e-tmp-dir\") pod \"node-resolver-d9ncc\" (UID: \"22c32d1e-2781-499f-a092-2898013fb37e\") " pod="openshift-dns/node-resolver-d9ncc"
Jul 29 14:01:11.637686 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.636862    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/24dbef56-b5b2-49e3-b2d6-e3ea78eeb731-system-cni-dir\") pod \"multus-additional-cni-plugins-bcfwk\" (UID: \"24dbef56-b5b2-49e3-b2d6-e3ea78eeb731\") " pod="openshift-multus/multus-additional-cni-plugins-bcfwk"
Jul 29 14:01:11.637686 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.636872    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/253b40f9-e444-40f9-a59c-1cd0f39c4dbe-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jlk7p\" (UID: \"253b40f9-e444-40f9-a59c-1cd0f39c4dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlk7p"
Jul 29 14:01:11.637686 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.636889    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqfmz\" (UniqueName: \"kubernetes.io/projected/045bee14-dd3a-4494-9a05-ae72fcf00f6d-kube-api-access-lqfmz\") pod \"iptables-alerter-mz4xm\" (UID: \"045bee14-dd3a-4494-9a05-ae72fcf00f6d\") " pod="openshift-network-operator/iptables-alerter-mz4xm"
Jul 29 14:01:11.637686 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.636914    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-var-lib-openvswitch\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.637686 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.636937    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-ovnkube-config\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.638133 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.636961    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d6541ac6-8d3b-4175-9881-3e5e82d72429-etc-sysctl-d\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.638133 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.636983    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d6541ac6-8d3b-4175-9881-3e5e82d72429-lib-modules\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.638133 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.637010    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/62eed460-c3d4-4d6e-a7ff-cf1290a96c53-konnectivity-ca\") pod \"konnectivity-agent-7xpvn\" (UID: \"62eed460-c3d4-4d6e-a7ff-cf1290a96c53\") " pod="kube-system/konnectivity-agent-7xpvn"
Jul 29 14:01:11.638133 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.637037    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/22c32d1e-2781-499f-a092-2898013fb37e-hosts-file\") pod \"node-resolver-d9ncc\" (UID: \"22c32d1e-2781-499f-a092-2898013fb37e\") " pod="openshift-dns/node-resolver-d9ncc"
Jul 29 14:01:11.638133 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.637097    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/22c32d1e-2781-499f-a092-2898013fb37e-hosts-file\") pod \"node-resolver-d9ncc\" (UID: \"22c32d1e-2781-499f-a092-2898013fb37e\") " pod="openshift-dns/node-resolver-d9ncc"
Jul 29 14:01:11.638133 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.637103    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/24dbef56-b5b2-49e3-b2d6-e3ea78eeb731-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bcfwk\" (UID: \"24dbef56-b5b2-49e3-b2d6-e3ea78eeb731\") " pod="openshift-multus/multus-additional-cni-plugins-bcfwk"
Jul 29 14:01:11.638133 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.637101    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22c32d1e-2781-499f-a092-2898013fb37e-tmp-dir\") pod \"node-resolver-d9ncc\" (UID: \"22c32d1e-2781-499f-a092-2898013fb37e\") " pod="openshift-dns/node-resolver-d9ncc"
Jul 29 14:01:11.638133 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.637122    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-cnibin\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.638133 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.637138    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-host-run-netns\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.638133 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.637155    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-host-run-multus-certs\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.638133 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.637177    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d6541ac6-8d3b-4175-9881-3e5e82d72429-sys\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.638133 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.637547    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/62eed460-c3d4-4d6e-a7ff-cf1290a96c53-konnectivity-ca\") pod \"konnectivity-agent-7xpvn\" (UID: \"62eed460-c3d4-4d6e-a7ff-cf1290a96c53\") " pod="kube-system/konnectivity-agent-7xpvn"
Jul 29 14:01:11.639101 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.639079    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/62eed460-c3d4-4d6e-a7ff-cf1290a96c53-agent-certs\") pod \"konnectivity-agent-7xpvn\" (UID: \"62eed460-c3d4-4d6e-a7ff-cf1290a96c53\") " pod="kube-system/konnectivity-agent-7xpvn"
Jul 29 14:01:11.642675 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:11.642660    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 14:01:11.642739 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:11.642677    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 14:01:11.642739 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:11.642686    2559 projected.go:196] Error preparing data for projected volume kube-api-access-tnwlt for pod openshift-network-diagnostics/network-check-target-gbj56: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 14:01:11.642739 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:11.642733    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d24b8b8-c850-495d-b0f7-ca7ea53a8a57-kube-api-access-tnwlt podName:6d24b8b8-c850-495d-b0f7-ca7ea53a8a57 nodeName:}" failed. No retries permitted until 2026-07-29 14:01:12.142720029 +0000 UTC m=+2.033391356 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tnwlt" (UniqueName: "kubernetes.io/projected/6d24b8b8-c850-495d-b0f7-ca7ea53a8a57-kube-api-access-tnwlt") pod "network-check-target-gbj56" (UID: "6d24b8b8-c850-495d-b0f7-ca7ea53a8a57") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 14:01:11.644859 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.644832    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtmq4\" (UniqueName: \"kubernetes.io/projected/22c32d1e-2781-499f-a092-2898013fb37e-kube-api-access-jtmq4\") pod \"node-resolver-d9ncc\" (UID: \"22c32d1e-2781-499f-a092-2898013fb37e\") " pod="openshift-dns/node-resolver-d9ncc"
Jul 29 14:01:11.644859 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.644843    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66d5t\" (UniqueName: \"kubernetes.io/projected/93be82f5-6d56-4bd3-9822-bef13d5738e1-kube-api-access-66d5t\") pod \"network-metrics-daemon-cwxl9\" (UID: \"93be82f5-6d56-4bd3-9822-bef13d5738e1\") " pod="openshift-multus/network-metrics-daemon-cwxl9"
Jul 29 14:01:11.645235 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.645217    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-872pb\" (UniqueName: \"kubernetes.io/projected/253b40f9-e444-40f9-a59c-1cd0f39c4dbe-kube-api-access-872pb\") pod \"aws-ebs-csi-driver-node-jlk7p\" (UID: \"253b40f9-e444-40f9-a59c-1cd0f39c4dbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlk7p"
Jul 29 14:01:11.738108 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738080    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-cni-binary-copy\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.738277 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738116    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-etc-kubernetes\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.738277 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738163    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-etc-kubernetes\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.738277 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738217    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-run-systemd\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.738277 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738249    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-etc-openvswitch\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.738277 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738269    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2317c90b-86b0-4ce3-a455-a973befb1b11-serviceca\") pod \"node-ca-nl5mz\" (UID: \"2317c90b-86b0-4ce3-a455-a973befb1b11\") " pod="openshift-image-registry/node-ca-nl5mz"
Jul 29 14:01:11.738521 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738286    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbctv\" (UniqueName: \"kubernetes.io/projected/2317c90b-86b0-4ce3-a455-a973befb1b11-kube-api-access-wbctv\") pod \"node-ca-nl5mz\" (UID: \"2317c90b-86b0-4ce3-a455-a973befb1b11\") " pod="openshift-image-registry/node-ca-nl5mz"
Jul 29 14:01:11.738521 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738288    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-run-systemd\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.738521 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738306    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-multus-socket-dir-parent\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.738521 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738326    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-systemd-units\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.738521 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738341    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4d8rw\" (UniqueName: \"kubernetes.io/projected/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-kube-api-access-4d8rw\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.738521 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738408    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-multus-socket-dir-parent\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.738521 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738415    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-systemd-units\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.738521 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738437    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-etc-openvswitch\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.738521 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738451    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d6541ac6-8d3b-4175-9881-3e5e82d72429-run\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.738521 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738484    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/24dbef56-b5b2-49e3-b2d6-e3ea78eeb731-cni-binary-copy\") pod \"multus-additional-cni-plugins-bcfwk\" (UID: \"24dbef56-b5b2-49e3-b2d6-e3ea78eeb731\") " pod="openshift-multus/multus-additional-cni-plugins-bcfwk"
Jul 29 14:01:11.738521 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738504    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d6541ac6-8d3b-4175-9881-3e5e82d72429-run\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.738521 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738510    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6n5tp\" (UniqueName: \"kubernetes.io/projected/24dbef56-b5b2-49e3-b2d6-e3ea78eeb731-kube-api-access-6n5tp\") pod \"multus-additional-cni-plugins-bcfwk\" (UID: \"24dbef56-b5b2-49e3-b2d6-e3ea78eeb731\") " pod="openshift-multus/multus-additional-cni-plugins-bcfwk"
Jul 29 14:01:11.739072 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738544    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/24dbef56-b5b2-49e3-b2d6-e3ea78eeb731-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bcfwk\" (UID: \"24dbef56-b5b2-49e3-b2d6-e3ea78eeb731\") " pod="openshift-multus/multus-additional-cni-plugins-bcfwk"
Jul 29 14:01:11.739072 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738572    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-log-socket\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.739072 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738596    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d6541ac6-8d3b-4175-9881-3e5e82d72429-etc-tuned\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.739072 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738620    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-host-var-lib-cni-multus\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.739072 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738642    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dh7l\" (UniqueName: \"kubernetes.io/projected/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-kube-api-access-7dh7l\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.739072 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738665    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-host-kubelet\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.739072 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738670    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-log-socket\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.739072 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738693    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-host-run-ovn-kubernetes\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.739072 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738704    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-host-var-lib-cni-multus\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.739072 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738716    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d6541ac6-8d3b-4175-9881-3e5e82d72429-etc-sysctl-conf\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.739072 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738664    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-cni-binary-copy\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.739072 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738724    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-host-kubelet\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.739072 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738740    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-ovn-node-metrics-cert\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.739072 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738768    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/24dbef56-b5b2-49e3-b2d6-e3ea78eeb731-cnibin\") pod \"multus-additional-cni-plugins-bcfwk\" (UID: \"24dbef56-b5b2-49e3-b2d6-e3ea78eeb731\") " pod="openshift-multus/multus-additional-cni-plugins-bcfwk"
Jul 29 14:01:11.739072 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738776    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-host-run-ovn-kubernetes\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.739072 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738779    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2317c90b-86b0-4ce3-a455-a973befb1b11-serviceca\") pod \"node-ca-nl5mz\" (UID: \"2317c90b-86b0-4ce3-a455-a973befb1b11\") " pod="openshift-image-registry/node-ca-nl5mz"
Jul 29 14:01:11.739072 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738925    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d6541ac6-8d3b-4175-9881-3e5e82d72429-etc-sysctl-conf\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.739878 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738791    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/045bee14-dd3a-4494-9a05-ae72fcf00f6d-iptables-alerter-script\") pod \"iptables-alerter-mz4xm\" (UID: \"045bee14-dd3a-4494-9a05-ae72fcf00f6d\") " pod="openshift-network-operator/iptables-alerter-mz4xm"
Jul 29 14:01:11.739878 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738940    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/24dbef56-b5b2-49e3-b2d6-e3ea78eeb731-cnibin\") pod \"multus-additional-cni-plugins-bcfwk\" (UID: \"24dbef56-b5b2-49e3-b2d6-e3ea78eeb731\") " pod="openshift-multus/multus-additional-cni-plugins-bcfwk"
Jul 29 14:01:11.739878 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738969    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-host-var-lib-kubelet\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.739878 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.738994    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-multus-daemon-config\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.739878 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739032    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/24dbef56-b5b2-49e3-b2d6-e3ea78eeb731-os-release\") pod \"multus-additional-cni-plugins-bcfwk\" (UID: \"24dbef56-b5b2-49e3-b2d6-e3ea78eeb731\") " pod="openshift-multus/multus-additional-cni-plugins-bcfwk"
Jul 29 14:01:11.739878 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739048    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/24dbef56-b5b2-49e3-b2d6-e3ea78eeb731-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bcfwk\" (UID: \"24dbef56-b5b2-49e3-b2d6-e3ea78eeb731\") " pod="openshift-multus/multus-additional-cni-plugins-bcfwk"
Jul 29 14:01:11.739878 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739048    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-host-var-lib-kubelet\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.739878 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739079    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/24dbef56-b5b2-49e3-b2d6-e3ea78eeb731-cni-binary-copy\") pod \"multus-additional-cni-plugins-bcfwk\" (UID: \"24dbef56-b5b2-49e3-b2d6-e3ea78eeb731\") " pod="openshift-multus/multus-additional-cni-plugins-bcfwk"
Jul 29 14:01:11.739878 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739085    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-multus-cni-dir\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.739878 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739126    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-multus-cni-dir\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.739878 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739122    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/24dbef56-b5b2-49e3-b2d6-e3ea78eeb731-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bcfwk\" (UID: \"24dbef56-b5b2-49e3-b2d6-e3ea78eeb731\") " pod="openshift-multus/multus-additional-cni-plugins-bcfwk"
Jul 29 14:01:11.739878 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739131    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/24dbef56-b5b2-49e3-b2d6-e3ea78eeb731-os-release\") pod \"multus-additional-cni-plugins-bcfwk\" (UID: \"24dbef56-b5b2-49e3-b2d6-e3ea78eeb731\") " pod="openshift-multus/multus-additional-cni-plugins-bcfwk"
Jul 29 14:01:11.739878 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739135    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-os-release\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.739878 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739160    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-host-var-lib-cni-bin\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.739878 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739185    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-host-slash\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.739878 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739195    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-os-release\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.739878 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739209    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-run-ovn\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.740682 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739243    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/24dbef56-b5b2-49e3-b2d6-e3ea78eeb731-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bcfwk\" (UID: \"24dbef56-b5b2-49e3-b2d6-e3ea78eeb731\") " pod="openshift-multus/multus-additional-cni-plugins-bcfwk"
Jul 29 14:01:11.740682 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739244    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d6541ac6-8d3b-4175-9881-3e5e82d72429-etc-modprobe-d\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.740682 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739274    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-host-slash\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.740682 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739247    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-host-var-lib-cni-bin\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.740682 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739276    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4p7r\" (UniqueName: \"kubernetes.io/projected/d6541ac6-8d3b-4175-9881-3e5e82d72429-kube-api-access-m4p7r\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.740682 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739300    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-run-ovn\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.740682 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739313    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/045bee14-dd3a-4494-9a05-ae72fcf00f6d-host-slash\") pod \"iptables-alerter-mz4xm\" (UID: \"045bee14-dd3a-4494-9a05-ae72fcf00f6d\") " pod="openshift-network-operator/iptables-alerter-mz4xm"
Jul 29 14:01:11.740682 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739330    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d6541ac6-8d3b-4175-9881-3e5e82d72429-etc-modprobe-d\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.740682 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739336    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/045bee14-dd3a-4494-9a05-ae72fcf00f6d-iptables-alerter-script\") pod \"iptables-alerter-mz4xm\" (UID: \"045bee14-dd3a-4494-9a05-ae72fcf00f6d\") " pod="openshift-network-operator/iptables-alerter-mz4xm"
Jul 29 14:01:11.740682 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739345    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/045bee14-dd3a-4494-9a05-ae72fcf00f6d-host-slash\") pod \"iptables-alerter-mz4xm\" (UID: \"045bee14-dd3a-4494-9a05-ae72fcf00f6d\") " pod="openshift-network-operator/iptables-alerter-mz4xm"
Jul 29 14:01:11.740682 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739356    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-hostroot\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.740682 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739380    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-host-run-netns\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.740682 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739401    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-host-run-netns\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.740682 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739404    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-run-openvswitch\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.740682 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739382    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-hostroot\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.740682 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739443    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-run-openvswitch\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.740682 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739443    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-host-cni-netd\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.740682 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739469    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-host-cni-netd\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.741369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739472    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.741369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739498    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-env-overrides\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.741369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739521    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-ovnkube-script-lib\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.741369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739558    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-host-run-k8s-cni-cncf-io\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.741369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739558    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.741369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739605    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-host-run-k8s-cni-cncf-io\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.741369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739630    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-multus-conf-dir\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.741369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739664    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6541ac6-8d3b-4175-9881-3e5e82d72429-tmp\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.741369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739682    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2317c90b-86b0-4ce3-a455-a973befb1b11-host\") pod \"node-ca-nl5mz\" (UID: \"2317c90b-86b0-4ce3-a455-a973befb1b11\") " pod="openshift-image-registry/node-ca-nl5mz"
Jul 29 14:01:11.741369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739704    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/24dbef56-b5b2-49e3-b2d6-e3ea78eeb731-system-cni-dir\") pod \"multus-additional-cni-plugins-bcfwk\" (UID: \"24dbef56-b5b2-49e3-b2d6-e3ea78eeb731\") " pod="openshift-multus/multus-additional-cni-plugins-bcfwk"
Jul 29 14:01:11.741369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739712    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-multus-conf-dir\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.741369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739727    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqfmz\" (UniqueName: \"kubernetes.io/projected/045bee14-dd3a-4494-9a05-ae72fcf00f6d-kube-api-access-lqfmz\") pod \"iptables-alerter-mz4xm\" (UID: \"045bee14-dd3a-4494-9a05-ae72fcf00f6d\") " pod="openshift-network-operator/iptables-alerter-mz4xm"
Jul 29 14:01:11.741369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739753    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-var-lib-openvswitch\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.741369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739758    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2317c90b-86b0-4ce3-a455-a973befb1b11-host\") pod \"node-ca-nl5mz\" (UID: \"2317c90b-86b0-4ce3-a455-a973befb1b11\") " pod="openshift-image-registry/node-ca-nl5mz"
Jul 29 14:01:11.741369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739806    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/24dbef56-b5b2-49e3-b2d6-e3ea78eeb731-system-cni-dir\") pod \"multus-additional-cni-plugins-bcfwk\" (UID: \"24dbef56-b5b2-49e3-b2d6-e3ea78eeb731\") " pod="openshift-multus/multus-additional-cni-plugins-bcfwk"
Jul 29 14:01:11.741369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739842    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-var-lib-openvswitch\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.741369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739851    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-multus-daemon-config\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.741829 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739870    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-ovnkube-config\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.741829 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739898    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d6541ac6-8d3b-4175-9881-3e5e82d72429-etc-sysctl-d\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.741829 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739936    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d6541ac6-8d3b-4175-9881-3e5e82d72429-lib-modules\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.741829 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739963    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/24dbef56-b5b2-49e3-b2d6-e3ea78eeb731-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bcfwk\" (UID: \"24dbef56-b5b2-49e3-b2d6-e3ea78eeb731\") " pod="openshift-multus/multus-additional-cni-plugins-bcfwk"
Jul 29 14:01:11.741829 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.739987    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-cnibin\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.741829 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.740009    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-host-run-netns\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.741829 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.740046    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-env-overrides\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.741829 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.740076    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-host-run-multus-certs\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.741829 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.740112    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d6541ac6-8d3b-4175-9881-3e5e82d72429-sys\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.741829 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.740119    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-host-run-multus-certs\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.741829 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.740130    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d6541ac6-8d3b-4175-9881-3e5e82d72429-lib-modules\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.741829 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.740137    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-ovnkube-script-lib\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.741829 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.740154    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-system-cni-dir\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.741829 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.740180    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-host-cni-bin\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.741829 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.740201    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d6541ac6-8d3b-4175-9881-3e5e82d72429-etc-sysconfig\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.741829 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.740223    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d6541ac6-8d3b-4175-9881-3e5e82d72429-etc-sysctl-d\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.741829 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.740251    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d6541ac6-8d3b-4175-9881-3e5e82d72429-sys\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.741829 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.740202    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-cnibin\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.742356 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.740271    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d6541ac6-8d3b-4175-9881-3e5e82d72429-etc-sysconfig\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.742356 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.740272    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-system-cni-dir\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.742356 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.740225    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6541ac6-8d3b-4175-9881-3e5e82d72429-etc-kubernetes\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.742356 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.740312    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6541ac6-8d3b-4175-9881-3e5e82d72429-etc-kubernetes\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.742356 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.740353    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-host-cni-bin\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.742356 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.740376    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-ovnkube-config\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.742356 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.740384    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d6541ac6-8d3b-4175-9881-3e5e82d72429-etc-systemd\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.742356 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.740390    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-host-run-netns\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.742356 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.740458    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d6541ac6-8d3b-4175-9881-3e5e82d72429-etc-systemd\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.742356 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.740502    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-node-log\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.742356 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.740536    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6541ac6-8d3b-4175-9881-3e5e82d72429-var-lib-kubelet\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.742356 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.740573    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6541ac6-8d3b-4175-9881-3e5e82d72429-var-lib-kubelet\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.742356 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.740578    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-node-log\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.742356 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.740585    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d6541ac6-8d3b-4175-9881-3e5e82d72429-host\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.742356 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.740612    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d6541ac6-8d3b-4175-9881-3e5e82d72429-host\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.742356 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.741018    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/24dbef56-b5b2-49e3-b2d6-e3ea78eeb731-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bcfwk\" (UID: \"24dbef56-b5b2-49e3-b2d6-e3ea78eeb731\") " pod="openshift-multus/multus-additional-cni-plugins-bcfwk"
Jul 29 14:01:11.742356 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.741267    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d6541ac6-8d3b-4175-9881-3e5e82d72429-etc-tuned\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.742356 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.742036    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-ovn-node-metrics-cert\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.742823 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.742243    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6541ac6-8d3b-4175-9881-3e5e82d72429-tmp\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.746930 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.746898    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dh7l\" (UniqueName: \"kubernetes.io/projected/44f77fb3-96c6-42ad-b76f-f605ab05c9d6-kube-api-access-7dh7l\") pod \"multus-hbw9p\" (UID: \"44f77fb3-96c6-42ad-b76f-f605ab05c9d6\") " pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.747519 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.747494    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbctv\" (UniqueName: \"kubernetes.io/projected/2317c90b-86b0-4ce3-a455-a973befb1b11-kube-api-access-wbctv\") pod \"node-ca-nl5mz\" (UID: \"2317c90b-86b0-4ce3-a455-a973befb1b11\") " pod="openshift-image-registry/node-ca-nl5mz"
Jul 29 14:01:11.747519 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.747501    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6n5tp\" (UniqueName: \"kubernetes.io/projected/24dbef56-b5b2-49e3-b2d6-e3ea78eeb731-kube-api-access-6n5tp\") pod \"multus-additional-cni-plugins-bcfwk\" (UID: \"24dbef56-b5b2-49e3-b2d6-e3ea78eeb731\") " pod="openshift-multus/multus-additional-cni-plugins-bcfwk"
Jul 29 14:01:11.747778 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.747760    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d8rw\" (UniqueName: \"kubernetes.io/projected/b93f6faa-c8bf-43e9-94e0-b3dcd97b7296-kube-api-access-4d8rw\") pod \"ovnkube-node-hdw8l\" (UID: \"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.747820 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.747791    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqfmz\" (UniqueName: \"kubernetes.io/projected/045bee14-dd3a-4494-9a05-ae72fcf00f6d-kube-api-access-lqfmz\") pod \"iptables-alerter-mz4xm\" (UID: \"045bee14-dd3a-4494-9a05-ae72fcf00f6d\") " pod="openshift-network-operator/iptables-alerter-mz4xm"
Jul 29 14:01:11.748117 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.748100    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4p7r\" (UniqueName: \"kubernetes.io/projected/d6541ac6-8d3b-4175-9881-3e5e82d72429-kube-api-access-m4p7r\") pod \"tuned-szq55\" (UID: \"d6541ac6-8d3b-4175-9881-3e5e82d72429\") " pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.859726 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.859651    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7xpvn"
Jul 29 14:01:11.867290 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:11.867264    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62eed460_c3d4_4d6e_a7ff_cf1290a96c53.slice/crio-afd3e9cfdcda4e749ed88a6ae7515209b0c88e91d1e8e0111e0afd19f57d70a4 WatchSource:0}: Error finding container afd3e9cfdcda4e749ed88a6ae7515209b0c88e91d1e8e0111e0afd19f57d70a4: Status 404 returned error can't find the container with id afd3e9cfdcda4e749ed88a6ae7515209b0c88e91d1e8e0111e0afd19f57d70a4
Jul 29 14:01:11.878705 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.878686    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlk7p"
Jul 29 14:01:11.885322 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.885298    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-d9ncc"
Jul 29 14:01:11.885837 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:11.885819    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod253b40f9_e444_40f9_a59c_1cd0f39c4dbe.slice/crio-d65d724cad7ad6cb9ac994a49e8e2cc8c40702c8561f6321e5458b8320e2b5e1 WatchSource:0}: Error finding container d65d724cad7ad6cb9ac994a49e8e2cc8c40702c8561f6321e5458b8320e2b5e1: Status 404 returned error can't find the container with id d65d724cad7ad6cb9ac994a49e8e2cc8c40702c8561f6321e5458b8320e2b5e1
Jul 29 14:01:11.890543 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.890522    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mz4xm"
Jul 29 14:01:11.891925 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:11.891906    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22c32d1e_2781_499f_a092_2898013fb37e.slice/crio-fedaa8889de6f74f3c114080e1f835b8dc8a10e88e1f14bf1ff87701e60c7fbb WatchSource:0}: Error finding container fedaa8889de6f74f3c114080e1f835b8dc8a10e88e1f14bf1ff87701e60c7fbb: Status 404 returned error can't find the container with id fedaa8889de6f74f3c114080e1f835b8dc8a10e88e1f14bf1ff87701e60c7fbb
Jul 29 14:01:11.897630 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.897607    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:11.897806 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:11.897787    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod045bee14_dd3a_4494_9a05_ae72fcf00f6d.slice/crio-8489225c94b9ddc57a246020b16c2e8191b4a3c0848b6bd6754f806382d2f469 WatchSource:0}: Error finding container 8489225c94b9ddc57a246020b16c2e8191b4a3c0848b6bd6754f806382d2f469: Status 404 returned error can't find the container with id 8489225c94b9ddc57a246020b16c2e8191b4a3c0848b6bd6754f806382d2f469
Jul 29 14:01:11.903149 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.903128    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-szq55"
Jul 29 14:01:11.905420 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:11.905401    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb93f6faa_c8bf_43e9_94e0_b3dcd97b7296.slice/crio-ecc2daf58d000da8c9fa3a8a9ccaa8c0fa2bf4d6d705ae7e18f5dece6e9f714f WatchSource:0}: Error finding container ecc2daf58d000da8c9fa3a8a9ccaa8c0fa2bf4d6d705ae7e18f5dece6e9f714f: Status 404 returned error can't find the container with id ecc2daf58d000da8c9fa3a8a9ccaa8c0fa2bf4d6d705ae7e18f5dece6e9f714f
Jul 29 14:01:11.907422 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.907402    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nl5mz"
Jul 29 14:01:11.912116 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:11.912097    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6541ac6_8d3b_4175_9881_3e5e82d72429.slice/crio-043c83d28014a348ad3c22b01c7dc0309012edef83f4cda7d9faeae426a15f00 WatchSource:0}: Error finding container 043c83d28014a348ad3c22b01c7dc0309012edef83f4cda7d9faeae426a15f00: Status 404 returned error can't find the container with id 043c83d28014a348ad3c22b01c7dc0309012edef83f4cda7d9faeae426a15f00
Jul 29 14:01:11.913292 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.913273    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bcfwk"
Jul 29 14:01:11.914896 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:11.914878    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2317c90b_86b0_4ce3_a455_a973befb1b11.slice/crio-bae21f96062aba50a3e73dbbd03abaa3bd0c7bc1c4303dcf5fe94668dafa80a8 WatchSource:0}: Error finding container bae21f96062aba50a3e73dbbd03abaa3bd0c7bc1c4303dcf5fe94668dafa80a8: Status 404 returned error can't find the container with id bae21f96062aba50a3e73dbbd03abaa3bd0c7bc1c4303dcf5fe94668dafa80a8
Jul 29 14:01:11.917908 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:11.917890    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hbw9p"
Jul 29 14:01:11.920986 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:11.920959    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24dbef56_b5b2_49e3_b2d6_e3ea78eeb731.slice/crio-7b14a26bbe1b8d8b7ba29e61e8ccdec518228c3c1d90c954112dd28235d3bf45 WatchSource:0}: Error finding container 7b14a26bbe1b8d8b7ba29e61e8ccdec518228c3c1d90c954112dd28235d3bf45: Status 404 returned error can't find the container with id 7b14a26bbe1b8d8b7ba29e61e8ccdec518228c3c1d90c954112dd28235d3bf45
Jul 29 14:01:11.925246 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:11.925227    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44f77fb3_96c6_42ad_b76f_f605ab05c9d6.slice/crio-4aa49cbfdc7680f1e869e0c2dee5603ec697ca0582f959be12752182c42e90e1 WatchSource:0}: Error finding container 4aa49cbfdc7680f1e869e0c2dee5603ec697ca0582f959be12752182c42e90e1: Status 404 returned error can't find the container with id 4aa49cbfdc7680f1e869e0c2dee5603ec697ca0582f959be12752182c42e90e1
Jul 29 14:01:12.143987 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:12.143907    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93be82f5-6d56-4bd3-9822-bef13d5738e1-metrics-certs\") pod \"network-metrics-daemon-cwxl9\" (UID: \"93be82f5-6d56-4bd3-9822-bef13d5738e1\") " pod="openshift-multus/network-metrics-daemon-cwxl9"
Jul 29 14:01:12.143987 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:12.143974    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b0c6da59-c35d-4d20-8346-3dd5afc3a8f3-original-pull-secret\") pod \"global-pull-secret-syncer-84fpq\" (UID: \"b0c6da59-c35d-4d20-8346-3dd5afc3a8f3\") " pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:01:12.144309 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:12.144001    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnwlt\" (UniqueName: \"kubernetes.io/projected/6d24b8b8-c850-495d-b0f7-ca7ea53a8a57-kube-api-access-tnwlt\") pod \"network-check-target-gbj56\" (UID: \"6d24b8b8-c850-495d-b0f7-ca7ea53a8a57\") " pod="openshift-network-diagnostics/network-check-target-gbj56"
Jul 29 14:01:12.144309 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:12.144132    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 14:01:12.144309 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:12.144148    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 14:01:12.144309 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:12.144160    2559 projected.go:196] Error preparing data for projected volume kube-api-access-tnwlt for pod openshift-network-diagnostics/network-check-target-gbj56: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 14:01:12.144309 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:12.144203    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d24b8b8-c850-495d-b0f7-ca7ea53a8a57-kube-api-access-tnwlt podName:6d24b8b8-c850-495d-b0f7-ca7ea53a8a57 nodeName:}" failed. No retries permitted until 2026-07-29 14:01:13.144190152 +0000 UTC m=+3.034861466 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tnwlt" (UniqueName: "kubernetes.io/projected/6d24b8b8-c850-495d-b0f7-ca7ea53a8a57-kube-api-access-tnwlt") pod "network-check-target-gbj56" (UID: "6d24b8b8-c850-495d-b0f7-ca7ea53a8a57") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 14:01:12.144588 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:12.144415    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 14:01:12.144588 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:12.144472    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0c6da59-c35d-4d20-8346-3dd5afc3a8f3-original-pull-secret podName:b0c6da59-c35d-4d20-8346-3dd5afc3a8f3 nodeName:}" failed. No retries permitted until 2026-07-29 14:01:13.144457901 +0000 UTC m=+3.035129228 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b0c6da59-c35d-4d20-8346-3dd5afc3a8f3-original-pull-secret") pod "global-pull-secret-syncer-84fpq" (UID: "b0c6da59-c35d-4d20-8346-3dd5afc3a8f3") : object "kube-system"/"original-pull-secret" not registered
Jul 29 14:01:12.144588 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:12.144534    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 14:01:12.144588 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:12.144578    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93be82f5-6d56-4bd3-9822-bef13d5738e1-metrics-certs podName:93be82f5-6d56-4bd3-9822-bef13d5738e1 nodeName:}" failed. No retries permitted until 2026-07-29 14:01:13.144565506 +0000 UTC m=+3.035236833 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93be82f5-6d56-4bd3-9822-bef13d5738e1-metrics-certs") pod "network-metrics-daemon-cwxl9" (UID: "93be82f5-6d56-4bd3-9822-bef13d5738e1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 14:01:12.370777 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:12.370552    2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 14:01:12.572514 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:12.572398    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 13:56:11 +0000 UTC" deadline="2028-05-12 21:52:46.435901124 +0000 UTC"
Jul 29 14:01:12.572514 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:12.572466    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15679h51m33.863439962s"
Jul 29 14:01:12.633404 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:12.633372    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbj56"
Jul 29 14:01:12.633564 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:12.633519    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbj56" podUID="6d24b8b8-c850-495d-b0f7-ca7ea53a8a57"
Jul 29 14:01:12.660967 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:12.660329    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-szq55" event={"ID":"d6541ac6-8d3b-4175-9881-3e5e82d72429","Type":"ContainerStarted","Data":"043c83d28014a348ad3c22b01c7dc0309012edef83f4cda7d9faeae426a15f00"}
Jul 29 14:01:12.671370 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:12.671334    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mz4xm" event={"ID":"045bee14-dd3a-4494-9a05-ae72fcf00f6d","Type":"ContainerStarted","Data":"8489225c94b9ddc57a246020b16c2e8191b4a3c0848b6bd6754f806382d2f469"}
Jul 29 14:01:12.676575 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:12.676545    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-d9ncc" event={"ID":"22c32d1e-2781-499f-a092-2898013fb37e","Type":"ContainerStarted","Data":"fedaa8889de6f74f3c114080e1f835b8dc8a10e88e1f14bf1ff87701e60c7fbb"}
Jul 29 14:01:12.688602 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:12.688570    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hbw9p" event={"ID":"44f77fb3-96c6-42ad-b76f-f605ab05c9d6","Type":"ContainerStarted","Data":"4aa49cbfdc7680f1e869e0c2dee5603ec697ca0582f959be12752182c42e90e1"}
Jul 29 14:01:12.692217 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:12.692187    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bcfwk" event={"ID":"24dbef56-b5b2-49e3-b2d6-e3ea78eeb731","Type":"ContainerStarted","Data":"7b14a26bbe1b8d8b7ba29e61e8ccdec518228c3c1d90c954112dd28235d3bf45"}
Jul 29 14:01:12.704402 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:12.704372    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nl5mz" event={"ID":"2317c90b-86b0-4ce3-a455-a973befb1b11","Type":"ContainerStarted","Data":"bae21f96062aba50a3e73dbbd03abaa3bd0c7bc1c4303dcf5fe94668dafa80a8"}
Jul 29 14:01:12.717252 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:12.717223    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l" event={"ID":"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296","Type":"ContainerStarted","Data":"ecc2daf58d000da8c9fa3a8a9ccaa8c0fa2bf4d6d705ae7e18f5dece6e9f714f"}
Jul 29 14:01:12.725513 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:12.725483    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlk7p" event={"ID":"253b40f9-e444-40f9-a59c-1cd0f39c4dbe","Type":"ContainerStarted","Data":"d65d724cad7ad6cb9ac994a49e8e2cc8c40702c8561f6321e5458b8320e2b5e1"}
Jul 29 14:01:12.741632 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:12.741598    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7xpvn" event={"ID":"62eed460-c3d4-4d6e-a7ff-cf1290a96c53","Type":"ContainerStarted","Data":"afd3e9cfdcda4e749ed88a6ae7515209b0c88e91d1e8e0111e0afd19f57d70a4"}
Jul 29 14:01:12.905805 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:12.905723    2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 14:01:13.151891 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:13.151858    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b0c6da59-c35d-4d20-8346-3dd5afc3a8f3-original-pull-secret\") pod \"global-pull-secret-syncer-84fpq\" (UID: \"b0c6da59-c35d-4d20-8346-3dd5afc3a8f3\") " pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:01:13.152082 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:13.151919    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnwlt\" (UniqueName: \"kubernetes.io/projected/6d24b8b8-c850-495d-b0f7-ca7ea53a8a57-kube-api-access-tnwlt\") pod \"network-check-target-gbj56\" (UID: \"6d24b8b8-c850-495d-b0f7-ca7ea53a8a57\") " pod="openshift-network-diagnostics/network-check-target-gbj56"
Jul 29 14:01:13.152082 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:13.151959    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93be82f5-6d56-4bd3-9822-bef13d5738e1-metrics-certs\") pod \"network-metrics-daemon-cwxl9\" (UID: \"93be82f5-6d56-4bd3-9822-bef13d5738e1\") " pod="openshift-multus/network-metrics-daemon-cwxl9"
Jul 29 14:01:13.152210 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:13.152143    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 14:01:13.152210 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:13.152208    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93be82f5-6d56-4bd3-9822-bef13d5738e1-metrics-certs podName:93be82f5-6d56-4bd3-9822-bef13d5738e1 nodeName:}" failed. No retries permitted until 2026-07-29 14:01:15.152187648 +0000 UTC m=+5.042858978 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93be82f5-6d56-4bd3-9822-bef13d5738e1-metrics-certs") pod "network-metrics-daemon-cwxl9" (UID: "93be82f5-6d56-4bd3-9822-bef13d5738e1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 14:01:13.153576 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:13.152588    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 14:01:13.153576 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:13.152639    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0c6da59-c35d-4d20-8346-3dd5afc3a8f3-original-pull-secret podName:b0c6da59-c35d-4d20-8346-3dd5afc3a8f3 nodeName:}" failed. No retries permitted until 2026-07-29 14:01:15.152624293 +0000 UTC m=+5.043295625 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b0c6da59-c35d-4d20-8346-3dd5afc3a8f3-original-pull-secret") pod "global-pull-secret-syncer-84fpq" (UID: "b0c6da59-c35d-4d20-8346-3dd5afc3a8f3") : object "kube-system"/"original-pull-secret" not registered
Jul 29 14:01:13.153576 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:13.152718    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 14:01:13.153576 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:13.152731    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 14:01:13.153576 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:13.152743    2559 projected.go:196] Error preparing data for projected volume kube-api-access-tnwlt for pod openshift-network-diagnostics/network-check-target-gbj56: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 14:01:13.153576 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:13.152777    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d24b8b8-c850-495d-b0f7-ca7ea53a8a57-kube-api-access-tnwlt podName:6d24b8b8-c850-495d-b0f7-ca7ea53a8a57 nodeName:}" failed. No retries permitted until 2026-07-29 14:01:15.152766143 +0000 UTC m=+5.043437456 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tnwlt" (UniqueName: "kubernetes.io/projected/6d24b8b8-c850-495d-b0f7-ca7ea53a8a57-kube-api-access-tnwlt") pod "network-check-target-gbj56" (UID: "6d24b8b8-c850-495d-b0f7-ca7ea53a8a57") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 14:01:13.572791 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:13.572702    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 13:56:11 +0000 UTC" deadline="2028-03-16 12:46:20.184197659 +0000 UTC"
Jul 29 14:01:13.572791 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:13.572755    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14302h45m6.61144899s"
Jul 29 14:01:13.631681 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:13.631001    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwxl9"
Jul 29 14:01:13.631681 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:13.631154    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwxl9" podUID="93be82f5-6d56-4bd3-9822-bef13d5738e1"
Jul 29 14:01:13.631681 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:13.631530    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:01:13.631681 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:13.631626    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84fpq" podUID="b0c6da59-c35d-4d20-8346-3dd5afc3a8f3"
Jul 29 14:01:13.695252 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:13.695036    2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 14:01:14.631090 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:14.631033    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbj56"
Jul 29 14:01:14.631563 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:14.631183    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbj56" podUID="6d24b8b8-c850-495d-b0f7-ca7ea53a8a57"
Jul 29 14:01:15.177929 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:15.177891    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b0c6da59-c35d-4d20-8346-3dd5afc3a8f3-original-pull-secret\") pod \"global-pull-secret-syncer-84fpq\" (UID: \"b0c6da59-c35d-4d20-8346-3dd5afc3a8f3\") " pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:01:15.178127 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:15.177954    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnwlt\" (UniqueName: \"kubernetes.io/projected/6d24b8b8-c850-495d-b0f7-ca7ea53a8a57-kube-api-access-tnwlt\") pod \"network-check-target-gbj56\" (UID: \"6d24b8b8-c850-495d-b0f7-ca7ea53a8a57\") " pod="openshift-network-diagnostics/network-check-target-gbj56"
Jul 29 14:01:15.178127 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:15.177996    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93be82f5-6d56-4bd3-9822-bef13d5738e1-metrics-certs\") pod \"network-metrics-daemon-cwxl9\" (UID: \"93be82f5-6d56-4bd3-9822-bef13d5738e1\") " pod="openshift-multus/network-metrics-daemon-cwxl9"
Jul 29 14:01:15.178127 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:15.178094    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 14:01:15.178127 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:15.178119    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 14:01:15.178349 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:15.178171    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0c6da59-c35d-4d20-8346-3dd5afc3a8f3-original-pull-secret podName:b0c6da59-c35d-4d20-8346-3dd5afc3a8f3 nodeName:}" failed. No retries permitted until 2026-07-29 14:01:19.178148227 +0000 UTC m=+9.068819559 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b0c6da59-c35d-4d20-8346-3dd5afc3a8f3-original-pull-secret") pod "global-pull-secret-syncer-84fpq" (UID: "b0c6da59-c35d-4d20-8346-3dd5afc3a8f3") : object "kube-system"/"original-pull-secret" not registered
Jul 29 14:01:15.178349 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:15.178193    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93be82f5-6d56-4bd3-9822-bef13d5738e1-metrics-certs podName:93be82f5-6d56-4bd3-9822-bef13d5738e1 nodeName:}" failed. No retries permitted until 2026-07-29 14:01:19.178182903 +0000 UTC m=+9.068854219 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93be82f5-6d56-4bd3-9822-bef13d5738e1-metrics-certs") pod "network-metrics-daemon-cwxl9" (UID: "93be82f5-6d56-4bd3-9822-bef13d5738e1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 14:01:15.178349 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:15.178246    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 14:01:15.178349 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:15.178263    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 14:01:15.178349 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:15.178276    2559 projected.go:196] Error preparing data for projected volume kube-api-access-tnwlt for pod openshift-network-diagnostics/network-check-target-gbj56: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 14:01:15.178349 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:15.178320    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d24b8b8-c850-495d-b0f7-ca7ea53a8a57-kube-api-access-tnwlt podName:6d24b8b8-c850-495d-b0f7-ca7ea53a8a57 nodeName:}" failed. No retries permitted until 2026-07-29 14:01:19.178308792 +0000 UTC m=+9.068980123 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tnwlt" (UniqueName: "kubernetes.io/projected/6d24b8b8-c850-495d-b0f7-ca7ea53a8a57-kube-api-access-tnwlt") pod "network-check-target-gbj56" (UID: "6d24b8b8-c850-495d-b0f7-ca7ea53a8a57") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 14:01:15.630991 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:15.630957    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:01:15.630991 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:15.630981    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwxl9"
Jul 29 14:01:15.631651 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:15.631150    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84fpq" podUID="b0c6da59-c35d-4d20-8346-3dd5afc3a8f3"
Jul 29 14:01:15.631651 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:15.631284    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwxl9" podUID="93be82f5-6d56-4bd3-9822-bef13d5738e1"
Jul 29 14:01:16.632954 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:16.632921    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbj56"
Jul 29 14:01:16.633472 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:16.633169    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbj56" podUID="6d24b8b8-c850-495d-b0f7-ca7ea53a8a57"
Jul 29 14:01:17.631302 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:17.631271    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:01:17.631478 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:17.631407    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84fpq" podUID="b0c6da59-c35d-4d20-8346-3dd5afc3a8f3"
Jul 29 14:01:17.631756 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:17.631732    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwxl9"
Jul 29 14:01:17.631853 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:17.631838    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwxl9" podUID="93be82f5-6d56-4bd3-9822-bef13d5738e1"
Jul 29 14:01:18.631572 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:18.631524    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbj56"
Jul 29 14:01:18.632090 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:18.631677    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbj56" podUID="6d24b8b8-c850-495d-b0f7-ca7ea53a8a57"
Jul 29 14:01:19.213821 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:19.213782    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b0c6da59-c35d-4d20-8346-3dd5afc3a8f3-original-pull-secret\") pod \"global-pull-secret-syncer-84fpq\" (UID: \"b0c6da59-c35d-4d20-8346-3dd5afc3a8f3\") " pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:01:19.213982 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:19.213847    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnwlt\" (UniqueName: \"kubernetes.io/projected/6d24b8b8-c850-495d-b0f7-ca7ea53a8a57-kube-api-access-tnwlt\") pod \"network-check-target-gbj56\" (UID: \"6d24b8b8-c850-495d-b0f7-ca7ea53a8a57\") " pod="openshift-network-diagnostics/network-check-target-gbj56"
Jul 29 14:01:19.213982 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:19.213889    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93be82f5-6d56-4bd3-9822-bef13d5738e1-metrics-certs\") pod \"network-metrics-daemon-cwxl9\" (UID: \"93be82f5-6d56-4bd3-9822-bef13d5738e1\") " pod="openshift-multus/network-metrics-daemon-cwxl9"
Jul 29 14:01:19.214187 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:19.214036    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 14:01:19.214187 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:19.214115    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93be82f5-6d56-4bd3-9822-bef13d5738e1-metrics-certs podName:93be82f5-6d56-4bd3-9822-bef13d5738e1 nodeName:}" failed. No retries permitted until 2026-07-29 14:01:27.214094408 +0000 UTC m=+17.104765724 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93be82f5-6d56-4bd3-9822-bef13d5738e1-metrics-certs") pod "network-metrics-daemon-cwxl9" (UID: "93be82f5-6d56-4bd3-9822-bef13d5738e1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 14:01:19.214537 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:19.214518    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 14:01:19.214611 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:19.214575    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0c6da59-c35d-4d20-8346-3dd5afc3a8f3-original-pull-secret podName:b0c6da59-c35d-4d20-8346-3dd5afc3a8f3 nodeName:}" failed. No retries permitted until 2026-07-29 14:01:27.214559519 +0000 UTC m=+17.105230836 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b0c6da59-c35d-4d20-8346-3dd5afc3a8f3-original-pull-secret") pod "global-pull-secret-syncer-84fpq" (UID: "b0c6da59-c35d-4d20-8346-3dd5afc3a8f3") : object "kube-system"/"original-pull-secret" not registered
Jul 29 14:01:19.214679 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:19.214652    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 14:01:19.214679 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:19.214667    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 14:01:19.214679 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:19.214679    2559 projected.go:196] Error preparing data for projected volume kube-api-access-tnwlt for pod openshift-network-diagnostics/network-check-target-gbj56: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 14:01:19.214821 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:19.214714    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d24b8b8-c850-495d-b0f7-ca7ea53a8a57-kube-api-access-tnwlt podName:6d24b8b8-c850-495d-b0f7-ca7ea53a8a57 nodeName:}" failed. No retries permitted until 2026-07-29 14:01:27.214702569 +0000 UTC m=+17.105373886 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tnwlt" (UniqueName: "kubernetes.io/projected/6d24b8b8-c850-495d-b0f7-ca7ea53a8a57-kube-api-access-tnwlt") pod "network-check-target-gbj56" (UID: "6d24b8b8-c850-495d-b0f7-ca7ea53a8a57") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 14:01:19.630718 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:19.630683    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:01:19.630886 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:19.630815    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84fpq" podUID="b0c6da59-c35d-4d20-8346-3dd5afc3a8f3"
Jul 29 14:01:19.631266 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:19.631244    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwxl9"
Jul 29 14:01:19.631433 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:19.631408    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwxl9" podUID="93be82f5-6d56-4bd3-9822-bef13d5738e1"
Jul 29 14:01:20.631994 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:20.631959    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbj56"
Jul 29 14:01:20.632469 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:20.632121    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbj56" podUID="6d24b8b8-c850-495d-b0f7-ca7ea53a8a57"
Jul 29 14:01:21.630679 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:21.630645    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:01:21.630679 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:21.630670    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwxl9"
Jul 29 14:01:21.630873 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:21.630765    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84fpq" podUID="b0c6da59-c35d-4d20-8346-3dd5afc3a8f3"
Jul 29 14:01:21.630873 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:21.630848    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwxl9" podUID="93be82f5-6d56-4bd3-9822-bef13d5738e1"
Jul 29 14:01:22.630658 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:22.630629    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbj56"
Jul 29 14:01:22.631043 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:22.630746    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbj56" podUID="6d24b8b8-c850-495d-b0f7-ca7ea53a8a57"
Jul 29 14:01:23.630732 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:23.630654    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwxl9"
Jul 29 14:01:23.631190 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:23.630779    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwxl9" podUID="93be82f5-6d56-4bd3-9822-bef13d5738e1"
Jul 29 14:01:23.631190 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:23.630836    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:01:23.631190 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:23.630942    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84fpq" podUID="b0c6da59-c35d-4d20-8346-3dd5afc3a8f3"
Jul 29 14:01:24.630815 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:24.630783    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbj56"
Jul 29 14:01:24.631311 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:24.630898    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbj56" podUID="6d24b8b8-c850-495d-b0f7-ca7ea53a8a57"
Jul 29 14:01:25.631350 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:25.631310    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwxl9"
Jul 29 14:01:25.631350 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:25.631347    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:01:25.631817 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:25.631448    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwxl9" podUID="93be82f5-6d56-4bd3-9822-bef13d5738e1"
Jul 29 14:01:25.631817 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:25.631578    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84fpq" podUID="b0c6da59-c35d-4d20-8346-3dd5afc3a8f3"
Jul 29 14:01:26.631345 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:26.631293    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbj56"
Jul 29 14:01:26.631510 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:26.631428    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbj56" podUID="6d24b8b8-c850-495d-b0f7-ca7ea53a8a57"
Jul 29 14:01:27.276783 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:27.276748    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnwlt\" (UniqueName: \"kubernetes.io/projected/6d24b8b8-c850-495d-b0f7-ca7ea53a8a57-kube-api-access-tnwlt\") pod \"network-check-target-gbj56\" (UID: \"6d24b8b8-c850-495d-b0f7-ca7ea53a8a57\") " pod="openshift-network-diagnostics/network-check-target-gbj56"
Jul 29 14:01:27.276986 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:27.276794    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93be82f5-6d56-4bd3-9822-bef13d5738e1-metrics-certs\") pod \"network-metrics-daemon-cwxl9\" (UID: \"93be82f5-6d56-4bd3-9822-bef13d5738e1\") " pod="openshift-multus/network-metrics-daemon-cwxl9"
Jul 29 14:01:27.276986 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:27.276854    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b0c6da59-c35d-4d20-8346-3dd5afc3a8f3-original-pull-secret\") pod \"global-pull-secret-syncer-84fpq\" (UID: \"b0c6da59-c35d-4d20-8346-3dd5afc3a8f3\") " pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:01:27.276986 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:27.276932    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 14:01:27.276986 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:27.276955    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 14:01:27.276986 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:27.276956    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 14:01:27.276986 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:27.276961    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 14:01:27.276986 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:27.276965    2559 projected.go:196] Error preparing data for projected volume kube-api-access-tnwlt for pod openshift-network-diagnostics/network-check-target-gbj56: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 14:01:27.277273 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:27.277013    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0c6da59-c35d-4d20-8346-3dd5afc3a8f3-original-pull-secret podName:b0c6da59-c35d-4d20-8346-3dd5afc3a8f3 nodeName:}" failed. No retries permitted until 2026-07-29 14:01:43.276995674 +0000 UTC m=+33.167667007 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b0c6da59-c35d-4d20-8346-3dd5afc3a8f3-original-pull-secret") pod "global-pull-secret-syncer-84fpq" (UID: "b0c6da59-c35d-4d20-8346-3dd5afc3a8f3") : object "kube-system"/"original-pull-secret" not registered
Jul 29 14:01:27.277273 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:27.277030    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d24b8b8-c850-495d-b0f7-ca7ea53a8a57-kube-api-access-tnwlt podName:6d24b8b8-c850-495d-b0f7-ca7ea53a8a57 nodeName:}" failed. No retries permitted until 2026-07-29 14:01:43.27702157 +0000 UTC m=+33.167692888 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tnwlt" (UniqueName: "kubernetes.io/projected/6d24b8b8-c850-495d-b0f7-ca7ea53a8a57-kube-api-access-tnwlt") pod "network-check-target-gbj56" (UID: "6d24b8b8-c850-495d-b0f7-ca7ea53a8a57") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 14:01:27.277273 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:27.277045    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93be82f5-6d56-4bd3-9822-bef13d5738e1-metrics-certs podName:93be82f5-6d56-4bd3-9822-bef13d5738e1 nodeName:}" failed. No retries permitted until 2026-07-29 14:01:43.277038074 +0000 UTC m=+33.167709397 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93be82f5-6d56-4bd3-9822-bef13d5738e1-metrics-certs") pod "network-metrics-daemon-cwxl9" (UID: "93be82f5-6d56-4bd3-9822-bef13d5738e1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 14:01:27.630382 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:27.630345    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwxl9"
Jul 29 14:01:27.630581 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:27.630359    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:01:27.630581 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:27.630500    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwxl9" podUID="93be82f5-6d56-4bd3-9822-bef13d5738e1"
Jul 29 14:01:27.630698 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:27.630578    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84fpq" podUID="b0c6da59-c35d-4d20-8346-3dd5afc3a8f3"
Jul 29 14:01:28.630980 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:28.630941    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbj56"
Jul 29 14:01:28.631450 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:28.631090    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbj56" podUID="6d24b8b8-c850-495d-b0f7-ca7ea53a8a57"
Jul 29 14:01:29.630562 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:29.630527    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:01:29.630723 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:29.630645    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84fpq" podUID="b0c6da59-c35d-4d20-8346-3dd5afc3a8f3"
Jul 29 14:01:29.630723 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:29.630702    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwxl9"
Jul 29 14:01:29.630849 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:29.630829    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwxl9" podUID="93be82f5-6d56-4bd3-9822-bef13d5738e1"
Jul 29 14:01:30.633795 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:30.633768    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbj56"
Jul 29 14:01:30.634208 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:30.633868    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbj56" podUID="6d24b8b8-c850-495d-b0f7-ca7ea53a8a57"
Jul 29 14:01:31.630345 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:31.630317    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwxl9"
Jul 29 14:01:31.630436 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:31.630325    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:01:31.630473 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:31.630426    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwxl9" podUID="93be82f5-6d56-4bd3-9822-bef13d5738e1"
Jul 29 14:01:31.630611 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:31.630587    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84fpq" podUID="b0c6da59-c35d-4d20-8346-3dd5afc3a8f3"
Jul 29 14:01:31.784582 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:31.784551    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-szq55" event={"ID":"d6541ac6-8d3b-4175-9881-3e5e82d72429","Type":"ContainerStarted","Data":"5a0c5ec2c76212822b785ac1ba928d09d3cc532ac1f276bde69dd7ba95331d9b"}
Jul 29 14:01:31.787594 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:31.787557    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-187.ec2.internal" event={"ID":"4dad8c770ea44aa6fdaf6d1c2a53cc88","Type":"ContainerStarted","Data":"ff35252689aa8383e7be276d88f2581bf23e16c614cc7630d6d5cf641bf1ce60"}
Jul 29 14:01:31.796403 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:31.796180    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdw8l_b93f6faa-c8bf-43e9-94e0-b3dcd97b7296/ovn-acl-logging/0.log"
Jul 29 14:01:31.799793 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:31.799703    2559 generic.go:356] "Generic (PLEG): container finished" podID="b93f6faa-c8bf-43e9-94e0-b3dcd97b7296" containerID="6e563e2868f3a67e9856c6b44baa97b83b5cc75c47f60244cd47f57a72cc75f2" exitCode=1
Jul 29 14:01:31.799793 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:31.799745    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l" event={"ID":"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296","Type":"ContainerStarted","Data":"95e6bc44c3e34a8f238a220f4f050616511aee31a7ca0bec87faa357ff8b8843"}
Jul 29 14:01:31.799936 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:31.799813    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l" event={"ID":"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296","Type":"ContainerDied","Data":"6e563e2868f3a67e9856c6b44baa97b83b5cc75c47f60244cd47f57a72cc75f2"}
Jul 29 14:01:31.799936 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:31.799861    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l" event={"ID":"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296","Type":"ContainerStarted","Data":"8c7054ce25c552cd8dd16489e3e26b700146655f3f7057a8401138db1f222dfb"}
Jul 29 14:01:31.822071 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:31.821382    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-187.ec2.internal" podStartSLOduration=20.821362836 podStartE2EDuration="20.821362836s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 14:01:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:01:31.821300694 +0000 UTC m=+21.711972030" watchObservedRunningTime="2026-07-29 14:01:31.821362836 +0000 UTC m=+21.712034174"
Jul 29 14:01:31.822263 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:31.822224    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-szq55" podStartSLOduration=2.482593974 podStartE2EDuration="21.822212112s" totalImagesPullingTime="19.339618138s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 14:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:01:31.806968316 +0000 UTC m=+21.697639651" watchObservedRunningTime="2026-07-29 14:01:31.822212112 +0000 UTC m=+21.712883447"
Jul 29 14:01:32.631093 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:32.631049    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbj56"
Jul 29 14:01:32.631322 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:32.631197    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbj56" podUID="6d24b8b8-c850-495d-b0f7-ca7ea53a8a57"
Jul 29 14:01:32.803290 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:32.803250    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hbw9p" event={"ID":"44f77fb3-96c6-42ad-b76f-f605ab05c9d6","Type":"ContainerStarted","Data":"d64ba3d7b73fe1c7e4288b1457d8af15ae1a594dd4bf2207c75e252da7a482c3"}
Jul 29 14:01:32.804752 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:32.804728    2559 generic.go:356] "Generic (PLEG): container finished" podID="24dbef56-b5b2-49e3-b2d6-e3ea78eeb731" containerID="b6b551ae498e5c185add3e64add9ed44471d15f69de54d5f77a8f415dbb578c9" exitCode=0
Jul 29 14:01:32.804882 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:32.804801    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bcfwk" event={"ID":"24dbef56-b5b2-49e3-b2d6-e3ea78eeb731","Type":"ContainerDied","Data":"b6b551ae498e5c185add3e64add9ed44471d15f69de54d5f77a8f415dbb578c9"}
Jul 29 14:01:32.806247 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:32.806224    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nl5mz" event={"ID":"2317c90b-86b0-4ce3-a455-a973befb1b11","Type":"ContainerStarted","Data":"2860abd8084dd044a62d07001f9aa11f98b0a24635bbd3f1eaec951d84e314c9"}
Jul 29 14:01:32.808981 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:32.808963    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdw8l_b93f6faa-c8bf-43e9-94e0-b3dcd97b7296/ovn-acl-logging/0.log"
Jul 29 14:01:32.809358 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:32.809339    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l" event={"ID":"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296","Type":"ContainerStarted","Data":"e6422ceaaa050ee16a58aabd20609695103406f34a760f7c515560ed091f3133"}
Jul 29 14:01:32.809423 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:32.809395    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l" event={"ID":"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296","Type":"ContainerStarted","Data":"6b8abd0bd62b397fe188af43559ca697a727074af3b4aa3ca4e69f31e49368f2"}
Jul 29 14:01:32.809474 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:32.809443    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l" event={"ID":"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296","Type":"ContainerStarted","Data":"bdcd785c4cb7b5e9178996711668e9b4d4c1785a1c4b8ee49c89e3e557cf9efb"}
Jul 29 14:01:32.810680 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:32.810658    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlk7p" event={"ID":"253b40f9-e444-40f9-a59c-1cd0f39c4dbe","Type":"ContainerStarted","Data":"cca2322e9a6f35ca55ff6461448184ec7b27e5d1264e50a37b9fa79e09f761b4"}
Jul 29 14:01:32.812017 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:32.811995    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7xpvn" event={"ID":"62eed460-c3d4-4d6e-a7ff-cf1290a96c53","Type":"ContainerStarted","Data":"550791650e97a8543e9296d334ba5e7af4b3043d6e2b49304fcc58cca563c12c"}
Jul 29 14:01:32.813554 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:32.813533    2559 generic.go:356] "Generic (PLEG): container finished" podID="770d58cac971d94e189e1619f4f30236" containerID="e063bf5d465d658c44acd66374faecb0a2b607ca47f7a6657a21b74c31bc1d2a" exitCode=0
Jul 29 14:01:32.813650 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:32.813615    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-187.ec2.internal" event={"ID":"770d58cac971d94e189e1619f4f30236","Type":"ContainerDied","Data":"e063bf5d465d658c44acd66374faecb0a2b607ca47f7a6657a21b74c31bc1d2a"}
Jul 29 14:01:32.814882 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:32.814846    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mz4xm" event={"ID":"045bee14-dd3a-4494-9a05-ae72fcf00f6d","Type":"ContainerStarted","Data":"06c732e8ab51ecee093773b023180087af335096b5d7e5a60299cc82f4f05f23"}
Jul 29 14:01:32.816244 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:32.816214    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-d9ncc" event={"ID":"22c32d1e-2781-499f-a092-2898013fb37e","Type":"ContainerStarted","Data":"d41eb06ac0dc6dabb57a9329efacee085922413caf4b4ccefba387c4652f2be5"}
Jul 29 14:01:32.822489 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:32.822452    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-hbw9p" podStartSLOduration=2.988930102 podStartE2EDuration="22.822441031s" totalImagesPullingTime="19.833510929s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 14:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:01:32.822210211 +0000 UTC m=+22.712881546" watchObservedRunningTime="2026-07-29 14:01:32.822441031 +0000 UTC m=+22.713112365"
Jul 29 14:01:32.836641 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:32.836602    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-nl5mz" podStartSLOduration=3.549281606 podStartE2EDuration="22.836588533s" totalImagesPullingTime="19.287306927s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 14:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:01:32.836236233 +0000 UTC m=+22.726907567" watchObservedRunningTime="2026-07-29 14:01:32.836588533 +0000 UTC m=+22.727259867"
Jul 29 14:01:32.873513 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:32.873468    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-d9ncc" podStartSLOduration=3.515185739 podStartE2EDuration="22.87345442s" totalImagesPullingTime="19.358268681s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 14:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:01:32.873141188 +0000 UTC m=+22.763812523" watchObservedRunningTime="2026-07-29 14:01:32.87345442 +0000 UTC m=+22.764125755"
Jul 29 14:01:32.889089 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:32.889030    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mz4xm" podStartSLOduration=3.554384778 podStartE2EDuration="22.889014975s" totalImagesPullingTime="19.334630197s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 14:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:01:32.88882503 +0000 UTC m=+22.779496367" watchObservedRunningTime="2026-07-29 14:01:32.889014975 +0000 UTC m=+22.779686309"
Jul 29 14:01:32.925435 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:32.925396    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-7xpvn" podStartSLOduration=3.590321016 podStartE2EDuration="22.925382336s" totalImagesPullingTime="19.33506132s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:01:32.924752051 +0000 UTC m=+22.815423388" watchObservedRunningTime="2026-07-29 14:01:32.925382336 +0000 UTC m=+22.816053668"
Jul 29 14:01:33.148981 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:33.148956    2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 29 14:01:33.587392 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:33.587283    2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-29T14:01:33.148975722Z","UUID":"a19c8d84-e3eb-4218-85ab-64231264242b","Handler":null,"Name":"","Endpoint":""}
Jul 29 14:01:33.588900 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:33.588877    2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 29 14:01:33.589020 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:33.588913    2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 29 14:01:33.631116 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:33.631085    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:01:33.631284 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:33.631215    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84fpq" podUID="b0c6da59-c35d-4d20-8346-3dd5afc3a8f3"
Jul 29 14:01:33.631284 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:33.631280    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwxl9"
Jul 29 14:01:33.631405 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:33.631378    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwxl9" podUID="93be82f5-6d56-4bd3-9822-bef13d5738e1"
Jul 29 14:01:33.820809 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:33.820588    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlk7p" event={"ID":"253b40f9-e444-40f9-a59c-1cd0f39c4dbe","Type":"ContainerStarted","Data":"3ed8c1e3f7e6aedd4e86fb33d7d46d33237e7048fe45eed239c836b940c5ced8"}
Jul 29 14:01:33.822899 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:33.822860    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-187.ec2.internal" event={"ID":"770d58cac971d94e189e1619f4f30236","Type":"ContainerStarted","Data":"d1ac3340e0e6748f3b8d63b6fee14934cb7985c1b2831d49fbd705e531c92037"}
Jul 29 14:01:33.842460 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:33.842367    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-187.ec2.internal" podStartSLOduration=22.842346662 podStartE2EDuration="22.842346662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 14:01:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:01:33.842298831 +0000 UTC m=+23.732970165" watchObservedRunningTime="2026-07-29 14:01:33.842346662 +0000 UTC m=+23.733017998"
Jul 29 14:01:34.055476 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:34.055439    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-7xpvn"
Jul 29 14:01:34.056085 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:34.056043    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-7xpvn"
Jul 29 14:01:34.630518 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:34.630487    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbj56"
Jul 29 14:01:34.630679 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:34.630609    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbj56" podUID="6d24b8b8-c850-495d-b0f7-ca7ea53a8a57"
Jul 29 14:01:34.828208 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:34.828172    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdw8l_b93f6faa-c8bf-43e9-94e0-b3dcd97b7296/ovn-acl-logging/0.log"
Jul 29 14:01:34.828736 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:34.828569    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l" event={"ID":"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296","Type":"ContainerStarted","Data":"569fdf8f391de4dce9c05d2e15a62cb5a470d5bdb2571818409225de676bd3e0"}
Jul 29 14:01:34.830760 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:34.830734    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlk7p" event={"ID":"253b40f9-e444-40f9-a59c-1cd0f39c4dbe","Type":"ContainerStarted","Data":"4a8f26ea5737f846dcbcb9f937f75a536e34926cf190acfce2e3a4828ccc7efb"}
Jul 29 14:01:34.830881 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:34.830821    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-7xpvn"
Jul 29 14:01:34.831615 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:34.831594    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-7xpvn"
Jul 29 14:01:34.865577 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:34.865531    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlk7p" podStartSLOduration=3.129983259 podStartE2EDuration="24.865517405s" totalImagesPullingTime="21.735534146s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 14:01:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:01:34.850203721 +0000 UTC m=+24.740875055" watchObservedRunningTime="2026-07-29 14:01:34.865517405 +0000 UTC m=+24.756188741"
Jul 29 14:01:35.630417 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:35.630381    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:01:35.630417 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:35.630413    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwxl9"
Jul 29 14:01:35.630664 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:35.630519    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84fpq" podUID="b0c6da59-c35d-4d20-8346-3dd5afc3a8f3"
Jul 29 14:01:35.630725 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:35.630679    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwxl9" podUID="93be82f5-6d56-4bd3-9822-bef13d5738e1"
Jul 29 14:01:36.631366 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:36.631292    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbj56"
Jul 29 14:01:36.631801 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:36.631405    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbj56" podUID="6d24b8b8-c850-495d-b0f7-ca7ea53a8a57"
Jul 29 14:01:37.630680 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:37.630507    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwxl9"
Jul 29 14:01:37.630843 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:37.630553    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:01:37.630843 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:37.630773    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwxl9" podUID="93be82f5-6d56-4bd3-9822-bef13d5738e1"
Jul 29 14:01:37.630843 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:37.630811    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84fpq" podUID="b0c6da59-c35d-4d20-8346-3dd5afc3a8f3"
Jul 29 14:01:37.838418 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:37.838382    2559 generic.go:356] "Generic (PLEG): container finished" podID="24dbef56-b5b2-49e3-b2d6-e3ea78eeb731" containerID="2585662d9148444deab7d2034f93df4796e8fc958c3cd65ea4d232f1560c4647" exitCode=0
Jul 29 14:01:37.838970 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:37.838458    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bcfwk" event={"ID":"24dbef56-b5b2-49e3-b2d6-e3ea78eeb731","Type":"ContainerDied","Data":"2585662d9148444deab7d2034f93df4796e8fc958c3cd65ea4d232f1560c4647"}
Jul 29 14:01:37.842037 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:37.842018    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdw8l_b93f6faa-c8bf-43e9-94e0-b3dcd97b7296/ovn-acl-logging/0.log"
Jul 29 14:01:37.842422 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:37.842396    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l" event={"ID":"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296","Type":"ContainerStarted","Data":"36322a20166ca52539ead161a942dfd9aeaae6b69bbcd96779b86d238f14c849"}
Jul 29 14:01:37.842678 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:37.842661    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:37.842843 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:37.842821    2559 scope.go:117] "RemoveContainer" containerID="6e563e2868f3a67e9856c6b44baa97b83b5cc75c47f60244cd47f57a72cc75f2"
Jul 29 14:01:37.859219 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:37.859197    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:38.631097 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:38.631050    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbj56"
Jul 29 14:01:38.631302 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:38.631209    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbj56" podUID="6d24b8b8-c850-495d-b0f7-ca7ea53a8a57"
Jul 29 14:01:38.847642 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:38.847616    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdw8l_b93f6faa-c8bf-43e9-94e0-b3dcd97b7296/ovn-acl-logging/0.log"
Jul 29 14:01:38.848088 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:38.847924    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l" event={"ID":"b93f6faa-c8bf-43e9-94e0-b3dcd97b7296","Type":"ContainerStarted","Data":"be108a4b6e57e79b9dd6edad2a3eed4eb90d3a599c57d555a203430046071b4b"}
Jul 29 14:01:38.848225 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:38.848204    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:38.848370 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:38.848342    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:38.865158 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:38.865132    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:01:38.876986 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:38.876946    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l" podStartSLOduration=9.490050947 podStartE2EDuration="28.876932025s" totalImagesPullingTime="19.386881078s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 14:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:01:38.876574634 +0000 UTC m=+28.767245969" watchObservedRunningTime="2026-07-29 14:01:38.876932025 +0000 UTC m=+28.767603351"
Jul 29 14:01:38.906401 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:38.906371    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-84fpq"]
Jul 29 14:01:38.906501 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:38.906483    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:01:38.906582 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:38.906564    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84fpq" podUID="b0c6da59-c35d-4d20-8346-3dd5afc3a8f3"
Jul 29 14:01:38.907052 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:38.907030    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gbj56"]
Jul 29 14:01:38.907171 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:38.907135    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbj56"
Jul 29 14:01:38.907231 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:38.907207    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbj56" podUID="6d24b8b8-c850-495d-b0f7-ca7ea53a8a57"
Jul 29 14:01:38.907879 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:38.907856    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cwxl9"]
Jul 29 14:01:38.907973 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:38.907958    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwxl9"
Jul 29 14:01:38.908097 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:38.908072    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwxl9" podUID="93be82f5-6d56-4bd3-9822-bef13d5738e1"
Jul 29 14:01:39.852699 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:39.852663    2559 generic.go:356] "Generic (PLEG): container finished" podID="24dbef56-b5b2-49e3-b2d6-e3ea78eeb731" containerID="900ecb40f58faec1fc3b0178ec97d7198d08b6155f2668fba16b3cdf3ce49bdc" exitCode=0
Jul 29 14:01:39.853089 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:39.852752    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bcfwk" event={"ID":"24dbef56-b5b2-49e3-b2d6-e3ea78eeb731","Type":"ContainerDied","Data":"900ecb40f58faec1fc3b0178ec97d7198d08b6155f2668fba16b3cdf3ce49bdc"}
Jul 29 14:01:40.631345 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:40.631310    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:01:40.631567 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:40.631390    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbj56"
Jul 29 14:01:40.631567 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:40.631406    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84fpq" podUID="b0c6da59-c35d-4d20-8346-3dd5afc3a8f3"
Jul 29 14:01:40.631567 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:40.631501    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbj56" podUID="6d24b8b8-c850-495d-b0f7-ca7ea53a8a57"
Jul 29 14:01:40.631567 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:40.631551    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwxl9"
Jul 29 14:01:40.631734 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:40.631638    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwxl9" podUID="93be82f5-6d56-4bd3-9822-bef13d5738e1"
Jul 29 14:01:41.860261 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:41.860224    2559 generic.go:356] "Generic (PLEG): container finished" podID="24dbef56-b5b2-49e3-b2d6-e3ea78eeb731" containerID="f6b2de4a5799239fa9a63cab1f8e5b0998a041923d62e5b53e217cd14b77be7d" exitCode=0
Jul 29 14:01:41.860665 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:41.860286    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bcfwk" event={"ID":"24dbef56-b5b2-49e3-b2d6-e3ea78eeb731","Type":"ContainerDied","Data":"f6b2de4a5799239fa9a63cab1f8e5b0998a041923d62e5b53e217cd14b77be7d"}
Jul 29 14:01:42.630320 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:42.630283    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:01:42.630511 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:42.630283    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwxl9"
Jul 29 14:01:42.630511 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:42.630413    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84fpq" podUID="b0c6da59-c35d-4d20-8346-3dd5afc3a8f3"
Jul 29 14:01:42.630620 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:42.630506    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwxl9" podUID="93be82f5-6d56-4bd3-9822-bef13d5738e1"
Jul 29 14:01:42.630620 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:42.630283    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbj56"
Jul 29 14:01:42.630620 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:42.630607    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbj56" podUID="6d24b8b8-c850-495d-b0f7-ca7ea53a8a57"
Jul 29 14:01:43.293284 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:43.293247    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b0c6da59-c35d-4d20-8346-3dd5afc3a8f3-original-pull-secret\") pod \"global-pull-secret-syncer-84fpq\" (UID: \"b0c6da59-c35d-4d20-8346-3dd5afc3a8f3\") " pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:01:43.293284 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:43.293292    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnwlt\" (UniqueName: \"kubernetes.io/projected/6d24b8b8-c850-495d-b0f7-ca7ea53a8a57-kube-api-access-tnwlt\") pod \"network-check-target-gbj56\" (UID: \"6d24b8b8-c850-495d-b0f7-ca7ea53a8a57\") " pod="openshift-network-diagnostics/network-check-target-gbj56"
Jul 29 14:01:43.293955 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:43.293324    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93be82f5-6d56-4bd3-9822-bef13d5738e1-metrics-certs\") pod \"network-metrics-daemon-cwxl9\" (UID: \"93be82f5-6d56-4bd3-9822-bef13d5738e1\") " pod="openshift-multus/network-metrics-daemon-cwxl9"
Jul 29 14:01:43.293955 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:43.293423    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 14:01:43.293955 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:43.293494    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0c6da59-c35d-4d20-8346-3dd5afc3a8f3-original-pull-secret podName:b0c6da59-c35d-4d20-8346-3dd5afc3a8f3 nodeName:}" failed. No retries permitted until 2026-07-29 14:02:15.293479363 +0000 UTC m=+65.184150680 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b0c6da59-c35d-4d20-8346-3dd5afc3a8f3-original-pull-secret") pod "global-pull-secret-syncer-84fpq" (UID: "b0c6da59-c35d-4d20-8346-3dd5afc3a8f3") : object "kube-system"/"original-pull-secret" not registered
Jul 29 14:01:43.293955 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:43.293423    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 14:01:43.293955 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:43.293565    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93be82f5-6d56-4bd3-9822-bef13d5738e1-metrics-certs podName:93be82f5-6d56-4bd3-9822-bef13d5738e1 nodeName:}" failed. No retries permitted until 2026-07-29 14:02:15.293551286 +0000 UTC m=+65.184222602 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93be82f5-6d56-4bd3-9822-bef13d5738e1-metrics-certs") pod "network-metrics-daemon-cwxl9" (UID: "93be82f5-6d56-4bd3-9822-bef13d5738e1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 14:01:43.293955 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:43.293436    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 14:01:43.293955 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:43.293589    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 14:01:43.293955 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:43.293603    2559 projected.go:196] Error preparing data for projected volume kube-api-access-tnwlt for pod openshift-network-diagnostics/network-check-target-gbj56: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 14:01:43.293955 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:43.293652    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d24b8b8-c850-495d-b0f7-ca7ea53a8a57-kube-api-access-tnwlt podName:6d24b8b8-c850-495d-b0f7-ca7ea53a8a57 nodeName:}" failed. No retries permitted until 2026-07-29 14:02:15.293644207 +0000 UTC m=+65.184315536 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tnwlt" (UniqueName: "kubernetes.io/projected/6d24b8b8-c850-495d-b0f7-ca7ea53a8a57-kube-api-access-tnwlt") pod "network-check-target-gbj56" (UID: "6d24b8b8-c850-495d-b0f7-ca7ea53a8a57") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 14:01:43.659032 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:43.658937    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:01:43.760049 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:43.760011    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:01:43.860039 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:43.859990    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:01:44.427300 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.427050    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-187.ec2.internal" event="NodeReady"
Jul 29 14:01:44.427777 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.427427    2559 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 29 14:01:44.486974 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.486940    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7d59476db5-kd6tg"]
Jul 29 14:01:44.490089 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.490051    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-msl82"]
Jul 29 14:01:44.490239 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.490187    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:01:44.491900 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.491880    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-msl82"
Jul 29 14:01:44.492050 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.492029    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-b76j4"]
Jul 29 14:01:44.493650 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.493629    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 29 14:01:44.493786 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.493708    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fnsd9\""
Jul 29 14:01:44.493786 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.493716    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b76j4"
Jul 29 14:01:44.494086 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.494051    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 29 14:01:44.495048 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.495029    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 29 14:01:44.495208 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.495121    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 29 14:01:44.495208 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.495139    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 29 14:01:44.495313 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.495260    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-v4vw8\""
Jul 29 14:01:44.500108 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.500087    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vwvk9\""
Jul 29 14:01:44.500219 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.500146    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 29 14:01:44.500219 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.500167    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 29 14:01:44.500312 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.500092    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 29 14:01:44.500707 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.500684    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 29 14:01:44.516842 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.516812    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-msl82"]
Jul 29 14:01:44.517501 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.517482    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b76j4"]
Jul 29 14:01:44.518171 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.518151    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d59476db5-kd6tg"]
Jul 29 14:01:44.560276 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.560243    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-zmjrn"]
Jul 29 14:01:44.562275 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.562243    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-zmjrn"
Jul 29 14:01:44.563682 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.563658    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-5nxst"]
Jul 29 14:01:44.565250 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.565109    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 29 14:01:44.565250 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.565121    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-xs6vx\""
Jul 29 14:01:44.565401 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.565254    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-5nxst"
Jul 29 14:01:44.565401 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.565312    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 29 14:01:44.567855 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.567834    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 29 14:01:44.567963 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.567876    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 29 14:01:44.568029 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.567982    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-czhw4\""
Jul 29 14:01:44.568265 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.568241    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 29 14:01:44.568351 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.568290    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 29 14:01:44.574981 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.574954    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-zmjrn"]
Jul 29 14:01:44.578525 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.578502    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-5nxst"]
Jul 29 14:01:44.603801 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.603776    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-bound-sa-token\") pod \"image-registry-7d59476db5-kd6tg\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") " pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:01:44.603923 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.603808    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfq9p\" (UniqueName: \"kubernetes.io/projected/0b17154a-2770-4e62-8cde-5114d45a8dc2-kube-api-access-tfq9p\") pod \"dns-default-msl82\" (UID: \"0b17154a-2770-4e62-8cde-5114d45a8dc2\") " pod="openshift-dns/dns-default-msl82"
Jul 29 14:01:44.603923 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.603836    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/88cdc98a-350e-42ff-a401-7f8ad06de6bb-trusted-ca\") pod \"image-registry-7d59476db5-kd6tg\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") " pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:01:44.603923 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.603893    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/697b63cb-b6c9-4178-892b-822d19e03056-cert\") pod \"ingress-canary-b76j4\" (UID: \"697b63cb-b6c9-4178-892b-822d19e03056\") " pod="openshift-ingress-canary/ingress-canary-b76j4"
Jul 29 14:01:44.604102 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.604019    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-registry-tls\") pod \"image-registry-7d59476db5-kd6tg\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") " pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:01:44.604102 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.604073    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qp5jd\" (UniqueName: \"kubernetes.io/projected/697b63cb-b6c9-4178-892b-822d19e03056-kube-api-access-qp5jd\") pod \"ingress-canary-b76j4\" (UID: \"697b63cb-b6c9-4178-892b-822d19e03056\") " pod="openshift-ingress-canary/ingress-canary-b76j4"
Jul 29 14:01:44.604205 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.604102    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/88cdc98a-350e-42ff-a401-7f8ad06de6bb-image-registry-private-configuration\") pod \"image-registry-7d59476db5-kd6tg\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") " pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:01:44.604205 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.604128    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/88cdc98a-350e-42ff-a401-7f8ad06de6bb-ca-trust-extracted\") pod \"image-registry-7d59476db5-kd6tg\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") " pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:01:44.604205 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.604155    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/88cdc98a-350e-42ff-a401-7f8ad06de6bb-installation-pull-secrets\") pod \"image-registry-7d59476db5-kd6tg\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") " pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:01:44.604205 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.604179    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg58r\" (UniqueName: \"kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-kube-api-access-rg58r\") pod \"image-registry-7d59476db5-kd6tg\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") " pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:01:44.604393 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.604206    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0b17154a-2770-4e62-8cde-5114d45a8dc2-config-volume\") pod \"dns-default-msl82\" (UID: \"0b17154a-2770-4e62-8cde-5114d45a8dc2\") " pod="openshift-dns/dns-default-msl82"
Jul 29 14:01:44.604393 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.604228    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0b17154a-2770-4e62-8cde-5114d45a8dc2-metrics-tls\") pod \"dns-default-msl82\" (UID: \"0b17154a-2770-4e62-8cde-5114d45a8dc2\") " pod="openshift-dns/dns-default-msl82"
Jul 29 14:01:44.604393 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.604285    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b17154a-2770-4e62-8cde-5114d45a8dc2-tmp-dir\") pod \"dns-default-msl82\" (UID: \"0b17154a-2770-4e62-8cde-5114d45a8dc2\") " pod="openshift-dns/dns-default-msl82"
Jul 29 14:01:44.604393 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.604329    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/88cdc98a-350e-42ff-a401-7f8ad06de6bb-registry-certificates\") pod \"image-registry-7d59476db5-kd6tg\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") " pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:01:44.630390 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.630363    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:01:44.630519 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.630365    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbj56"
Jul 29 14:01:44.630581 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.630370    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwxl9"
Jul 29 14:01:44.633469 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.633446    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 14:01:44.633581 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.633491    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 14:01:44.633780 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.633671    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 14:01:44.633876 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.633854    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6wqn8\""
Jul 29 14:01:44.634229 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.634201    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 14:01:44.634307 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.634209    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7g4zm\""
Jul 29 14:01:44.694920 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.694837    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:01:44.704781 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.704748    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b17154a-2770-4e62-8cde-5114d45a8dc2-tmp-dir\") pod \"dns-default-msl82\" (UID: \"0b17154a-2770-4e62-8cde-5114d45a8dc2\") " pod="openshift-dns/dns-default-msl82"
Jul 29 14:01:44.704928 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.704790    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/49f8acf6-08de-4400-adfd-fa32d9a066b5-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-5nxst\" (UID: \"49f8acf6-08de-4400-adfd-fa32d9a066b5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-5nxst"
Jul 29 14:01:44.704928 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.704818    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/88cdc98a-350e-42ff-a401-7f8ad06de6bb-registry-certificates\") pod \"image-registry-7d59476db5-kd6tg\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") " pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:01:44.704928 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.704843    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/88728d56-a8af-49d4-a52c-674e9ac76cd9-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-zmjrn\" (UID: \"88728d56-a8af-49d4-a52c-674e9ac76cd9\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-zmjrn"
Jul 29 14:01:44.704928 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.704923    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-bound-sa-token\") pod \"image-registry-7d59476db5-kd6tg\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") " pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:01:44.705159 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.704951    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfq9p\" (UniqueName: \"kubernetes.io/projected/0b17154a-2770-4e62-8cde-5114d45a8dc2-kube-api-access-tfq9p\") pod \"dns-default-msl82\" (UID: \"0b17154a-2770-4e62-8cde-5114d45a8dc2\") " pod="openshift-dns/dns-default-msl82"
Jul 29 14:01:44.705159 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.704995    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/88cdc98a-350e-42ff-a401-7f8ad06de6bb-trusted-ca\") pod \"image-registry-7d59476db5-kd6tg\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") " pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:01:44.705159 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.705024    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/49f8acf6-08de-4400-adfd-fa32d9a066b5-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-5nxst\" (UID: \"49f8acf6-08de-4400-adfd-fa32d9a066b5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-5nxst"
Jul 29 14:01:44.705159 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.705133    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b17154a-2770-4e62-8cde-5114d45a8dc2-tmp-dir\") pod \"dns-default-msl82\" (UID: \"0b17154a-2770-4e62-8cde-5114d45a8dc2\") " pod="openshift-dns/dns-default-msl82"
Jul 29 14:01:44.705325 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.705169    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/697b63cb-b6c9-4178-892b-822d19e03056-cert\") pod \"ingress-canary-b76j4\" (UID: \"697b63cb-b6c9-4178-892b-822d19e03056\") " pod="openshift-ingress-canary/ingress-canary-b76j4"
Jul 29 14:01:44.705325 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.705221    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-registry-tls\") pod \"image-registry-7d59476db5-kd6tg\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") " pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:01:44.705325 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.705251    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qp5jd\" (UniqueName: \"kubernetes.io/projected/697b63cb-b6c9-4178-892b-822d19e03056-kube-api-access-qp5jd\") pod \"ingress-canary-b76j4\" (UID: \"697b63cb-b6c9-4178-892b-822d19e03056\") " pod="openshift-ingress-canary/ingress-canary-b76j4"
Jul 29 14:01:44.705325 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.705289    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/88cdc98a-350e-42ff-a401-7f8ad06de6bb-image-registry-private-configuration\") pod \"image-registry-7d59476db5-kd6tg\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") " pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:01:44.705522 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:44.705376    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 14:01:44.705522 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:44.705399    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d59476db5-kd6tg: secret "image-registry-tls" not found
Jul 29 14:01:44.705522 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.705441    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/88cdc98a-350e-42ff-a401-7f8ad06de6bb-ca-trust-extracted\") pod \"image-registry-7d59476db5-kd6tg\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") " pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:01:44.705522 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:44.705474    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-registry-tls podName:88cdc98a-350e-42ff-a401-7f8ad06de6bb nodeName:}" failed. No retries permitted until 2026-07-29 14:01:45.205454033 +0000 UTC m=+35.096125364 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-registry-tls") pod "image-registry-7d59476db5-kd6tg" (UID: "88cdc98a-350e-42ff-a401-7f8ad06de6bb") : secret "image-registry-tls" not found
Jul 29 14:01:44.705522 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.705511    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/88cdc98a-350e-42ff-a401-7f8ad06de6bb-installation-pull-secrets\") pod \"image-registry-7d59476db5-kd6tg\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") " pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:01:44.705769 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.705520    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/88cdc98a-350e-42ff-a401-7f8ad06de6bb-registry-certificates\") pod \"image-registry-7d59476db5-kd6tg\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") " pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:01:44.705769 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.705541    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg58r\" (UniqueName: \"kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-kube-api-access-rg58r\") pod \"image-registry-7d59476db5-kd6tg\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") " pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:01:44.705769 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.705573    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw465\" (UniqueName: \"kubernetes.io/projected/49f8acf6-08de-4400-adfd-fa32d9a066b5-kube-api-access-nw465\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-5nxst\" (UID: \"49f8acf6-08de-4400-adfd-fa32d9a066b5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-5nxst"
Jul 29 14:01:44.705769 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.705605    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88728d56-a8af-49d4-a52c-674e9ac76cd9-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-zmjrn\" (UID: \"88728d56-a8af-49d4-a52c-674e9ac76cd9\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-zmjrn"
Jul 29 14:01:44.705769 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.705639    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0b17154a-2770-4e62-8cde-5114d45a8dc2-config-volume\") pod \"dns-default-msl82\" (UID: \"0b17154a-2770-4e62-8cde-5114d45a8dc2\") " pod="openshift-dns/dns-default-msl82"
Jul 29 14:01:44.705769 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.705663    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0b17154a-2770-4e62-8cde-5114d45a8dc2-metrics-tls\") pod \"dns-default-msl82\" (UID: \"0b17154a-2770-4e62-8cde-5114d45a8dc2\") " pod="openshift-dns/dns-default-msl82"
Jul 29 14:01:44.705769 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.705722    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/88cdc98a-350e-42ff-a401-7f8ad06de6bb-ca-trust-extracted\") pod \"image-registry-7d59476db5-kd6tg\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") " pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:01:44.705769 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:44.705767    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 14:01:44.706153 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:44.705886    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b17154a-2770-4e62-8cde-5114d45a8dc2-metrics-tls podName:0b17154a-2770-4e62-8cde-5114d45a8dc2 nodeName:}" failed. No retries permitted until 2026-07-29 14:01:45.205871252 +0000 UTC m=+35.096542580 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0b17154a-2770-4e62-8cde-5114d45a8dc2-metrics-tls") pod "dns-default-msl82" (UID: "0b17154a-2770-4e62-8cde-5114d45a8dc2") : secret "dns-default-metrics-tls" not found
Jul 29 14:01:44.706153 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.705962    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/88cdc98a-350e-42ff-a401-7f8ad06de6bb-trusted-ca\") pod \"image-registry-7d59476db5-kd6tg\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") " pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:01:44.706263 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.706243    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0b17154a-2770-4e62-8cde-5114d45a8dc2-config-volume\") pod \"dns-default-msl82\" (UID: \"0b17154a-2770-4e62-8cde-5114d45a8dc2\") " pod="openshift-dns/dns-default-msl82"
Jul 29 14:01:44.709780 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.709745    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/88cdc98a-350e-42ff-a401-7f8ad06de6bb-installation-pull-secrets\") pod \"image-registry-7d59476db5-kd6tg\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") " pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:01:44.709780 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.709760    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/88cdc98a-350e-42ff-a401-7f8ad06de6bb-image-registry-private-configuration\") pod \"image-registry-7d59476db5-kd6tg\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") " pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:01:44.709984 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.709766    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/697b63cb-b6c9-4178-892b-822d19e03056-cert\") pod \"ingress-canary-b76j4\" (UID: \"697b63cb-b6c9-4178-892b-822d19e03056\") " pod="openshift-ingress-canary/ingress-canary-b76j4"
Jul 29 14:01:44.717433 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.717386    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfq9p\" (UniqueName: \"kubernetes.io/projected/0b17154a-2770-4e62-8cde-5114d45a8dc2-kube-api-access-tfq9p\") pod \"dns-default-msl82\" (UID: \"0b17154a-2770-4e62-8cde-5114d45a8dc2\") " pod="openshift-dns/dns-default-msl82"
Jul 29 14:01:44.717751 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.717728    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-bound-sa-token\") pod \"image-registry-7d59476db5-kd6tg\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") " pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:01:44.718567 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.718538    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg58r\" (UniqueName: \"kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-kube-api-access-rg58r\") pod \"image-registry-7d59476db5-kd6tg\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") " pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:01:44.719675 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.719654    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qp5jd\" (UniqueName: \"kubernetes.io/projected/697b63cb-b6c9-4178-892b-822d19e03056-kube-api-access-qp5jd\") pod \"ingress-canary-b76j4\" (UID: \"697b63cb-b6c9-4178-892b-822d19e03056\") " pod="openshift-ingress-canary/ingress-canary-b76j4"
Jul 29 14:01:44.794546 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.794508    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:01:44.806511 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.806484    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw465\" (UniqueName: \"kubernetes.io/projected/49f8acf6-08de-4400-adfd-fa32d9a066b5-kube-api-access-nw465\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-5nxst\" (UID: \"49f8acf6-08de-4400-adfd-fa32d9a066b5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-5nxst"
Jul 29 14:01:44.806657 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.806515    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88728d56-a8af-49d4-a52c-674e9ac76cd9-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-zmjrn\" (UID: \"88728d56-a8af-49d4-a52c-674e9ac76cd9\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-zmjrn"
Jul 29 14:01:44.806657 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.806568    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/49f8acf6-08de-4400-adfd-fa32d9a066b5-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-5nxst\" (UID: \"49f8acf6-08de-4400-adfd-fa32d9a066b5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-5nxst"
Jul 29 14:01:44.806657 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.806591    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/88728d56-a8af-49d4-a52c-674e9ac76cd9-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-zmjrn\" (UID: \"88728d56-a8af-49d4-a52c-674e9ac76cd9\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-zmjrn"
Jul 29 14:01:44.806818 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.806658    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/49f8acf6-08de-4400-adfd-fa32d9a066b5-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-5nxst\" (UID: \"49f8acf6-08de-4400-adfd-fa32d9a066b5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-5nxst"
Jul 29 14:01:44.806818 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:44.806730    2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 14:01:44.806818 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:44.806813    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88728d56-a8af-49d4-a52c-674e9ac76cd9-networking-console-plugin-cert podName:88728d56-a8af-49d4-a52c-674e9ac76cd9 nodeName:}" failed. No retries permitted until 2026-07-29 14:01:45.306789877 +0000 UTC m=+35.197461204 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/88728d56-a8af-49d4-a52c-674e9ac76cd9-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-zmjrn" (UID: "88728d56-a8af-49d4-a52c-674e9ac76cd9") : secret "networking-console-plugin-cert" not found
Jul 29 14:01:44.807370 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.807341    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/88728d56-a8af-49d4-a52c-674e9ac76cd9-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-zmjrn\" (UID: \"88728d56-a8af-49d4-a52c-674e9ac76cd9\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-zmjrn"
Jul 29 14:01:44.807473 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.807418    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/49f8acf6-08de-4400-adfd-fa32d9a066b5-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-5nxst\" (UID: \"49f8acf6-08de-4400-adfd-fa32d9a066b5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-5nxst"
Jul 29 14:01:44.809355 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.809333    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/49f8acf6-08de-4400-adfd-fa32d9a066b5-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-5nxst\" (UID: \"49f8acf6-08de-4400-adfd-fa32d9a066b5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-5nxst"
Jul 29 14:01:44.814651 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.814629    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b76j4"
Jul 29 14:01:44.816541 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.816515    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw465\" (UniqueName: \"kubernetes.io/projected/49f8acf6-08de-4400-adfd-fa32d9a066b5-kube-api-access-nw465\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-5nxst\" (UID: \"49f8acf6-08de-4400-adfd-fa32d9a066b5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-5nxst"
Jul 29 14:01:44.882323 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.882287    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-5nxst"
Jul 29 14:01:44.899948 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.899900    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:01:44.988621 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:44.988505    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:01:45.210068 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:45.210019    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-registry-tls\") pod \"image-registry-7d59476db5-kd6tg\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") " pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:01:45.210235 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:45.210106    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0b17154a-2770-4e62-8cde-5114d45a8dc2-metrics-tls\") pod \"dns-default-msl82\" (UID: \"0b17154a-2770-4e62-8cde-5114d45a8dc2\") " pod="openshift-dns/dns-default-msl82"
Jul 29 14:01:45.210235 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:45.210185    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 14:01:45.210235 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:45.210204    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d59476db5-kd6tg: secret "image-registry-tls" not found
Jul 29 14:01:45.210347 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:45.210253    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 14:01:45.210347 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:45.210259    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-registry-tls podName:88cdc98a-350e-42ff-a401-7f8ad06de6bb nodeName:}" failed. No retries permitted until 2026-07-29 14:01:46.210241913 +0000 UTC m=+36.100913227 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-registry-tls") pod "image-registry-7d59476db5-kd6tg" (UID: "88cdc98a-350e-42ff-a401-7f8ad06de6bb") : secret "image-registry-tls" not found
Jul 29 14:01:45.210347 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:45.210312    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b17154a-2770-4e62-8cde-5114d45a8dc2-metrics-tls podName:0b17154a-2770-4e62-8cde-5114d45a8dc2 nodeName:}" failed. No retries permitted until 2026-07-29 14:01:46.21029594 +0000 UTC m=+36.100967270 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0b17154a-2770-4e62-8cde-5114d45a8dc2-metrics-tls") pod "dns-default-msl82" (UID: "0b17154a-2770-4e62-8cde-5114d45a8dc2") : secret "dns-default-metrics-tls" not found
Jul 29 14:01:45.310929 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:45.310893    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88728d56-a8af-49d4-a52c-674e9ac76cd9-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-zmjrn\" (UID: \"88728d56-a8af-49d4-a52c-674e9ac76cd9\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-zmjrn"
Jul 29 14:01:45.311126 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:45.311052    2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 14:01:45.311183 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:45.311140    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88728d56-a8af-49d4-a52c-674e9ac76cd9-networking-console-plugin-cert podName:88728d56-a8af-49d4-a52c-674e9ac76cd9 nodeName:}" failed. No retries permitted until 2026-07-29 14:01:46.311121945 +0000 UTC m=+36.201793259 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/88728d56-a8af-49d4-a52c-674e9ac76cd9-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-zmjrn" (UID: "88728d56-a8af-49d4-a52c-674e9ac76cd9") : secret "networking-console-plugin-cert" not found
Jul 29 14:01:46.030374 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:46.030349    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-5nxst"]
Jul 29 14:01:46.033789 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:46.033760    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49f8acf6_08de_4400_adfd_fa32d9a066b5.slice/crio-76aaf54a6fdf9d02b655c4a3980166e8fd9ac28aabfd1fd231584f315bac476c WatchSource:0}: Error finding container 76aaf54a6fdf9d02b655c4a3980166e8fd9ac28aabfd1fd231584f315bac476c: Status 404 returned error can't find the container with id 76aaf54a6fdf9d02b655c4a3980166e8fd9ac28aabfd1fd231584f315bac476c
Jul 29 14:01:46.036612 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:46.036562    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b76j4"]
Jul 29 14:01:46.039945 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:46.039922    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod697b63cb_b6c9_4178_892b_822d19e03056.slice/crio-9332f4587adaa834d539a385fd496be9934bd6864904041b9ceeae700d53fffc WatchSource:0}: Error finding container 9332f4587adaa834d539a385fd496be9934bd6864904041b9ceeae700d53fffc: Status 404 returned error can't find the container with id 9332f4587adaa834d539a385fd496be9934bd6864904041b9ceeae700d53fffc
Jul 29 14:01:46.218904 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:46.218874    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-registry-tls\") pod \"image-registry-7d59476db5-kd6tg\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") " pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:01:46.219037 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:46.218923    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0b17154a-2770-4e62-8cde-5114d45a8dc2-metrics-tls\") pod \"dns-default-msl82\" (UID: \"0b17154a-2770-4e62-8cde-5114d45a8dc2\") " pod="openshift-dns/dns-default-msl82"
Jul 29 14:01:46.219037 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:46.219024    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 14:01:46.219132 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:46.219045    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d59476db5-kd6tg: secret "image-registry-tls" not found
Jul 29 14:01:46.219132 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:46.219053    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 14:01:46.219132 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:46.219120    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-registry-tls podName:88cdc98a-350e-42ff-a401-7f8ad06de6bb nodeName:}" failed. No retries permitted until 2026-07-29 14:01:48.219100984 +0000 UTC m=+38.109772296 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-registry-tls") pod "image-registry-7d59476db5-kd6tg" (UID: "88cdc98a-350e-42ff-a401-7f8ad06de6bb") : secret "image-registry-tls" not found
Jul 29 14:01:46.219228 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:46.219134    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b17154a-2770-4e62-8cde-5114d45a8dc2-metrics-tls podName:0b17154a-2770-4e62-8cde-5114d45a8dc2 nodeName:}" failed. No retries permitted until 2026-07-29 14:01:48.219128755 +0000 UTC m=+38.109800067 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0b17154a-2770-4e62-8cde-5114d45a8dc2-metrics-tls") pod "dns-default-msl82" (UID: "0b17154a-2770-4e62-8cde-5114d45a8dc2") : secret "dns-default-metrics-tls" not found
Jul 29 14:01:46.319413 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:46.319381    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88728d56-a8af-49d4-a52c-674e9ac76cd9-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-zmjrn\" (UID: \"88728d56-a8af-49d4-a52c-674e9ac76cd9\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-zmjrn"
Jul 29 14:01:46.319558 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:46.319522    2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 14:01:46.319595 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:46.319583    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88728d56-a8af-49d4-a52c-674e9ac76cd9-networking-console-plugin-cert podName:88728d56-a8af-49d4-a52c-674e9ac76cd9 nodeName:}" failed. No retries permitted until 2026-07-29 14:01:48.319568248 +0000 UTC m=+38.210239562 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/88728d56-a8af-49d4-a52c-674e9ac76cd9-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-zmjrn" (UID: "88728d56-a8af-49d4-a52c-674e9ac76cd9") : secret "networking-console-plugin-cert" not found
Jul 29 14:01:46.874112 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:46.874035    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-5nxst" event={"ID":"49f8acf6-08de-4400-adfd-fa32d9a066b5","Type":"ContainerStarted","Data":"76aaf54a6fdf9d02b655c4a3980166e8fd9ac28aabfd1fd231584f315bac476c"}
Jul 29 14:01:46.875510 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:46.875442    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b76j4" event={"ID":"697b63cb-b6c9-4178-892b-822d19e03056","Type":"ContainerStarted","Data":"9332f4587adaa834d539a385fd496be9934bd6864904041b9ceeae700d53fffc"}
Jul 29 14:01:46.878874 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:46.878848    2559 generic.go:356] "Generic (PLEG): container finished" podID="24dbef56-b5b2-49e3-b2d6-e3ea78eeb731" containerID="5c4e7021d8c92f63cdfcc3835d3f58ed1998f191e0ff61617005cf6e037de8bd" exitCode=0
Jul 29 14:01:46.878985 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:46.878902    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bcfwk" event={"ID":"24dbef56-b5b2-49e3-b2d6-e3ea78eeb731","Type":"ContainerDied","Data":"5c4e7021d8c92f63cdfcc3835d3f58ed1998f191e0ff61617005cf6e037de8bd"}
Jul 29 14:01:47.752138 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:47.752096    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:01:47.853111 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:47.853078    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:01:47.885172 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:47.885141    2559 generic.go:356] "Generic (PLEG): container finished" podID="24dbef56-b5b2-49e3-b2d6-e3ea78eeb731" containerID="9bfc6ef1daee4efac407e5fbeab7799fbe89a38fdf74c4a7a3573eb6677b9608" exitCode=0
Jul 29 14:01:47.885340 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:47.885183    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bcfwk" event={"ID":"24dbef56-b5b2-49e3-b2d6-e3ea78eeb731","Type":"ContainerDied","Data":"9bfc6ef1daee4efac407e5fbeab7799fbe89a38fdf74c4a7a3573eb6677b9608"}
Jul 29 14:01:47.916233 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:47.916202    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:01:48.236839 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:48.236812    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-registry-tls\") pod \"image-registry-7d59476db5-kd6tg\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") " pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:01:48.236986 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:48.236875    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0b17154a-2770-4e62-8cde-5114d45a8dc2-metrics-tls\") pod \"dns-default-msl82\" (UID: \"0b17154a-2770-4e62-8cde-5114d45a8dc2\") " pod="openshift-dns/dns-default-msl82"
Jul 29 14:01:48.236986 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:48.236965    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 14:01:48.237101 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:48.237029    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b17154a-2770-4e62-8cde-5114d45a8dc2-metrics-tls podName:0b17154a-2770-4e62-8cde-5114d45a8dc2 nodeName:}" failed. No retries permitted until 2026-07-29 14:01:52.237012113 +0000 UTC m=+42.127683426 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0b17154a-2770-4e62-8cde-5114d45a8dc2-metrics-tls") pod "dns-default-msl82" (UID: "0b17154a-2770-4e62-8cde-5114d45a8dc2") : secret "dns-default-metrics-tls" not found
Jul 29 14:01:48.237101 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:48.236969    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 14:01:48.237101 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:48.237084    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d59476db5-kd6tg: secret "image-registry-tls" not found
Jul 29 14:01:48.237228 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:48.237142    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-registry-tls podName:88cdc98a-350e-42ff-a401-7f8ad06de6bb nodeName:}" failed. No retries permitted until 2026-07-29 14:01:52.237124467 +0000 UTC m=+42.127795802 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-registry-tls") pod "image-registry-7d59476db5-kd6tg" (UID: "88cdc98a-350e-42ff-a401-7f8ad06de6bb") : secret "image-registry-tls" not found
Jul 29 14:01:48.337783 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:48.337741    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88728d56-a8af-49d4-a52c-674e9ac76cd9-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-zmjrn\" (UID: \"88728d56-a8af-49d4-a52c-674e9ac76cd9\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-zmjrn"
Jul 29 14:01:48.337940 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:48.337915    2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 14:01:48.338012 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:48.338001    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88728d56-a8af-49d4-a52c-674e9ac76cd9-networking-console-plugin-cert podName:88728d56-a8af-49d4-a52c-674e9ac76cd9 nodeName:}" failed. No retries permitted until 2026-07-29 14:01:52.337979161 +0000 UTC m=+42.228650479 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/88728d56-a8af-49d4-a52c-674e9ac76cd9-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-zmjrn" (UID: "88728d56-a8af-49d4-a52c-674e9ac76cd9") : secret "networking-console-plugin-cert" not found
Jul 29 14:01:49.058405 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:49.058337    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:01:49.159384 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:49.159348    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:01:49.891339 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:49.891307    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-5nxst" event={"ID":"49f8acf6-08de-4400-adfd-fa32d9a066b5","Type":"ContainerStarted","Data":"009d292d17495cf90fd93e02e140933e1f24eee413fef6e7d79ca904badd9d80"}
Jul 29 14:01:49.892692 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:49.892671    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b76j4" event={"ID":"697b63cb-b6c9-4178-892b-822d19e03056","Type":"ContainerStarted","Data":"60f9795b5cee506f1d8cbf515d5be0c9c1170386d63b2b05cd04226fd977cfdc"}
Jul 29 14:01:49.896087 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:49.896047    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bcfwk" event={"ID":"24dbef56-b5b2-49e3-b2d6-e3ea78eeb731","Type":"ContainerStarted","Data":"ea03a3225c9105d7de11c66b9d63657db1a868b0d0de3a9f570fa6e8258f66e2"}
Jul 29 14:01:49.909507 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:49.909462    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-5nxst" podStartSLOduration=2.95550808 podStartE2EDuration="5.909447172s" totalImagesPullingTime="2.953939092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:01:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:01:49.907810848 +0000 UTC m=+39.798482184" watchObservedRunningTime="2026-07-29 14:01:49.909447172 +0000 UTC m=+39.800118522"
Jul 29 14:01:49.932790 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:49.932705    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bcfwk" podStartSLOduration=10.410513325 podStartE2EDuration="39.932686651s" totalImagesPullingTime="29.522173326s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 14:01:10 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:01:49.930683323 +0000 UTC m=+39.821354658" watchObservedRunningTime="2026-07-29 14:01:49.932686651 +0000 UTC m=+39.823357985"
Jul 29 14:01:49.950433 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:49.948377    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-b76j4" podStartSLOduration=3.002313569 podStartE2EDuration="5.948358772s" totalImagesPullingTime="2.946045203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:01:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:01:49.947294892 +0000 UTC m=+39.837966227" watchObservedRunningTime="2026-07-29 14:01:49.948358772 +0000 UTC m=+39.839030107"
Jul 29 14:01:50.168289 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:50.168208    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:01:50.206811 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:50.206785    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-jd9td"]
Jul 29 14:01:50.225850 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:50.225821    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-jd9td"]
Jul 29 14:01:50.225986 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:50.225955    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-jd9td"
Jul 29 14:01:50.228835 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:50.228813    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 29 14:01:50.228960 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:50.228849    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-knlbg\""
Jul 29 14:01:50.228960 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:50.228849    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 29 14:01:50.272253 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:50.272218    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:01:50.354808 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:50.354777    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvkb8\" (UniqueName: \"kubernetes.io/projected/0c24a7a4-774d-4b1b-a581-c583863a2c8d-kube-api-access-mvkb8\") pod \"migrator-6865c9967-jd9td\" (UID: \"0c24a7a4-774d-4b1b-a581-c583863a2c8d\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-jd9td"
Jul 29 14:01:50.368442 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:50.368416    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:01:50.455875 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:50.455801    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvkb8\" (UniqueName: \"kubernetes.io/projected/0c24a7a4-774d-4b1b-a581-c583863a2c8d-kube-api-access-mvkb8\") pod \"migrator-6865c9967-jd9td\" (UID: \"0c24a7a4-774d-4b1b-a581-c583863a2c8d\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-jd9td"
Jul 29 14:01:50.465480 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:50.465451    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvkb8\" (UniqueName: \"kubernetes.io/projected/0c24a7a4-774d-4b1b-a581-c583863a2c8d-kube-api-access-mvkb8\") pod \"migrator-6865c9967-jd9td\" (UID: \"0c24a7a4-774d-4b1b-a581-c583863a2c8d\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-jd9td"
Jul 29 14:01:50.535551 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:50.535526    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-jd9td"
Jul 29 14:01:50.680270 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:50.680237    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-jd9td"]
Jul 29 14:01:50.683973 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:01:50.683938    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c24a7a4_774d_4b1b_a581_c583863a2c8d.slice/crio-046b5ebbafe611937b2468d67ca068c2866ac2d08ec89de3e1156a2ae260eb50 WatchSource:0}: Error finding container 046b5ebbafe611937b2468d67ca068c2866ac2d08ec89de3e1156a2ae260eb50: Status 404 returned error can't find the container with id 046b5ebbafe611937b2468d67ca068c2866ac2d08ec89de3e1156a2ae260eb50
Jul 29 14:01:50.899629 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:50.899593    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-jd9td" event={"ID":"0c24a7a4-774d-4b1b-a581-c583863a2c8d","Type":"ContainerStarted","Data":"046b5ebbafe611937b2468d67ca068c2866ac2d08ec89de3e1156a2ae260eb50"}
Jul 29 14:01:52.271973 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:52.271933    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-registry-tls\") pod \"image-registry-7d59476db5-kd6tg\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") " pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:01:52.272414 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:52.271985    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0b17154a-2770-4e62-8cde-5114d45a8dc2-metrics-tls\") pod \"dns-default-msl82\" (UID: \"0b17154a-2770-4e62-8cde-5114d45a8dc2\") " pod="openshift-dns/dns-default-msl82"
Jul 29 14:01:52.272414 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:52.272127    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 14:01:52.272414 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:52.272133    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 14:01:52.272414 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:52.272160    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d59476db5-kd6tg: secret "image-registry-tls" not found
Jul 29 14:01:52.272414 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:52.272184    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b17154a-2770-4e62-8cde-5114d45a8dc2-metrics-tls podName:0b17154a-2770-4e62-8cde-5114d45a8dc2 nodeName:}" failed. No retries permitted until 2026-07-29 14:02:00.272168566 +0000 UTC m=+50.162839879 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0b17154a-2770-4e62-8cde-5114d45a8dc2-metrics-tls") pod "dns-default-msl82" (UID: "0b17154a-2770-4e62-8cde-5114d45a8dc2") : secret "dns-default-metrics-tls" not found
Jul 29 14:01:52.272414 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:52.272228    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-registry-tls podName:88cdc98a-350e-42ff-a401-7f8ad06de6bb nodeName:}" failed. No retries permitted until 2026-07-29 14:02:00.272209436 +0000 UTC m=+50.162880762 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-registry-tls") pod "image-registry-7d59476db5-kd6tg" (UID: "88cdc98a-350e-42ff-a401-7f8ad06de6bb") : secret "image-registry-tls" not found
Jul 29 14:01:52.373231 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:52.373195    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88728d56-a8af-49d4-a52c-674e9ac76cd9-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-zmjrn\" (UID: \"88728d56-a8af-49d4-a52c-674e9ac76cd9\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-zmjrn"
Jul 29 14:01:52.373371 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:52.373334    2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 14:01:52.373452 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:52.373400    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88728d56-a8af-49d4-a52c-674e9ac76cd9-networking-console-plugin-cert podName:88728d56-a8af-49d4-a52c-674e9ac76cd9 nodeName:}" failed. No retries permitted until 2026-07-29 14:02:00.373382777 +0000 UTC m=+50.264054090 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/88728d56-a8af-49d4-a52c-674e9ac76cd9-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-zmjrn" (UID: "88728d56-a8af-49d4-a52c-674e9ac76cd9") : secret "networking-console-plugin-cert" not found
Jul 29 14:01:52.907356 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:52.907286    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-jd9td" event={"ID":"0c24a7a4-774d-4b1b-a581-c583863a2c8d","Type":"ContainerStarted","Data":"3a611e71946e9cc67dbf14e991c209f53837ece83d2d5f735da49b9fc03995c2"}
Jul 29 14:01:52.907356 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:52.907336    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-jd9td" event={"ID":"0c24a7a4-774d-4b1b-a581-c583863a2c8d","Type":"ContainerStarted","Data":"7994bd7fa3871491b9036df1259a865d365c745c286b593f3cafd26c7a16160d"}
Jul 29 14:01:52.923347 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:52.923304    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-jd9td" podStartSLOduration=0.978354352 podStartE2EDuration="2.92328984s" totalImagesPullingTime="1.944935488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:01:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:01:52.922496501 +0000 UTC m=+42.813167838" watchObservedRunningTime="2026-07-29 14:01:52.92328984 +0000 UTC m=+42.813961175"
Jul 29 14:01:52.993714 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:52.993682    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:01:53.052564 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:53.052523    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:01:54.169245 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:54.169214    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:01:54.544843 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:54.544803    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:01:54.644392 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:54.644360    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:01:54.721464 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:54.721432    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7s84h"]
Jul 29 14:01:54.745995 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:54.745967    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:01:54.755195 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:54.755177    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7s84h"]
Jul 29 14:01:54.755282 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:54.755271    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7s84h"
Jul 29 14:01:54.758728 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:54.758709    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 29 14:01:54.759623 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:54.759603    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 29 14:01:54.760386 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:54.760366    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jrmfz\""
Jul 29 14:01:54.761555 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:54.761234    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 29 14:01:54.763889 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:54.763873    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 29 14:01:54.825385 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:54.825307    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:01:54.894625 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:54.894589    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gz64d\" (UniqueName: \"kubernetes.io/projected/8a8fd4f5-53e9-484a-b530-30e018cb3e51-kube-api-access-gz64d\") pod \"insights-runtime-extractor-7s84h\" (UID: \"8a8fd4f5-53e9-484a-b530-30e018cb3e51\") " pod="openshift-insights/insights-runtime-extractor-7s84h"
Jul 29 14:01:54.894773 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:54.894643    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8a8fd4f5-53e9-484a-b530-30e018cb3e51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7s84h\" (UID: \"8a8fd4f5-53e9-484a-b530-30e018cb3e51\") " pod="openshift-insights/insights-runtime-extractor-7s84h"
Jul 29 14:01:54.894773 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:54.894711    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8a8fd4f5-53e9-484a-b530-30e018cb3e51-crio-socket\") pod \"insights-runtime-extractor-7s84h\" (UID: \"8a8fd4f5-53e9-484a-b530-30e018cb3e51\") " pod="openshift-insights/insights-runtime-extractor-7s84h"
Jul 29 14:01:54.894773 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:54.894737    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a8fd4f5-53e9-484a-b530-30e018cb3e51-data-volume\") pod \"insights-runtime-extractor-7s84h\" (UID: \"8a8fd4f5-53e9-484a-b530-30e018cb3e51\") " pod="openshift-insights/insights-runtime-extractor-7s84h"
Jul 29 14:01:54.894874 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:54.894780    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8a8fd4f5-53e9-484a-b530-30e018cb3e51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7s84h\" (UID: \"8a8fd4f5-53e9-484a-b530-30e018cb3e51\") " pod="openshift-insights/insights-runtime-extractor-7s84h"
Jul 29 14:01:54.903133 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:54.903105    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:01:54.995725 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:54.995687    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8a8fd4f5-53e9-484a-b530-30e018cb3e51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7s84h\" (UID: \"8a8fd4f5-53e9-484a-b530-30e018cb3e51\") " pod="openshift-insights/insights-runtime-extractor-7s84h"
Jul 29 14:01:54.995725 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:54.995728    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8a8fd4f5-53e9-484a-b530-30e018cb3e51-crio-socket\") pod \"insights-runtime-extractor-7s84h\" (UID: \"8a8fd4f5-53e9-484a-b530-30e018cb3e51\") " pod="openshift-insights/insights-runtime-extractor-7s84h"
Jul 29 14:01:54.995945 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:54.995745    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a8fd4f5-53e9-484a-b530-30e018cb3e51-data-volume\") pod \"insights-runtime-extractor-7s84h\" (UID: \"8a8fd4f5-53e9-484a-b530-30e018cb3e51\") " pod="openshift-insights/insights-runtime-extractor-7s84h"
Jul 29 14:01:54.995945 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:54.995784    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8a8fd4f5-53e9-484a-b530-30e018cb3e51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7s84h\" (UID: \"8a8fd4f5-53e9-484a-b530-30e018cb3e51\") " pod="openshift-insights/insights-runtime-extractor-7s84h"
Jul 29 14:01:54.995945 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:54.995823    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 14:01:54.995945 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:54.995840    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gz64d\" (UniqueName: \"kubernetes.io/projected/8a8fd4f5-53e9-484a-b530-30e018cb3e51-kube-api-access-gz64d\") pod \"insights-runtime-extractor-7s84h\" (UID: \"8a8fd4f5-53e9-484a-b530-30e018cb3e51\") " pod="openshift-insights/insights-runtime-extractor-7s84h"
Jul 29 14:01:54.995945 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:54.995912    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8a8fd4f5-53e9-484a-b530-30e018cb3e51-kube-rbac-proxy-cm podName:8a8fd4f5-53e9-484a-b530-30e018cb3e51 nodeName:}" failed. No retries permitted until 2026-07-29 14:01:55.495891215 +0000 UTC m=+45.386562552 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8a8fd4f5-53e9-484a-b530-30e018cb3e51-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7s84h" (UID: "8a8fd4f5-53e9-484a-b530-30e018cb3e51") : configmap "kube-rbac-proxy" not found
Jul 29 14:01:54.995945 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:54.995929    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8a8fd4f5-53e9-484a-b530-30e018cb3e51-crio-socket\") pod \"insights-runtime-extractor-7s84h\" (UID: \"8a8fd4f5-53e9-484a-b530-30e018cb3e51\") " pod="openshift-insights/insights-runtime-extractor-7s84h"
Jul 29 14:01:54.996263 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:54.995953    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 14:01:54.996263 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:54.996018    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a8fd4f5-53e9-484a-b530-30e018cb3e51-insights-runtime-extractor-tls podName:8a8fd4f5-53e9-484a-b530-30e018cb3e51 nodeName:}" failed. No retries permitted until 2026-07-29 14:01:55.49600144 +0000 UTC m=+45.386672771 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8a8fd4f5-53e9-484a-b530-30e018cb3e51-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7s84h" (UID: "8a8fd4f5-53e9-484a-b530-30e018cb3e51") : secret "insights-runtime-extractor-tls" not found
Jul 29 14:01:54.996263 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:54.996146    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a8fd4f5-53e9-484a-b530-30e018cb3e51-data-volume\") pod \"insights-runtime-extractor-7s84h\" (UID: \"8a8fd4f5-53e9-484a-b530-30e018cb3e51\") " pod="openshift-insights/insights-runtime-extractor-7s84h"
Jul 29 14:01:55.001198 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:55.001173    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:01:55.005029 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:55.005010    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gz64d\" (UniqueName: \"kubernetes.io/projected/8a8fd4f5-53e9-484a-b530-30e018cb3e51-kube-api-access-gz64d\") pod \"insights-runtime-extractor-7s84h\" (UID: \"8a8fd4f5-53e9-484a-b530-30e018cb3e51\") " pod="openshift-insights/insights-runtime-extractor-7s84h"
Jul 29 14:01:55.290095 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:55.290045    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:01:55.391375 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:55.391342    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:01:55.500829 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:55.500794    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8a8fd4f5-53e9-484a-b530-30e018cb3e51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7s84h\" (UID: \"8a8fd4f5-53e9-484a-b530-30e018cb3e51\") " pod="openshift-insights/insights-runtime-extractor-7s84h"
Jul 29 14:01:55.501042 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:55.500851    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8a8fd4f5-53e9-484a-b530-30e018cb3e51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7s84h\" (UID: \"8a8fd4f5-53e9-484a-b530-30e018cb3e51\") " pod="openshift-insights/insights-runtime-extractor-7s84h"
Jul 29 14:01:55.501042 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:55.500919    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 14:01:55.501042 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:55.500960    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 14:01:55.501042 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:55.500981    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8a8fd4f5-53e9-484a-b530-30e018cb3e51-kube-rbac-proxy-cm podName:8a8fd4f5-53e9-484a-b530-30e018cb3e51 nodeName:}" failed. No retries permitted until 2026-07-29 14:01:56.500967492 +0000 UTC m=+46.391638804 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8a8fd4f5-53e9-484a-b530-30e018cb3e51-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7s84h" (UID: "8a8fd4f5-53e9-484a-b530-30e018cb3e51") : configmap "kube-rbac-proxy" not found
Jul 29 14:01:55.501042 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:55.501011    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a8fd4f5-53e9-484a-b530-30e018cb3e51-insights-runtime-extractor-tls podName:8a8fd4f5-53e9-484a-b530-30e018cb3e51 nodeName:}" failed. No retries permitted until 2026-07-29 14:01:56.500995595 +0000 UTC m=+46.391666917 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8a8fd4f5-53e9-484a-b530-30e018cb3e51-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7s84h" (UID: "8a8fd4f5-53e9-484a-b530-30e018cb3e51") : secret "insights-runtime-extractor-tls" not found
Jul 29 14:01:56.498780 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:56.498747    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-d9ncc_22c32d1e-2781-499f-a092-2898013fb37e/dns-node-resolver/0.log"
Jul 29 14:01:56.507285 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:56.507252    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8a8fd4f5-53e9-484a-b530-30e018cb3e51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7s84h\" (UID: \"8a8fd4f5-53e9-484a-b530-30e018cb3e51\") " pod="openshift-insights/insights-runtime-extractor-7s84h"
Jul 29 14:01:56.507408 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:56.507320    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8a8fd4f5-53e9-484a-b530-30e018cb3e51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7s84h\" (UID: \"8a8fd4f5-53e9-484a-b530-30e018cb3e51\") " pod="openshift-insights/insights-runtime-extractor-7s84h"
Jul 29 14:01:56.507408 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:56.507360    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 14:01:56.507503 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:56.507417    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8a8fd4f5-53e9-484a-b530-30e018cb3e51-kube-rbac-proxy-cm podName:8a8fd4f5-53e9-484a-b530-30e018cb3e51 nodeName:}" failed. No retries permitted until 2026-07-29 14:01:58.507402214 +0000 UTC m=+48.398073526 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8a8fd4f5-53e9-484a-b530-30e018cb3e51-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7s84h" (UID: "8a8fd4f5-53e9-484a-b530-30e018cb3e51") : configmap "kube-rbac-proxy" not found
Jul 29 14:01:56.507503 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:56.507463    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 14:01:56.507592 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:56.507522    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a8fd4f5-53e9-484a-b530-30e018cb3e51-insights-runtime-extractor-tls podName:8a8fd4f5-53e9-484a-b530-30e018cb3e51 nodeName:}" failed. No retries permitted until 2026-07-29 14:01:58.507506511 +0000 UTC m=+48.398177826 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8a8fd4f5-53e9-484a-b530-30e018cb3e51-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7s84h" (UID: "8a8fd4f5-53e9-484a-b530-30e018cb3e51") : secret "insights-runtime-extractor-tls" not found
Jul 29 14:01:56.895417 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:56.895318    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:01:56.956742 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:56.956712    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:01:57.496993 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:57.496966    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nl5mz_2317c90b-86b0-4ce3-a455-a973befb1b11/node-ca/0.log"
Jul 29 14:01:57.505380 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:57.505352    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:01:58.497945 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:58.497917    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b76j4_697b63cb-b6c9-4178-892b-822d19e03056/serve-healthcheck-canary/0.log"
Jul 29 14:01:58.521260 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:58.521226    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8a8fd4f5-53e9-484a-b530-30e018cb3e51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7s84h\" (UID: \"8a8fd4f5-53e9-484a-b530-30e018cb3e51\") " pod="openshift-insights/insights-runtime-extractor-7s84h"
Jul 29 14:01:58.521661 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:58.521299    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8a8fd4f5-53e9-484a-b530-30e018cb3e51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7s84h\" (UID: \"8a8fd4f5-53e9-484a-b530-30e018cb3e51\") " pod="openshift-insights/insights-runtime-extractor-7s84h"
Jul 29 14:01:58.521661 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:58.521353    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 14:01:58.521661 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:58.521413    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8a8fd4f5-53e9-484a-b530-30e018cb3e51-kube-rbac-proxy-cm podName:8a8fd4f5-53e9-484a-b530-30e018cb3e51 nodeName:}" failed. No retries permitted until 2026-07-29 14:02:02.521399607 +0000 UTC m=+52.412070920 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8a8fd4f5-53e9-484a-b530-30e018cb3e51-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7s84h" (UID: "8a8fd4f5-53e9-484a-b530-30e018cb3e51") : configmap "kube-rbac-proxy" not found
Jul 29 14:01:58.521661 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:58.521443    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 14:01:58.521661 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:01:58.521491    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a8fd4f5-53e9-484a-b530-30e018cb3e51-insights-runtime-extractor-tls podName:8a8fd4f5-53e9-484a-b530-30e018cb3e51 nodeName:}" failed. No retries permitted until 2026-07-29 14:02:02.521478461 +0000 UTC m=+52.412149788 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8a8fd4f5-53e9-484a-b530-30e018cb3e51-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7s84h" (UID: "8a8fd4f5-53e9-484a-b530-30e018cb3e51") : secret "insights-runtime-extractor-tls" not found
Jul 29 14:01:58.777853 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:01:58.777762    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:00.206425 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:00.206374    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:00.333554 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:00.333516    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-registry-tls\") pod \"image-registry-7d59476db5-kd6tg\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") " pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:02:00.333738 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:00.333562    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0b17154a-2770-4e62-8cde-5114d45a8dc2-metrics-tls\") pod \"dns-default-msl82\" (UID: \"0b17154a-2770-4e62-8cde-5114d45a8dc2\") " pod="openshift-dns/dns-default-msl82"
Jul 29 14:02:00.333738 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:02:00.333648    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 14:02:00.333738 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:02:00.333655    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 14:02:00.333738 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:02:00.333671    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d59476db5-kd6tg: secret "image-registry-tls" not found
Jul 29 14:02:00.333738 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:02:00.333698    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b17154a-2770-4e62-8cde-5114d45a8dc2-metrics-tls podName:0b17154a-2770-4e62-8cde-5114d45a8dc2 nodeName:}" failed. No retries permitted until 2026-07-29 14:02:16.333686154 +0000 UTC m=+66.224357480 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0b17154a-2770-4e62-8cde-5114d45a8dc2-metrics-tls") pod "dns-default-msl82" (UID: "0b17154a-2770-4e62-8cde-5114d45a8dc2") : secret "dns-default-metrics-tls" not found
Jul 29 14:02:00.333738 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:02:00.333711    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-registry-tls podName:88cdc98a-350e-42ff-a401-7f8ad06de6bb nodeName:}" failed. No retries permitted until 2026-07-29 14:02:16.333705004 +0000 UTC m=+66.224376317 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-registry-tls") pod "image-registry-7d59476db5-kd6tg" (UID: "88cdc98a-350e-42ff-a401-7f8ad06de6bb") : secret "image-registry-tls" not found
Jul 29 14:02:00.434702 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:00.434668    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88728d56-a8af-49d4-a52c-674e9ac76cd9-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-zmjrn\" (UID: \"88728d56-a8af-49d4-a52c-674e9ac76cd9\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-zmjrn"
Jul 29 14:02:00.434855 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:02:00.434806    2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 14:02:00.434897 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:02:00.434862    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88728d56-a8af-49d4-a52c-674e9ac76cd9-networking-console-plugin-cert podName:88728d56-a8af-49d4-a52c-674e9ac76cd9 nodeName:}" failed. No retries permitted until 2026-07-29 14:02:16.434846408 +0000 UTC m=+66.325517741 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/88728d56-a8af-49d4-a52c-674e9ac76cd9-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-zmjrn" (UID: "88728d56-a8af-49d4-a52c-674e9ac76cd9") : secret "networking-console-plugin-cert" not found
Jul 29 14:02:01.524437 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:01.524388    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:01.625368 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:01.625327    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:01.789820 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:01.789744    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:01.888335 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:01.888298    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:02.195002 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:02.194918    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:02.551148 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:02.551115    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8a8fd4f5-53e9-484a-b530-30e018cb3e51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7s84h\" (UID: \"8a8fd4f5-53e9-484a-b530-30e018cb3e51\") " pod="openshift-insights/insights-runtime-extractor-7s84h"
Jul 29 14:02:02.551557 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:02.551183    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8a8fd4f5-53e9-484a-b530-30e018cb3e51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7s84h\" (UID: \"8a8fd4f5-53e9-484a-b530-30e018cb3e51\") " pod="openshift-insights/insights-runtime-extractor-7s84h"
Jul 29 14:02:02.551557 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:02:02.551275    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 14:02:02.551557 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:02:02.551348    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a8fd4f5-53e9-484a-b530-30e018cb3e51-insights-runtime-extractor-tls podName:8a8fd4f5-53e9-484a-b530-30e018cb3e51 nodeName:}" failed. No retries permitted until 2026-07-29 14:02:10.551332569 +0000 UTC m=+60.442003881 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8a8fd4f5-53e9-484a-b530-30e018cb3e51-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7s84h" (UID: "8a8fd4f5-53e9-484a-b530-30e018cb3e51") : secret "insights-runtime-extractor-tls" not found
Jul 29 14:02:02.551677 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:02.551659    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8a8fd4f5-53e9-484a-b530-30e018cb3e51-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7s84h\" (UID: \"8a8fd4f5-53e9-484a-b530-30e018cb3e51\") " pod="openshift-insights/insights-runtime-extractor-7s84h"
Jul 29 14:02:03.382214 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:03.382184    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:03.461051 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:03.461022    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:03.790146 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:03.790119    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:03.987122 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:03.987079    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:04.593267 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:04.593236    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:05.187077 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:05.187033    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:05.248682 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:05.248649    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:05.393518 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:05.393489    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:05.783865 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:05.783833    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:06.041771 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:06.041676    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:06.102507 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:06.102477    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:06.398338 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:06.398257    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:06.988894 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:06.988853    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:07.388939 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:07.388901    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:07.712800 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:07.712728    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:07.813665 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:07.813624    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:08.195251 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:08.195222    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:08.582230 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:08.582183    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:08.635213 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:08.635181    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:08.790237 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:08.790207    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:09.195917 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:09.195886    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:09.783297 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:09.783259    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:10.387005 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:10.386972    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:10.593915 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:10.593884    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:10.613456 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:10.613431    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8a8fd4f5-53e9-484a-b530-30e018cb3e51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7s84h\" (UID: \"8a8fd4f5-53e9-484a-b530-30e018cb3e51\") " pod="openshift-insights/insights-runtime-extractor-7s84h"
Jul 29 14:02:10.615638 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:10.615607    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8a8fd4f5-53e9-484a-b530-30e018cb3e51-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7s84h\" (UID: \"8a8fd4f5-53e9-484a-b530-30e018cb3e51\") " pod="openshift-insights/insights-runtime-extractor-7s84h"
Jul 29 14:02:10.666497 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:10.666441    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jrmfz\""
Jul 29 14:02:10.673554 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:10.673529    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7s84h"
Jul 29 14:02:10.791649 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:10.791615    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:10.821416 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:10.821385    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7s84h"]
Jul 29 14:02:10.824195 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:02:10.824170    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a8fd4f5_53e9_484a_b530_30e018cb3e51.slice/crio-6cd0c37a4089bfb7c89d4647fc22120e36282be96b89930e70a296e40d376eba WatchSource:0}: Error finding container 6cd0c37a4089bfb7c89d4647fc22120e36282be96b89930e70a296e40d376eba: Status 404 returned error can't find the container with id 6cd0c37a4089bfb7c89d4647fc22120e36282be96b89930e70a296e40d376eba
Jul 29 14:02:10.869525 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:10.869497    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hdw8l"
Jul 29 14:02:10.948600 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:10.948506    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7s84h" event={"ID":"8a8fd4f5-53e9-484a-b530-30e018cb3e51","Type":"ContainerStarted","Data":"6cd0c37a4089bfb7c89d4647fc22120e36282be96b89930e70a296e40d376eba"}
Jul 29 14:02:11.188768 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:11.188729    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:13.147558 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:13.147515    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:13.248962 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:13.248927    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:15.352994 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:15.352959    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b0c6da59-c35d-4d20-8346-3dd5afc3a8f3-original-pull-secret\") pod \"global-pull-secret-syncer-84fpq\" (UID: \"b0c6da59-c35d-4d20-8346-3dd5afc3a8f3\") " pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:02:15.353465 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:15.353005    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnwlt\" (UniqueName: \"kubernetes.io/projected/6d24b8b8-c850-495d-b0f7-ca7ea53a8a57-kube-api-access-tnwlt\") pod \"network-check-target-gbj56\" (UID: \"6d24b8b8-c850-495d-b0f7-ca7ea53a8a57\") " pod="openshift-network-diagnostics/network-check-target-gbj56"
Jul 29 14:02:15.353465 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:15.353036    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93be82f5-6d56-4bd3-9822-bef13d5738e1-metrics-certs\") pod \"network-metrics-daemon-cwxl9\" (UID: \"93be82f5-6d56-4bd3-9822-bef13d5738e1\") " pod="openshift-multus/network-metrics-daemon-cwxl9"
Jul 29 14:02:15.357132 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:15.357101    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 14:02:15.357263 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:15.357218    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 14:02:15.358610 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:15.358592    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 14:02:15.366340 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:15.366309    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b0c6da59-c35d-4d20-8346-3dd5afc3a8f3-original-pull-secret\") pod \"global-pull-secret-syncer-84fpq\" (UID: \"b0c6da59-c35d-4d20-8346-3dd5afc3a8f3\") " pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:02:15.366340 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:15.366335    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93be82f5-6d56-4bd3-9822-bef13d5738e1-metrics-certs\") pod \"network-metrics-daemon-cwxl9\" (UID: \"93be82f5-6d56-4bd3-9822-bef13d5738e1\") " pod="openshift-multus/network-metrics-daemon-cwxl9"
Jul 29 14:02:15.366513 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:15.366496    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 14:02:15.377288 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:15.377264    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnwlt\" (UniqueName: \"kubernetes.io/projected/6d24b8b8-c850-495d-b0f7-ca7ea53a8a57-kube-api-access-tnwlt\") pod \"network-check-target-gbj56\" (UID: \"6d24b8b8-c850-495d-b0f7-ca7ea53a8a57\") " pod="openshift-network-diagnostics/network-check-target-gbj56"
Jul 29 14:02:15.541590 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:15.541547    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84fpq"
Jul 29 14:02:15.551095 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:15.551070    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6wqn8\""
Jul 29 14:02:15.555568 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:15.555545    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7g4zm\""
Jul 29 14:02:15.558876 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:15.558857    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbj56"
Jul 29 14:02:15.563378 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:15.563356    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwxl9"
Jul 29 14:02:15.703012 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:15.702940    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-84fpq"]
Jul 29 14:02:15.706215 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:02:15.706175    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0c6da59_c35d_4d20_8346_3dd5afc3a8f3.slice/crio-991b7c51b653237c3e82d8549a8663f6d8413f30a974e6174d64591bf4dcd773 WatchSource:0}: Error finding container 991b7c51b653237c3e82d8549a8663f6d8413f30a974e6174d64591bf4dcd773: Status 404 returned error can't find the container with id 991b7c51b653237c3e82d8549a8663f6d8413f30a974e6174d64591bf4dcd773
Jul 29 14:02:15.718883 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:15.718853    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gbj56"]
Jul 29 14:02:15.721659 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:02:15.721632    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d24b8b8_c850_495d_b0f7_ca7ea53a8a57.slice/crio-c767c4b4b23dc6066bbbdab03648a927fe1ce5e3dbbcfd628998e48081d43e67 WatchSource:0}: Error finding container c767c4b4b23dc6066bbbdab03648a927fe1ce5e3dbbcfd628998e48081d43e67: Status 404 returned error can't find the container with id c767c4b4b23dc6066bbbdab03648a927fe1ce5e3dbbcfd628998e48081d43e67
Jul 29 14:02:15.739504 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:15.739480    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cwxl9"]
Jul 29 14:02:15.741866 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:02:15.741844    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93be82f5_6d56_4bd3_9822_bef13d5738e1.slice/crio-a107e0538ebf03f004141b04eea223e462c110580be0cb60b9d236afcb993446 WatchSource:0}: Error finding container a107e0538ebf03f004141b04eea223e462c110580be0cb60b9d236afcb993446: Status 404 returned error can't find the container with id a107e0538ebf03f004141b04eea223e462c110580be0cb60b9d236afcb993446
Jul 29 14:02:15.965174 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:15.965101    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7s84h" event={"ID":"8a8fd4f5-53e9-484a-b530-30e018cb3e51","Type":"ContainerStarted","Data":"69463f7681c1314b66b3949136441d05c1a095342dfbb054cdc451b60009d005"}
Jul 29 14:02:15.966166 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:15.966141    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-84fpq" event={"ID":"b0c6da59-c35d-4d20-8346-3dd5afc3a8f3","Type":"ContainerStarted","Data":"991b7c51b653237c3e82d8549a8663f6d8413f30a974e6174d64591bf4dcd773"}
Jul 29 14:02:15.967171 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:15.967152    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cwxl9" event={"ID":"93be82f5-6d56-4bd3-9822-bef13d5738e1","Type":"ContainerStarted","Data":"a107e0538ebf03f004141b04eea223e462c110580be0cb60b9d236afcb993446"}
Jul 29 14:02:15.968103 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:15.968084    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gbj56" event={"ID":"6d24b8b8-c850-495d-b0f7-ca7ea53a8a57","Type":"ContainerStarted","Data":"c767c4b4b23dc6066bbbdab03648a927fe1ce5e3dbbcfd628998e48081d43e67"}
Jul 29 14:02:16.362201 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:16.362158    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-registry-tls\") pod \"image-registry-7d59476db5-kd6tg\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") " pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:02:16.362811 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:16.362237    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0b17154a-2770-4e62-8cde-5114d45a8dc2-metrics-tls\") pod \"dns-default-msl82\" (UID: \"0b17154a-2770-4e62-8cde-5114d45a8dc2\") " pod="openshift-dns/dns-default-msl82"
Jul 29 14:02:16.365485 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:16.365110    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-registry-tls\") pod \"image-registry-7d59476db5-kd6tg\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") " pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:02:16.365485 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:16.365434    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0b17154a-2770-4e62-8cde-5114d45a8dc2-metrics-tls\") pod \"dns-default-msl82\" (UID: \"0b17154a-2770-4e62-8cde-5114d45a8dc2\") " pod="openshift-dns/dns-default-msl82"
Jul 29 14:02:16.463363 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:16.463320    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88728d56-a8af-49d4-a52c-674e9ac76cd9-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-zmjrn\" (UID: \"88728d56-a8af-49d4-a52c-674e9ac76cd9\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-zmjrn"
Jul 29 14:02:16.466604 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:16.466548    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88728d56-a8af-49d4-a52c-674e9ac76cd9-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-zmjrn\" (UID: \"88728d56-a8af-49d4-a52c-674e9ac76cd9\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-zmjrn"
Jul 29 14:02:16.605987 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:16.605713    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fnsd9\""
Jul 29 14:02:16.613586 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:16.613521    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:02:16.614467 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:16.614444    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-v4vw8\""
Jul 29 14:02:16.622349 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:16.622122    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-msl82"
Jul 29 14:02:16.680644 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:16.680364    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-xs6vx\""
Jul 29 14:02:16.686587 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:16.686385    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-zmjrn"
Jul 29 14:02:16.798447 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:16.798413    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-msl82"]
Jul 29 14:02:16.832293 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:16.832223    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d59476db5-kd6tg"]
Jul 29 14:02:17.038629 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:02:17.038489    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b17154a_2770_4e62_8cde_5114d45a8dc2.slice/crio-b78f0d3e97124e52e3f09fef5ab943cc20370b0878f6925932ba97e1448dd48f WatchSource:0}: Error finding container b78f0d3e97124e52e3f09fef5ab943cc20370b0878f6925932ba97e1448dd48f: Status 404 returned error can't find the container with id b78f0d3e97124e52e3f09fef5ab943cc20370b0878f6925932ba97e1448dd48f
Jul 29 14:02:17.039952 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:02:17.039909    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88cdc98a_350e_42ff_a401_7f8ad06de6bb.slice/crio-fd57d9a9d07a54b96307b69362b3b3aa623055357aea3270780228eea809a6cf WatchSource:0}: Error finding container fd57d9a9d07a54b96307b69362b3b3aa623055357aea3270780228eea809a6cf: Status 404 returned error can't find the container with id fd57d9a9d07a54b96307b69362b3b3aa623055357aea3270780228eea809a6cf
Jul 29 14:02:17.195594 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:17.195508    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-zmjrn"]
Jul 29 14:02:17.199505 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:02:17.199473    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88728d56_a8af_49d4_a52c_674e9ac76cd9.slice/crio-10ab448d94e2d940ad67b43ca24bae506a2f8782aea23a1ccb70ceeddc940c81 WatchSource:0}: Error finding container 10ab448d94e2d940ad67b43ca24bae506a2f8782aea23a1ccb70ceeddc940c81: Status 404 returned error can't find the container with id 10ab448d94e2d940ad67b43ca24bae506a2f8782aea23a1ccb70ceeddc940c81
Jul 29 14:02:17.526043 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:17.525996    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:17.578565 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:17.578540    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:17.979004 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:17.978859    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cwxl9" event={"ID":"93be82f5-6d56-4bd3-9822-bef13d5738e1","Type":"ContainerStarted","Data":"c60f6e044f16b84ca39ac5abb5a875906016b83db1201eb9e3256152ee211b85"}
Jul 29 14:02:17.979004 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:17.978929    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cwxl9" event={"ID":"93be82f5-6d56-4bd3-9822-bef13d5738e1","Type":"ContainerStarted","Data":"6046029d7b66d1ff8dabcc0adc766148481304bb437186560e689f105b38c76b"}
Jul 29 14:02:17.980742 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:17.980710    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-msl82" event={"ID":"0b17154a-2770-4e62-8cde-5114d45a8dc2","Type":"ContainerStarted","Data":"b78f0d3e97124e52e3f09fef5ab943cc20370b0878f6925932ba97e1448dd48f"}
Jul 29 14:02:17.983320 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:17.983127    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d59476db5-kd6tg" event={"ID":"88cdc98a-350e-42ff-a401-7f8ad06de6bb","Type":"ContainerStarted","Data":"3e524fc48b3a4d4f811ad2fa6173b322c4dbe6fdab4134bf60091068d1d706fe"}
Jul 29 14:02:17.983320 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:17.983259    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d59476db5-kd6tg" event={"ID":"88cdc98a-350e-42ff-a401-7f8ad06de6bb","Type":"ContainerStarted","Data":"fd57d9a9d07a54b96307b69362b3b3aa623055357aea3270780228eea809a6cf"}
Jul 29 14:02:17.983320 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:17.983295    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:02:17.984594 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:17.984518    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-zmjrn" event={"ID":"88728d56-a8af-49d4-a52c-674e9ac76cd9","Type":"ContainerStarted","Data":"10ab448d94e2d940ad67b43ca24bae506a2f8782aea23a1ccb70ceeddc940c81"}
Jul 29 14:02:18.007385 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:18.007302    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-cwxl9" podStartSLOduration=66.410174751 podStartE2EDuration="1m8.007282976s" totalImagesPullingTime="1.597108225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:02:18.006577339 +0000 UTC m=+67.897248674" watchObservedRunningTime="2026-07-29 14:02:18.007282976 +0000 UTC m=+67.897954314"
Jul 29 14:02:18.029223 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:18.029168    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7d59476db5-kd6tg" podStartSLOduration=34.029149962 podStartE2EDuration="34.029149962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:01:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:02:18.028454859 +0000 UTC m=+67.919126224" watchObservedRunningTime="2026-07-29 14:02:18.029149962 +0000 UTC m=+67.919821299"
Jul 29 14:02:20.390764 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:20.390728    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7d59476db5-kd6tg"]
Jul 29 14:02:20.417430 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:20.417021    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c7b4478f4-qzl8z"]
Jul 29 14:02:20.436301 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:20.436274    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c7b4478f4-qzl8z"]
Jul 29 14:02:20.436465 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:20.436392    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c7b4478f4-qzl8z"
Jul 29 14:02:20.439376 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:20.439188    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 29 14:02:20.439376 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:20.439233    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 29 14:02:20.439581 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:20.439473    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 29 14:02:20.439581 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:20.439578    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 29 14:02:20.439693 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:20.439577    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 29 14:02:20.439843 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:20.439769    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 29 14:02:20.440633 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:20.440612    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 29 14:02:20.495345 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:20.495303    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c3573041-c26b-45c3-9349-0d007619d06f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5c7b4478f4-qzl8z\" (UID: \"c3573041-c26b-45c3-9349-0d007619d06f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c7b4478f4-qzl8z"
Jul 29 14:02:20.495496 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:20.495352    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c3573041-c26b-45c3-9349-0d007619d06f-ca\") pod \"cluster-proxy-proxy-agent-5c7b4478f4-qzl8z\" (UID: \"c3573041-c26b-45c3-9349-0d007619d06f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c7b4478f4-qzl8z"
Jul 29 14:02:20.495496 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:20.495383    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c3573041-c26b-45c3-9349-0d007619d06f-hub\") pod \"cluster-proxy-proxy-agent-5c7b4478f4-qzl8z\" (UID: \"c3573041-c26b-45c3-9349-0d007619d06f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c7b4478f4-qzl8z"
Jul 29 14:02:20.495496 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:20.495420    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fz7tp\" (UniqueName: \"kubernetes.io/projected/c3573041-c26b-45c3-9349-0d007619d06f-kube-api-access-fz7tp\") pod \"cluster-proxy-proxy-agent-5c7b4478f4-qzl8z\" (UID: \"c3573041-c26b-45c3-9349-0d007619d06f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c7b4478f4-qzl8z"
Jul 29 14:02:20.495496 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:20.495449    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c3573041-c26b-45c3-9349-0d007619d06f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5c7b4478f4-qzl8z\" (UID: \"c3573041-c26b-45c3-9349-0d007619d06f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c7b4478f4-qzl8z"
Jul 29 14:02:20.495681 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:20.495509    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3573041-c26b-45c3-9349-0d007619d06f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5c7b4478f4-qzl8z\" (UID: \"c3573041-c26b-45c3-9349-0d007619d06f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c7b4478f4-qzl8z"
Jul 29 14:02:20.560115 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:20.560076    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:20.595904 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:20.595875    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c3573041-c26b-45c3-9349-0d007619d06f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5c7b4478f4-qzl8z\" (UID: \"c3573041-c26b-45c3-9349-0d007619d06f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c7b4478f4-qzl8z"
Jul 29 14:02:20.596092 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:20.595916    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c3573041-c26b-45c3-9349-0d007619d06f-ca\") pod \"cluster-proxy-proxy-agent-5c7b4478f4-qzl8z\" (UID: \"c3573041-c26b-45c3-9349-0d007619d06f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c7b4478f4-qzl8z"
Jul 29 14:02:20.596092 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:20.595942    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c3573041-c26b-45c3-9349-0d007619d06f-hub\") pod \"cluster-proxy-proxy-agent-5c7b4478f4-qzl8z\" (UID: \"c3573041-c26b-45c3-9349-0d007619d06f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c7b4478f4-qzl8z"
Jul 29 14:02:20.596092 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:20.595978    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fz7tp\" (UniqueName: \"kubernetes.io/projected/c3573041-c26b-45c3-9349-0d007619d06f-kube-api-access-fz7tp\") pod \"cluster-proxy-proxy-agent-5c7b4478f4-qzl8z\" (UID: \"c3573041-c26b-45c3-9349-0d007619d06f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c7b4478f4-qzl8z"
Jul 29 14:02:20.596092 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:20.596005    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c3573041-c26b-45c3-9349-0d007619d06f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5c7b4478f4-qzl8z\" (UID: \"c3573041-c26b-45c3-9349-0d007619d06f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c7b4478f4-qzl8z"
Jul 29 14:02:20.596092 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:20.596034    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3573041-c26b-45c3-9349-0d007619d06f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5c7b4478f4-qzl8z\" (UID: \"c3573041-c26b-45c3-9349-0d007619d06f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c7b4478f4-qzl8z"
Jul 29 14:02:20.596826 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:20.596800    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3573041-c26b-45c3-9349-0d007619d06f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5c7b4478f4-qzl8z\" (UID: \"c3573041-c26b-45c3-9349-0d007619d06f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c7b4478f4-qzl8z"
Jul 29 14:02:20.598915 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:20.598839    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c3573041-c26b-45c3-9349-0d007619d06f-ca\") pod \"cluster-proxy-proxy-agent-5c7b4478f4-qzl8z\" (UID: \"c3573041-c26b-45c3-9349-0d007619d06f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c7b4478f4-qzl8z"
Jul 29 14:02:20.599088 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:20.599050    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c3573041-c26b-45c3-9349-0d007619d06f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5c7b4478f4-qzl8z\" (UID: \"c3573041-c26b-45c3-9349-0d007619d06f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c7b4478f4-qzl8z"
Jul 29 14:02:20.599562 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:20.599538    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c3573041-c26b-45c3-9349-0d007619d06f-hub\") pod \"cluster-proxy-proxy-agent-5c7b4478f4-qzl8z\" (UID: \"c3573041-c26b-45c3-9349-0d007619d06f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c7b4478f4-qzl8z"
Jul 29 14:02:20.599692 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:20.599649    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c3573041-c26b-45c3-9349-0d007619d06f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5c7b4478f4-qzl8z\" (UID: \"c3573041-c26b-45c3-9349-0d007619d06f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c7b4478f4-qzl8z"
Jul 29 14:02:20.606735 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:20.606706    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fz7tp\" (UniqueName: \"kubernetes.io/projected/c3573041-c26b-45c3-9349-0d007619d06f-kube-api-access-fz7tp\") pod \"cluster-proxy-proxy-agent-5c7b4478f4-qzl8z\" (UID: \"c3573041-c26b-45c3-9349-0d007619d06f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c7b4478f4-qzl8z"
Jul 29 14:02:20.642576 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:20.642506    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:20.740982 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:20.740934    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:20.763725 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:20.763694    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c7b4478f4-qzl8z"
Jul 29 14:02:20.834951 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:20.834919    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:21.649801 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:21.649646    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c7b4478f4-qzl8z"]
Jul 29 14:02:21.882350 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:02:21.882318    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc3573041_c26b_45c3_9349_0d007619d06f.slice/crio-221457b74519009b4945e41a1e002a7471f2a82eecbb7fcc079e2e0674d4683f WatchSource:0}: Error finding container 221457b74519009b4945e41a1e002a7471f2a82eecbb7fcc079e2e0674d4683f: Status 404 returned error can't find the container with id 221457b74519009b4945e41a1e002a7471f2a82eecbb7fcc079e2e0674d4683f
Jul 29 14:02:22.001430 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:22.001391    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c7b4478f4-qzl8z" event={"ID":"c3573041-c26b-45c3-9349-0d007619d06f","Type":"ContainerStarted","Data":"221457b74519009b4945e41a1e002a7471f2a82eecbb7fcc079e2e0674d4683f"}
Jul 29 14:02:23.007725 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:23.007631    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7s84h" event={"ID":"8a8fd4f5-53e9-484a-b530-30e018cb3e51","Type":"ContainerStarted","Data":"dc4790e8114ff288cad66061dcb13224083f20e5079c00153e9c96d539385ef2"}
Jul 29 14:02:23.009363 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:23.009315    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-zmjrn" event={"ID":"88728d56-a8af-49d4-a52c-674e9ac76cd9","Type":"ContainerStarted","Data":"1213229a9cd2e949ee753b400ef41cdfaffb149420ee349d797fd52c41736324"}
Jul 29 14:02:23.011384 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:23.011322    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-84fpq" event={"ID":"b0c6da59-c35d-4d20-8346-3dd5afc3a8f3","Type":"ContainerStarted","Data":"3962418e6ff0a4a139da438b3815d1f26c94b2f6cadc25b79e6c0a1470d1add6"}
Jul 29 14:02:23.013535 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:23.013508    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-msl82" event={"ID":"0b17154a-2770-4e62-8cde-5114d45a8dc2","Type":"ContainerStarted","Data":"d75956089f608b48744f523c8aa2ce2af54ffd9e54e8359053036e71e3beddca"}
Jul 29 14:02:23.013672 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:23.013556    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-msl82" event={"ID":"0b17154a-2770-4e62-8cde-5114d45a8dc2","Type":"ContainerStarted","Data":"0cbf393fd93415c0cc096c62e60eee237e1061615ea7f31e774d03c56155e198"}
Jul 29 14:02:23.013672 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:23.013639    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-msl82"
Jul 29 14:02:23.024989 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:23.024641    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-zmjrn" podStartSLOduration=33.769779473 podStartE2EDuration="39.024624865s" totalImagesPullingTime="5.254845392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:01:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:02:23.023345746 +0000 UTC m=+72.914017082" watchObservedRunningTime="2026-07-29 14:02:23.024624865 +0000 UTC m=+72.915296203"
Jul 29 14:02:23.040546 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:23.040439    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-msl82" podStartSLOduration=33.617935228 podStartE2EDuration="39.040419395s" totalImagesPullingTime="5.422484167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:01:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:02:23.038957859 +0000 UTC m=+72.929629218" watchObservedRunningTime="2026-07-29 14:02:23.040419395 +0000 UTC m=+72.931090731"
Jul 29 14:02:23.056939 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:23.056888    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-84fpq" podStartSLOduration=66.885763803 podStartE2EDuration="1m13.056869694s" totalImagesPullingTime="6.171105891s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 14:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:02:23.056397586 +0000 UTC m=+72.947068923" watchObservedRunningTime="2026-07-29 14:02:23.056869694 +0000 UTC m=+72.947541030"
Jul 29 14:02:24.484104 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:24.484046    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:24.539951 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:24.539895    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:24.733935 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:24.733895    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:25.024671 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:25.024616    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gbj56" event={"ID":"6d24b8b8-c850-495d-b0f7-ca7ea53a8a57","Type":"ContainerStarted","Data":"97190bfbdb5f632e7e392606bc86ef6b36930d2a514294037abf439337b28b75"}
Jul 29 14:02:25.024846 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:25.024706    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-gbj56"
Jul 29 14:02:25.908938 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:25.908886    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:25.974575 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:25.974521    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:28.140199 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:28.140159    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:28.200650 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:28.200613    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:29.568026 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:29.567984    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:29.627397 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:29.627366    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:30.049053 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:30.049018    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c7b4478f4-qzl8z" event={"ID":"c3573041-c26b-45c3-9349-0d007619d06f","Type":"ContainerStarted","Data":"67ed55756d288fe07dd817fa44b2bb5b438f4a8f8dbdc79cfe0d0b39f5e7dcba"}
Jul 29 14:02:30.049256 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:30.049082    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c7b4478f4-qzl8z" event={"ID":"c3573041-c26b-45c3-9349-0d007619d06f","Type":"ContainerStarted","Data":"c5ec223f3f94143316dff91bd0783df39f627c825e2f7379cdec6fd35dd8176a"}
Jul 29 14:02:30.049256 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:30.049101    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c7b4478f4-qzl8z" event={"ID":"c3573041-c26b-45c3-9349-0d007619d06f","Type":"ContainerStarted","Data":"3b9fc05b261ab1b6fc928b25c614d2ee6ea24dde35393329cf5984bbc4ce36bc"}
Jul 29 14:02:30.050778 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:30.050750    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7s84h" event={"ID":"8a8fd4f5-53e9-484a-b530-30e018cb3e51","Type":"ContainerStarted","Data":"db9ab0fd475a15af1ebb1e8bac30238f4d0885fe36f8acdff90a84b67e205d93"}
Jul 29 14:02:30.071204 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:30.071156    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-gbj56" podStartSLOduration=70.977293226 podStartE2EDuration="1m20.071143395s" totalImagesPullingTime="9.093850169s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:01:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:02:25.040691535 +0000 UTC m=+74.931362873" watchObservedRunningTime="2026-07-29 14:02:30.071143395 +0000 UTC m=+79.961814729"
Jul 29 14:02:30.071381 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:30.071362    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c7b4478f4-qzl8z" podStartSLOduration=2.893928045 podStartE2EDuration="10.071357353s" totalImagesPullingTime="7.177429308s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:02:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:02:30.070856147 +0000 UTC m=+79.961527479" watchObservedRunningTime="2026-07-29 14:02:30.071357353 +0000 UTC m=+79.962028687"
Jul 29 14:02:30.092747 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:30.092706    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7s84h" podStartSLOduration=18.039887249 podStartE2EDuration="36.092692756s" totalImagesPullingTime="18.052805507s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 14:01:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:02:30.091781557 +0000 UTC m=+79.982452893" watchObservedRunningTime="2026-07-29 14:02:30.092692756 +0000 UTC m=+79.983364090"
Jul 29 14:02:33.020958 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:33.020844    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-msl82"
Jul 29 14:02:34.440770 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:34.440698    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:34.541786 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:34.541749    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:34.639175 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:34.639146    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:36.599879 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:36.599847    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:37.219939 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:37.219902    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:37.318762 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:37.318726    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:38.566146 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:38.566112    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:38.797262 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:38.797224    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:38.859420 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:38.859348    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hk7bj"]
Jul 29 14:02:38.863763 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:38.863738    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hk7bj"
Jul 29 14:02:38.866703 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:38.866680    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 29 14:02:38.867316 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:38.867299    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 29 14:02:38.867369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:38.867311    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vrdkg\""
Jul 29 14:02:38.868642 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:38.868621    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 29 14:02:38.868780 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:38.868665    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 29 14:02:38.868780 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:38.868708    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 29 14:02:38.868780 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:38.868777    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 29 14:02:38.928004 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:38.927961    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:38.939511 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:38.939482    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/42fb3745-8927-4c55-a75e-8838f8fcd33a-node-exporter-accelerators-collector-config\") pod \"node-exporter-hk7bj\" (UID: \"42fb3745-8927-4c55-a75e-8838f8fcd33a\") " pod="openshift-monitoring/node-exporter-hk7bj"
Jul 29 14:02:38.939665 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:38.939530    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/42fb3745-8927-4c55-a75e-8838f8fcd33a-node-exporter-textfile\") pod \"node-exporter-hk7bj\" (UID: \"42fb3745-8927-4c55-a75e-8838f8fcd33a\") " pod="openshift-monitoring/node-exporter-hk7bj"
Jul 29 14:02:38.939665 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:38.939581    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/42fb3745-8927-4c55-a75e-8838f8fcd33a-sys\") pod \"node-exporter-hk7bj\" (UID: \"42fb3745-8927-4c55-a75e-8838f8fcd33a\") " pod="openshift-monitoring/node-exporter-hk7bj"
Jul 29 14:02:38.939665 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:38.939605    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/42fb3745-8927-4c55-a75e-8838f8fcd33a-root\") pod \"node-exporter-hk7bj\" (UID: \"42fb3745-8927-4c55-a75e-8838f8fcd33a\") " pod="openshift-monitoring/node-exporter-hk7bj"
Jul 29 14:02:38.939665 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:38.939652    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcnmx\" (UniqueName: \"kubernetes.io/projected/42fb3745-8927-4c55-a75e-8838f8fcd33a-kube-api-access-jcnmx\") pod \"node-exporter-hk7bj\" (UID: \"42fb3745-8927-4c55-a75e-8838f8fcd33a\") " pod="openshift-monitoring/node-exporter-hk7bj"
Jul 29 14:02:38.939788 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:38.939677    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42fb3745-8927-4c55-a75e-8838f8fcd33a-metrics-client-ca\") pod \"node-exporter-hk7bj\" (UID: \"42fb3745-8927-4c55-a75e-8838f8fcd33a\") " pod="openshift-monitoring/node-exporter-hk7bj"
Jul 29 14:02:38.939788 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:38.939696    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/42fb3745-8927-4c55-a75e-8838f8fcd33a-node-exporter-wtmp\") pod \"node-exporter-hk7bj\" (UID: \"42fb3745-8927-4c55-a75e-8838f8fcd33a\") " pod="openshift-monitoring/node-exporter-hk7bj"
Jul 29 14:02:38.939788 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:38.939719    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/42fb3745-8927-4c55-a75e-8838f8fcd33a-node-exporter-tls\") pod \"node-exporter-hk7bj\" (UID: \"42fb3745-8927-4c55-a75e-8838f8fcd33a\") " pod="openshift-monitoring/node-exporter-hk7bj"
Jul 29 14:02:38.939788 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:38.939755    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/42fb3745-8927-4c55-a75e-8838f8fcd33a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hk7bj\" (UID: \"42fb3745-8927-4c55-a75e-8838f8fcd33a\") " pod="openshift-monitoring/node-exporter-hk7bj"
Jul 29 14:02:39.018283 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:39.018240    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:39.040105 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:39.040050    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/42fb3745-8927-4c55-a75e-8838f8fcd33a-node-exporter-wtmp\") pod \"node-exporter-hk7bj\" (UID: \"42fb3745-8927-4c55-a75e-8838f8fcd33a\") " pod="openshift-monitoring/node-exporter-hk7bj"
Jul 29 14:02:39.040105 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:39.040110    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/42fb3745-8927-4c55-a75e-8838f8fcd33a-node-exporter-tls\") pod \"node-exporter-hk7bj\" (UID: \"42fb3745-8927-4c55-a75e-8838f8fcd33a\") " pod="openshift-monitoring/node-exporter-hk7bj"
Jul 29 14:02:39.040336 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:39.040135    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/42fb3745-8927-4c55-a75e-8838f8fcd33a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hk7bj\" (UID: \"42fb3745-8927-4c55-a75e-8838f8fcd33a\") " pod="openshift-monitoring/node-exporter-hk7bj"
Jul 29 14:02:39.040336 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:39.040168    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/42fb3745-8927-4c55-a75e-8838f8fcd33a-node-exporter-accelerators-collector-config\") pod \"node-exporter-hk7bj\" (UID: \"42fb3745-8927-4c55-a75e-8838f8fcd33a\") " pod="openshift-monitoring/node-exporter-hk7bj"
Jul 29 14:02:39.040336 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:39.040211    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/42fb3745-8927-4c55-a75e-8838f8fcd33a-node-exporter-textfile\") pod \"node-exporter-hk7bj\" (UID: \"42fb3745-8927-4c55-a75e-8838f8fcd33a\") " pod="openshift-monitoring/node-exporter-hk7bj"
Jul 29 14:02:39.040336 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:39.040231    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/42fb3745-8927-4c55-a75e-8838f8fcd33a-node-exporter-wtmp\") pod \"node-exporter-hk7bj\" (UID: \"42fb3745-8927-4c55-a75e-8838f8fcd33a\") " pod="openshift-monitoring/node-exporter-hk7bj"
Jul 29 14:02:39.040336 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:39.040278    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/42fb3745-8927-4c55-a75e-8838f8fcd33a-sys\") pod \"node-exporter-hk7bj\" (UID: \"42fb3745-8927-4c55-a75e-8838f8fcd33a\") " pod="openshift-monitoring/node-exporter-hk7bj"
Jul 29 14:02:39.040336 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:39.040236    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/42fb3745-8927-4c55-a75e-8838f8fcd33a-sys\") pod \"node-exporter-hk7bj\" (UID: \"42fb3745-8927-4c55-a75e-8838f8fcd33a\") " pod="openshift-monitoring/node-exporter-hk7bj"
Jul 29 14:02:39.040336 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:39.040318    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/42fb3745-8927-4c55-a75e-8838f8fcd33a-root\") pod \"node-exporter-hk7bj\" (UID: \"42fb3745-8927-4c55-a75e-8838f8fcd33a\") " pod="openshift-monitoring/node-exporter-hk7bj"
Jul 29 14:02:39.040707 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:39.040361    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcnmx\" (UniqueName: \"kubernetes.io/projected/42fb3745-8927-4c55-a75e-8838f8fcd33a-kube-api-access-jcnmx\") pod \"node-exporter-hk7bj\" (UID: \"42fb3745-8927-4c55-a75e-8838f8fcd33a\") " pod="openshift-monitoring/node-exporter-hk7bj"
Jul 29 14:02:39.040707 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:39.040402    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42fb3745-8927-4c55-a75e-8838f8fcd33a-metrics-client-ca\") pod \"node-exporter-hk7bj\" (UID: \"42fb3745-8927-4c55-a75e-8838f8fcd33a\") " pod="openshift-monitoring/node-exporter-hk7bj"
Jul 29 14:02:39.040707 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:39.040411    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/42fb3745-8927-4c55-a75e-8838f8fcd33a-root\") pod \"node-exporter-hk7bj\" (UID: \"42fb3745-8927-4c55-a75e-8838f8fcd33a\") " pod="openshift-monitoring/node-exporter-hk7bj"
Jul 29 14:02:39.040707 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:39.040574    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/42fb3745-8927-4c55-a75e-8838f8fcd33a-node-exporter-textfile\") pod \"node-exporter-hk7bj\" (UID: \"42fb3745-8927-4c55-a75e-8838f8fcd33a\") " pod="openshift-monitoring/node-exporter-hk7bj"
Jul 29 14:02:39.040901 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:39.040871    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/42fb3745-8927-4c55-a75e-8838f8fcd33a-node-exporter-accelerators-collector-config\") pod \"node-exporter-hk7bj\" (UID: \"42fb3745-8927-4c55-a75e-8838f8fcd33a\") " pod="openshift-monitoring/node-exporter-hk7bj"
Jul 29 14:02:39.041011 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:39.040991    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42fb3745-8927-4c55-a75e-8838f8fcd33a-metrics-client-ca\") pod \"node-exporter-hk7bj\" (UID: \"42fb3745-8927-4c55-a75e-8838f8fcd33a\") " pod="openshift-monitoring/node-exporter-hk7bj"
Jul 29 14:02:39.042695 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:39.042676    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/42fb3745-8927-4c55-a75e-8838f8fcd33a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hk7bj\" (UID: \"42fb3745-8927-4c55-a75e-8838f8fcd33a\") " pod="openshift-monitoring/node-exporter-hk7bj"
Jul 29 14:02:39.042757 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:39.042680    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/42fb3745-8927-4c55-a75e-8838f8fcd33a-node-exporter-tls\") pod \"node-exporter-hk7bj\" (UID: \"42fb3745-8927-4c55-a75e-8838f8fcd33a\") " pod="openshift-monitoring/node-exporter-hk7bj"
Jul 29 14:02:39.047940 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:39.047903    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcnmx\" (UniqueName: \"kubernetes.io/projected/42fb3745-8927-4c55-a75e-8838f8fcd33a-kube-api-access-jcnmx\") pod \"node-exporter-hk7bj\" (UID: \"42fb3745-8927-4c55-a75e-8838f8fcd33a\") " pod="openshift-monitoring/node-exporter-hk7bj"
Jul 29 14:02:39.118193 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:39.118113    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:39.173276 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:39.173078    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hk7bj"
Jul 29 14:02:39.184107 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:02:39.184081    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42fb3745_8927_4c55_a75e_8838f8fcd33a.slice/crio-948d7ae78b6d26fe59753058ffecbd540180fea8fdff7e45abaac4edbbd8a6f5 WatchSource:0}: Error finding container 948d7ae78b6d26fe59753058ffecbd540180fea8fdff7e45abaac4edbbd8a6f5: Status 404 returned error can't find the container with id 948d7ae78b6d26fe59753058ffecbd540180fea8fdff7e45abaac4edbbd8a6f5
Jul 29 14:02:39.218553 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:39.218517    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:39.297159 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:39.297129    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:39.684947 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:39.684904    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:39.830387 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:39.830352    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:39.914532 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:39.914497    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:40.085867 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:40.085829    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hk7bj" event={"ID":"42fb3745-8927-4c55-a75e-8838f8fcd33a","Type":"ContainerStarted","Data":"948d7ae78b6d26fe59753058ffecbd540180fea8fdff7e45abaac4edbbd8a6f5"}
Jul 29 14:02:40.144330 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:40.144288    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:40.396626 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:40.396605    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:02:41.090893 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:41.090862    2559 generic.go:356] "Generic (PLEG): container finished" podID="42fb3745-8927-4c55-a75e-8838f8fcd33a" containerID="665ab1224ceee1c45263c7eb54a4803a86ae3ef27c30bcb5b9a86d865ba496fa" exitCode=0
Jul 29 14:02:41.091315 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:41.090907    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hk7bj" event={"ID":"42fb3745-8927-4c55-a75e-8838f8fcd33a","Type":"ContainerDied","Data":"665ab1224ceee1c45263c7eb54a4803a86ae3ef27c30bcb5b9a86d865ba496fa"}
Jul 29 14:02:41.777780 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:41.777738    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:41.857610 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:41.857572    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:41.995479 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:41.995435    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:42.096519 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:42.096438    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hk7bj" event={"ID":"42fb3745-8927-4c55-a75e-8838f8fcd33a","Type":"ContainerStarted","Data":"506c118b81679849effff4da17e820f7b6bf6c9a44831d07f3a9c42d91600e89"}
Jul 29 14:02:42.096519 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:42.096496    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hk7bj" event={"ID":"42fb3745-8927-4c55-a75e-8838f8fcd33a","Type":"ContainerStarted","Data":"982e6825bb74642cd2e76548af0ce93be2a7186438027e93762f1b75e45d184d"}
Jul 29 14:02:42.097036 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:42.096892    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:42.119282 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:42.119227    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hk7bj" podStartSLOduration=2.89337619 podStartE2EDuration="4.119196772s" totalImagesPullingTime="1.161900023s" totalInitContainerRuntime="63.920559ms" isStatefulPod=true podCreationTimestamp="2026-07-29 14:02:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:02:42.117708615 +0000 UTC m=+92.008379949" watchObservedRunningTime="2026-07-29 14:02:42.119196772 +0000 UTC m=+92.009868106"
Jul 29 14:02:42.195537 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:42.195505    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:42.319225 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:42.319194    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:42.417941 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:42.417850    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:42.499285 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:42.499251    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:42.560095 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:42.560049    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:42.612391 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:42.612360    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:42.782042 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:42.782001    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:42.882900 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:42.882871    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:43.000479 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:43.000437    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:43.101565 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:43.101482    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:43.216475 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:43.216441    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:43.301749 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:43.301715    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:43.395216 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:43.395137    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:43.617293 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:43.617254    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:43.717076 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:43.716963    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:43.817816 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:43.817779    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:44.066702 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:44.066668    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:44.175711 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:44.175683    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:44.377341 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:44.377244    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:45.171553 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:45.171515    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:45.411157 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:45.411118    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7d59476db5-kd6tg" podUID="88cdc98a-350e-42ff-a401-7f8ad06de6bb" containerName="registry" containerID="cri-o://3e524fc48b3a4d4f811ad2fa6173b322c4dbe6fdab4134bf60091068d1d706fe" gracePeriod=30
Jul 29 14:02:45.610716 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:45.610680    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:45.648130 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:45.648107    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:02:45.796641 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:45.796607    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/88cdc98a-350e-42ff-a401-7f8ad06de6bb-image-registry-private-configuration\") pod \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") "
Jul 29 14:02:45.796794 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:45.796647    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rg58r\" (UniqueName: \"kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-kube-api-access-rg58r\") pod \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") "
Jul 29 14:02:45.796794 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:45.796684    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/88cdc98a-350e-42ff-a401-7f8ad06de6bb-trusted-ca\") pod \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") "
Jul 29 14:02:45.796794 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:45.796715    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/88cdc98a-350e-42ff-a401-7f8ad06de6bb-ca-trust-extracted\") pod \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") "
Jul 29 14:02:45.796964 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:45.796885    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-registry-tls\") pod \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") "
Jul 29 14:02:45.796964 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:45.796926    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/88cdc98a-350e-42ff-a401-7f8ad06de6bb-installation-pull-secrets\") pod \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") "
Jul 29 14:02:45.796964 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:45.796954    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-bound-sa-token\") pod \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") "
Jul 29 14:02:45.797141 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:45.796985    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/88cdc98a-350e-42ff-a401-7f8ad06de6bb-registry-certificates\") pod \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\" (UID: \"88cdc98a-350e-42ff-a401-7f8ad06de6bb\") "
Jul 29 14:02:45.797192 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:45.797163    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88cdc98a-350e-42ff-a401-7f8ad06de6bb-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "88cdc98a-350e-42ff-a401-7f8ad06de6bb" (UID: "88cdc98a-350e-42ff-a401-7f8ad06de6bb"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 14:02:45.797406 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:45.797292    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/88cdc98a-350e-42ff-a401-7f8ad06de6bb-trusted-ca\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:02:45.797530 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:45.797497    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88cdc98a-350e-42ff-a401-7f8ad06de6bb-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "88cdc98a-350e-42ff-a401-7f8ad06de6bb" (UID: "88cdc98a-350e-42ff-a401-7f8ad06de6bb"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 14:02:45.799194 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:45.799172    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88cdc98a-350e-42ff-a401-7f8ad06de6bb-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "88cdc98a-350e-42ff-a401-7f8ad06de6bb" (UID: "88cdc98a-350e-42ff-a401-7f8ad06de6bb"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 14:02:45.799389 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:45.799354    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "88cdc98a-350e-42ff-a401-7f8ad06de6bb" (UID: "88cdc98a-350e-42ff-a401-7f8ad06de6bb"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:02:45.799487 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:45.799430    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-kube-api-access-rg58r" (OuterVolumeSpecName: "kube-api-access-rg58r") pod "88cdc98a-350e-42ff-a401-7f8ad06de6bb" (UID: "88cdc98a-350e-42ff-a401-7f8ad06de6bb"). InnerVolumeSpecName "kube-api-access-rg58r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:02:45.799583 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:45.799558    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "88cdc98a-350e-42ff-a401-7f8ad06de6bb" (UID: "88cdc98a-350e-42ff-a401-7f8ad06de6bb"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:02:45.799652 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:45.799622    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88cdc98a-350e-42ff-a401-7f8ad06de6bb-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "88cdc98a-350e-42ff-a401-7f8ad06de6bb" (UID: "88cdc98a-350e-42ff-a401-7f8ad06de6bb"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 14:02:45.806017 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:45.805994    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88cdc98a-350e-42ff-a401-7f8ad06de6bb-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "88cdc98a-350e-42ff-a401-7f8ad06de6bb" (UID: "88cdc98a-350e-42ff-a401-7f8ad06de6bb"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:02:45.898618 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:45.898581    2559 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/88cdc98a-350e-42ff-a401-7f8ad06de6bb-image-registry-private-configuration\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:02:45.898618 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:45.898612    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rg58r\" (UniqueName: \"kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-kube-api-access-rg58r\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:02:45.898795 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:45.898627    2559 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/88cdc98a-350e-42ff-a401-7f8ad06de6bb-ca-trust-extracted\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:02:45.898795 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:45.898640    2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-registry-tls\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:02:45.898795 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:45.898655    2559 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/88cdc98a-350e-42ff-a401-7f8ad06de6bb-installation-pull-secrets\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:02:45.898795 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:45.898666    2559 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/88cdc98a-350e-42ff-a401-7f8ad06de6bb-bound-sa-token\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:02:45.898795 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:45.898680    2559 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/88cdc98a-350e-42ff-a401-7f8ad06de6bb-registry-certificates\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:02:46.109841 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:46.109761    2559 generic.go:356] "Generic (PLEG): container finished" podID="88cdc98a-350e-42ff-a401-7f8ad06de6bb" containerID="3e524fc48b3a4d4f811ad2fa6173b322c4dbe6fdab4134bf60091068d1d706fe" exitCode=0
Jul 29 14:02:46.109841 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:46.109812    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d59476db5-kd6tg"
Jul 29 14:02:46.109841 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:46.109830    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d59476db5-kd6tg" event={"ID":"88cdc98a-350e-42ff-a401-7f8ad06de6bb","Type":"ContainerDied","Data":"3e524fc48b3a4d4f811ad2fa6173b322c4dbe6fdab4134bf60091068d1d706fe"}
Jul 29 14:02:46.110041 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:46.109868    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d59476db5-kd6tg" event={"ID":"88cdc98a-350e-42ff-a401-7f8ad06de6bb","Type":"ContainerDied","Data":"fd57d9a9d07a54b96307b69362b3b3aa623055357aea3270780228eea809a6cf"}
Jul 29 14:02:46.110041 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:46.109899    2559 scope.go:117] "RemoveContainer" containerID="3e524fc48b3a4d4f811ad2fa6173b322c4dbe6fdab4134bf60091068d1d706fe"
Jul 29 14:02:46.119010 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:46.118986    2559 scope.go:117] "RemoveContainer" containerID="3e524fc48b3a4d4f811ad2fa6173b322c4dbe6fdab4134bf60091068d1d706fe"
Jul 29 14:02:46.119398 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:02:46.119378    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e524fc48b3a4d4f811ad2fa6173b322c4dbe6fdab4134bf60091068d1d706fe\": container with ID starting with 3e524fc48b3a4d4f811ad2fa6173b322c4dbe6fdab4134bf60091068d1d706fe not found: ID does not exist" containerID="3e524fc48b3a4d4f811ad2fa6173b322c4dbe6fdab4134bf60091068d1d706fe"
Jul 29 14:02:46.119466 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:46.119411    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e524fc48b3a4d4f811ad2fa6173b322c4dbe6fdab4134bf60091068d1d706fe"} err="failed to get container status \"3e524fc48b3a4d4f811ad2fa6173b322c4dbe6fdab4134bf60091068d1d706fe\": rpc error: code = NotFound desc = could not find container \"3e524fc48b3a4d4f811ad2fa6173b322c4dbe6fdab4134bf60091068d1d706fe\": container with ID starting with 3e524fc48b3a4d4f811ad2fa6173b322c4dbe6fdab4134bf60091068d1d706fe not found: ID does not exist"
Jul 29 14:02:46.131836 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:46.131814    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7d59476db5-kd6tg"]
Jul 29 14:02:46.135693 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:46.135672    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7d59476db5-kd6tg"]
Jul 29 14:02:46.635960 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:46.635925    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88cdc98a-350e-42ff-a401-7f8ad06de6bb" path="/var/lib/kubelet/pods/88cdc98a-350e-42ff-a401-7f8ad06de6bb/volumes"
Jul 29 14:02:48.170214 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:48.170174    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:48.271001 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:48.270966    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:51.180787 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:51.180752    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:51.237762 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:51.237727    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:53.579186 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:53.579149    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66d68d769d-8pp8r"]
Jul 29 14:02:53.579626 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:53.579404    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88cdc98a-350e-42ff-a401-7f8ad06de6bb" containerName="registry"
Jul 29 14:02:53.579626 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:53.579414    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="88cdc98a-350e-42ff-a401-7f8ad06de6bb" containerName="registry"
Jul 29 14:02:53.579626 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:53.579455    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="88cdc98a-350e-42ff-a401-7f8ad06de6bb" containerName="registry"
Jul 29 14:02:53.883541 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:53.883427    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66d68d769d-8pp8r"]
Jul 29 14:02:53.883541 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:53.883488    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66d68d769d-8pp8r"
Jul 29 14:02:53.886596 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:53.886574    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 29 14:02:53.886718 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:53.886673    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 29 14:02:53.888283 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:53.888254    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 29 14:02:53.888283 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:53.888261    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 29 14:02:53.888426 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:53.888296    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-g27sc\""
Jul 29 14:02:53.888426 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:53.888338    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 29 14:02:53.888426 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:53.888261    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 29 14:02:53.888426 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:53.888273    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 29 14:02:54.056709 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:54.056670    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-console-oauth-config\") pod \"console-66d68d769d-8pp8r\" (UID: \"6c74ce7c-5ceb-4343-9d0f-c11efeeaa103\") " pod="openshift-console/console-66d68d769d-8pp8r"
Jul 29 14:02:54.056709 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:54.056710    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qp4kw\" (UniqueName: \"kubernetes.io/projected/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-kube-api-access-qp4kw\") pod \"console-66d68d769d-8pp8r\" (UID: \"6c74ce7c-5ceb-4343-9d0f-c11efeeaa103\") " pod="openshift-console/console-66d68d769d-8pp8r"
Jul 29 14:02:54.056926 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:54.056748    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-console-serving-cert\") pod \"console-66d68d769d-8pp8r\" (UID: \"6c74ce7c-5ceb-4343-9d0f-c11efeeaa103\") " pod="openshift-console/console-66d68d769d-8pp8r"
Jul 29 14:02:54.056926 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:54.056786    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-oauth-serving-cert\") pod \"console-66d68d769d-8pp8r\" (UID: \"6c74ce7c-5ceb-4343-9d0f-c11efeeaa103\") " pod="openshift-console/console-66d68d769d-8pp8r"
Jul 29 14:02:54.056926 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:54.056842    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-service-ca\") pod \"console-66d68d769d-8pp8r\" (UID: \"6c74ce7c-5ceb-4343-9d0f-c11efeeaa103\") " pod="openshift-console/console-66d68d769d-8pp8r"
Jul 29 14:02:54.056926 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:54.056893    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-console-config\") pod \"console-66d68d769d-8pp8r\" (UID: \"6c74ce7c-5ceb-4343-9d0f-c11efeeaa103\") " pod="openshift-console/console-66d68d769d-8pp8r"
Jul 29 14:02:54.157662 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:54.157584    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-console-config\") pod \"console-66d68d769d-8pp8r\" (UID: \"6c74ce7c-5ceb-4343-9d0f-c11efeeaa103\") " pod="openshift-console/console-66d68d769d-8pp8r"
Jul 29 14:02:54.157662 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:54.157636    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-console-oauth-config\") pod \"console-66d68d769d-8pp8r\" (UID: \"6c74ce7c-5ceb-4343-9d0f-c11efeeaa103\") " pod="openshift-console/console-66d68d769d-8pp8r"
Jul 29 14:02:54.157662 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:54.157660    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qp4kw\" (UniqueName: \"kubernetes.io/projected/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-kube-api-access-qp4kw\") pod \"console-66d68d769d-8pp8r\" (UID: \"6c74ce7c-5ceb-4343-9d0f-c11efeeaa103\") " pod="openshift-console/console-66d68d769d-8pp8r"
Jul 29 14:02:54.157927 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:54.157699    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-console-serving-cert\") pod \"console-66d68d769d-8pp8r\" (UID: \"6c74ce7c-5ceb-4343-9d0f-c11efeeaa103\") " pod="openshift-console/console-66d68d769d-8pp8r"
Jul 29 14:02:54.157927 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:54.157719    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-oauth-serving-cert\") pod \"console-66d68d769d-8pp8r\" (UID: \"6c74ce7c-5ceb-4343-9d0f-c11efeeaa103\") " pod="openshift-console/console-66d68d769d-8pp8r"
Jul 29 14:02:54.157927 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:54.157745    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-service-ca\") pod \"console-66d68d769d-8pp8r\" (UID: \"6c74ce7c-5ceb-4343-9d0f-c11efeeaa103\") " pod="openshift-console/console-66d68d769d-8pp8r"
Jul 29 14:02:54.158295 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:54.158274    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-console-config\") pod \"console-66d68d769d-8pp8r\" (UID: \"6c74ce7c-5ceb-4343-9d0f-c11efeeaa103\") " pod="openshift-console/console-66d68d769d-8pp8r"
Jul 29 14:02:54.158376 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:54.158334    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-service-ca\") pod \"console-66d68d769d-8pp8r\" (UID: \"6c74ce7c-5ceb-4343-9d0f-c11efeeaa103\") " pod="openshift-console/console-66d68d769d-8pp8r"
Jul 29 14:02:54.158376 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:54.158358    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-oauth-serving-cert\") pod \"console-66d68d769d-8pp8r\" (UID: \"6c74ce7c-5ceb-4343-9d0f-c11efeeaa103\") " pod="openshift-console/console-66d68d769d-8pp8r"
Jul 29 14:02:54.160068 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:54.160035    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-console-oauth-config\") pod \"console-66d68d769d-8pp8r\" (UID: \"6c74ce7c-5ceb-4343-9d0f-c11efeeaa103\") " pod="openshift-console/console-66d68d769d-8pp8r"
Jul 29 14:02:54.160300 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:54.160282    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-console-serving-cert\") pod \"console-66d68d769d-8pp8r\" (UID: \"6c74ce7c-5ceb-4343-9d0f-c11efeeaa103\") " pod="openshift-console/console-66d68d769d-8pp8r"
Jul 29 14:02:54.166338 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:54.166318    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qp4kw\" (UniqueName: \"kubernetes.io/projected/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-kube-api-access-qp4kw\") pod \"console-66d68d769d-8pp8r\" (UID: \"6c74ce7c-5ceb-4343-9d0f-c11efeeaa103\") " pod="openshift-console/console-66d68d769d-8pp8r"
Jul 29 14:02:54.192732 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:54.192703    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66d68d769d-8pp8r"
Jul 29 14:02:54.202248 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:54.202213    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:54.257737 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:54.257703    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:54.351858 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:54.351831    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66d68d769d-8pp8r"]
Jul 29 14:02:54.354553 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:02:54.354526    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c74ce7c_5ceb_4343_9d0f_c11efeeaa103.slice/crio-c1595cc70f525a8c1b62c89b18054e554eb4ce7aca9e080e35a8c92017b6e102 WatchSource:0}: Error finding container c1595cc70f525a8c1b62c89b18054e554eb4ce7aca9e080e35a8c92017b6e102: Status 404 returned error can't find the container with id c1595cc70f525a8c1b62c89b18054e554eb4ce7aca9e080e35a8c92017b6e102
Jul 29 14:02:55.143777 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:55.143741    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66d68d769d-8pp8r" event={"ID":"6c74ce7c-5ceb-4343-9d0f-c11efeeaa103","Type":"ContainerStarted","Data":"c1595cc70f525a8c1b62c89b18054e554eb4ce7aca9e080e35a8c92017b6e102"}
Jul 29 14:02:56.030976 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:56.030947    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-gbj56"
Jul 29 14:02:56.879973 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:56.879937    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:56.980579 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:56.980530    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:02:57.808119 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:02:57.808042    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:03:00.164603 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:00.164552    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66d68d769d-8pp8r" event={"ID":"6c74ce7c-5ceb-4343-9d0f-c11efeeaa103","Type":"ContainerStarted","Data":"b954c3f9b28dcd986e2ccc9d71bd947feacb80de060d89815c0d151f8c5c5f9d"}
Jul 29 14:03:00.183354 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:00.183302    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66d68d769d-8pp8r" podStartSLOduration=1.879232325 podStartE2EDuration="7.183287906s" totalImagesPullingTime="5.304055581s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:02:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:03:00.182192016 +0000 UTC m=+110.072863350" watchObservedRunningTime="2026-07-29 14:03:00.183287906 +0000 UTC m=+110.073959240"
Jul 29 14:03:01.599121 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:01.599079    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:03:01.654712 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:01.654675    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:03:02.201301 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:02.201268    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-9dcc87b6c-pqk2w"]
Jul 29 14:03:02.209375 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:02.209354    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9dcc87b6c-pqk2w"
Jul 29 14:03:02.215685 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:02.215647    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9dcc87b6c-pqk2w"]
Jul 29 14:03:02.219053 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:02.219028    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 29 14:03:02.226486 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:02.226441    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-console-oauth-config\") pod \"console-9dcc87b6c-pqk2w\" (UID: \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\") " pod="openshift-console/console-9dcc87b6c-pqk2w"
Jul 29 14:03:02.226619 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:02.226493    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-console-config\") pod \"console-9dcc87b6c-pqk2w\" (UID: \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\") " pod="openshift-console/console-9dcc87b6c-pqk2w"
Jul 29 14:03:02.226619 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:02.226541    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-oauth-serving-cert\") pod \"console-9dcc87b6c-pqk2w\" (UID: \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\") " pod="openshift-console/console-9dcc87b6c-pqk2w"
Jul 29 14:03:02.226619 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:02.226572    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4xsg\" (UniqueName: \"kubernetes.io/projected/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-kube-api-access-l4xsg\") pod \"console-9dcc87b6c-pqk2w\" (UID: \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\") " pod="openshift-console/console-9dcc87b6c-pqk2w"
Jul 29 14:03:02.226856 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:02.226627    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-trusted-ca-bundle\") pod \"console-9dcc87b6c-pqk2w\" (UID: \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\") " pod="openshift-console/console-9dcc87b6c-pqk2w"
Jul 29 14:03:02.226856 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:02.226664    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-console-serving-cert\") pod \"console-9dcc87b6c-pqk2w\" (UID: \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\") " pod="openshift-console/console-9dcc87b6c-pqk2w"
Jul 29 14:03:02.226856 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:02.226715    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-service-ca\") pod \"console-9dcc87b6c-pqk2w\" (UID: \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\") " pod="openshift-console/console-9dcc87b6c-pqk2w"
Jul 29 14:03:02.327810 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:02.327768    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-oauth-serving-cert\") pod \"console-9dcc87b6c-pqk2w\" (UID: \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\") " pod="openshift-console/console-9dcc87b6c-pqk2w"
Jul 29 14:03:02.327810 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:02.327813    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4xsg\" (UniqueName: \"kubernetes.io/projected/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-kube-api-access-l4xsg\") pod \"console-9dcc87b6c-pqk2w\" (UID: \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\") " pod="openshift-console/console-9dcc87b6c-pqk2w"
Jul 29 14:03:02.328093 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:02.327847    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-trusted-ca-bundle\") pod \"console-9dcc87b6c-pqk2w\" (UID: \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\") " pod="openshift-console/console-9dcc87b6c-pqk2w"
Jul 29 14:03:02.328093 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:02.327880    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-console-serving-cert\") pod \"console-9dcc87b6c-pqk2w\" (UID: \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\") " pod="openshift-console/console-9dcc87b6c-pqk2w"
Jul 29 14:03:02.328093 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:02.327919    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-service-ca\") pod \"console-9dcc87b6c-pqk2w\" (UID: \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\") " pod="openshift-console/console-9dcc87b6c-pqk2w"
Jul 29 14:03:02.328093 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:02.327979    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-console-oauth-config\") pod \"console-9dcc87b6c-pqk2w\" (UID: \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\") " pod="openshift-console/console-9dcc87b6c-pqk2w"
Jul 29 14:03:02.328093 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:02.328037    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-console-config\") pod \"console-9dcc87b6c-pqk2w\" (UID: \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\") " pod="openshift-console/console-9dcc87b6c-pqk2w"
Jul 29 14:03:02.328634 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:02.328605    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-oauth-serving-cert\") pod \"console-9dcc87b6c-pqk2w\" (UID: \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\") " pod="openshift-console/console-9dcc87b6c-pqk2w"
Jul 29 14:03:02.328813 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:02.328793    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-console-config\") pod \"console-9dcc87b6c-pqk2w\" (UID: \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\") " pod="openshift-console/console-9dcc87b6c-pqk2w"
Jul 29 14:03:02.328899 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:02.328832    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-service-ca\") pod \"console-9dcc87b6c-pqk2w\" (UID: \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\") " pod="openshift-console/console-9dcc87b6c-pqk2w"
Jul 29 14:03:02.329027 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:02.329006    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-trusted-ca-bundle\") pod \"console-9dcc87b6c-pqk2w\" (UID: \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\") " pod="openshift-console/console-9dcc87b6c-pqk2w"
Jul 29 14:03:02.330479 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:02.330459    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-console-serving-cert\") pod \"console-9dcc87b6c-pqk2w\" (UID: \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\") " pod="openshift-console/console-9dcc87b6c-pqk2w"
Jul 29 14:03:02.330560 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:02.330490    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-console-oauth-config\") pod \"console-9dcc87b6c-pqk2w\" (UID: \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\") " pod="openshift-console/console-9dcc87b6c-pqk2w"
Jul 29 14:03:02.336636 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:02.336608    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4xsg\" (UniqueName: \"kubernetes.io/projected/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-kube-api-access-l4xsg\") pod \"console-9dcc87b6c-pqk2w\" (UID: \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\") " pod="openshift-console/console-9dcc87b6c-pqk2w"
Jul 29 14:03:02.519639 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:02.519602    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9dcc87b6c-pqk2w"
Jul 29 14:03:02.670370 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:02.670265    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9dcc87b6c-pqk2w"]
Jul 29 14:03:02.672712 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:03:02.672687    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a3f5275_f909_4f04_a43b_8e49f8a38d9b.slice/crio-84c0977df1ab8d9422082972adb39fda6dd78284abb2f663ad7f722b1ad3625d WatchSource:0}: Error finding container 84c0977df1ab8d9422082972adb39fda6dd78284abb2f663ad7f722b1ad3625d: Status 404 returned error can't find the container with id 84c0977df1ab8d9422082972adb39fda6dd78284abb2f663ad7f722b1ad3625d
Jul 29 14:03:03.176266 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:03.176180    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9dcc87b6c-pqk2w" event={"ID":"6a3f5275-f909-4f04-a43b-8e49f8a38d9b","Type":"ContainerStarted","Data":"85fc78cab5965d89ce975f997362fc354529dd5c33696bfe3ad92a5f8ef9e7d4"}
Jul 29 14:03:03.176266 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:03.176232    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9dcc87b6c-pqk2w" event={"ID":"6a3f5275-f909-4f04-a43b-8e49f8a38d9b","Type":"ContainerStarted","Data":"84c0977df1ab8d9422082972adb39fda6dd78284abb2f663ad7f722b1ad3625d"}
Jul 29 14:03:03.196023 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:03.195973    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-9dcc87b6c-pqk2w" podStartSLOduration=1.195958924 podStartE2EDuration="1.195958924s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:03:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:03:03.19454194 +0000 UTC m=+113.085213274" watchObservedRunningTime="2026-07-29 14:03:03.195958924 +0000 UTC m=+113.086630259"
Jul 29 14:03:04.193010 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:04.192975    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66d68d769d-8pp8r"
Jul 29 14:03:04.193599 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:04.193086    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66d68d769d-8pp8r"
Jul 29 14:03:04.197631 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:04.197605    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66d68d769d-8pp8r"
Jul 29 14:03:05.184319 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:05.184288    2559 generic.go:356] "Generic (PLEG): container finished" podID="49f8acf6-08de-4400-adfd-fa32d9a066b5" containerID="009d292d17495cf90fd93e02e140933e1f24eee413fef6e7d79ca904badd9d80" exitCode=0
Jul 29 14:03:05.184549 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:05.184370    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-5nxst" event={"ID":"49f8acf6-08de-4400-adfd-fa32d9a066b5","Type":"ContainerDied","Data":"009d292d17495cf90fd93e02e140933e1f24eee413fef6e7d79ca904badd9d80"}
Jul 29 14:03:05.184792 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:05.184773    2559 scope.go:117] "RemoveContainer" containerID="009d292d17495cf90fd93e02e140933e1f24eee413fef6e7d79ca904badd9d80"
Jul 29 14:03:05.188589 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:05.188564    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66d68d769d-8pp8r"
Jul 29 14:03:05.464211 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:05.464129    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:03:05.517091 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:05.517049    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:03:06.190049 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:06.190017    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-5nxst" event={"ID":"49f8acf6-08de-4400-adfd-fa32d9a066b5","Type":"ContainerStarted","Data":"86c634393becbab29ae1aa2819ec5245267af9d09f2cca960646500adfcd54fa"}
Jul 29 14:03:12.519941 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:12.519900    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-9dcc87b6c-pqk2w"
Jul 29 14:03:12.520430 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:12.519987    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-9dcc87b6c-pqk2w"
Jul 29 14:03:12.524876 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:12.524850    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-9dcc87b6c-pqk2w"
Jul 29 14:03:13.078902 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:13.078866    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:03:13.129829 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:13.129802    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hk7bj_42fb3745-8927-4c55-a75e-8838f8fcd33a/init-textfile/0.log"
Jul 29 14:03:13.216515 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:13.216491    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-9dcc87b6c-pqk2w"
Jul 29 14:03:13.263552 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:13.263512    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66d68d769d-8pp8r"]
Jul 29 14:03:13.336043 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:13.335975    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hk7bj_42fb3745-8927-4c55-a75e-8838f8fcd33a/node-exporter/0.log"
Jul 29 14:03:13.529555 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:13.529520    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hk7bj_42fb3745-8927-4c55-a75e-8838f8fcd33a/kube-rbac-proxy/0.log"
Jul 29 14:03:13.731790 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:13.731708    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:03:14.615812 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:14.615777    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:03:14.675984 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:14.675955    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:03:15.415135 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:15.415098    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:03:16.011508 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:16.011476    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:03:16.071075 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:16.071025    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:03:19.330507 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:19.330479    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-zmjrn_88728d56-a8af-49d4-a52c-674e9ac76cd9/networking-console-plugin/0.log"
Jul 29 14:03:19.775968 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:19.775932    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:03:19.878025 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:19.877990    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:03:19.930550 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:19.930521    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-66d68d769d-8pp8r_6c74ce7c-5ceb-4343-9d0f-c11efeeaa103/console/0.log"
Jul 29 14:03:19.939436 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:19.939408    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:03:20.130584 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:20.130508    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-9dcc87b6c-pqk2w_6a3f5275-f909-4f04-a43b-8e49f8a38d9b/console/0.log"
Jul 29 14:03:20.929906 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:20.929879    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b76j4_697b63cb-b6c9-4178-892b-822d19e03056/serve-healthcheck-canary/0.log"
Jul 29 14:03:23.833655 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:23.833621    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:03:26.874649 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:26.874613    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:03:26.975854 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:26.975819    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:03:38.284444 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:38.284377    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-66d68d769d-8pp8r" podUID="6c74ce7c-5ceb-4343-9d0f-c11efeeaa103" containerName="console" containerID="cri-o://b954c3f9b28dcd986e2ccc9d71bd947feacb80de060d89815c0d151f8c5c5f9d" gracePeriod=15
Jul 29 14:03:38.517623 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:38.517601    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66d68d769d-8pp8r"
Jul 29 14:03:38.586531 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:38.586444    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-console-oauth-config\") pod \"6c74ce7c-5ceb-4343-9d0f-c11efeeaa103\" (UID: \"6c74ce7c-5ceb-4343-9d0f-c11efeeaa103\") "
Jul 29 14:03:38.586531 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:38.586479    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-oauth-serving-cert\") pod \"6c74ce7c-5ceb-4343-9d0f-c11efeeaa103\" (UID: \"6c74ce7c-5ceb-4343-9d0f-c11efeeaa103\") "
Jul 29 14:03:38.586531 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:38.586501    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qp4kw\" (UniqueName: \"kubernetes.io/projected/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-kube-api-access-qp4kw\") pod \"6c74ce7c-5ceb-4343-9d0f-c11efeeaa103\" (UID: \"6c74ce7c-5ceb-4343-9d0f-c11efeeaa103\") "
Jul 29 14:03:38.586531 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:38.586526    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-service-ca\") pod \"6c74ce7c-5ceb-4343-9d0f-c11efeeaa103\" (UID: \"6c74ce7c-5ceb-4343-9d0f-c11efeeaa103\") "
Jul 29 14:03:38.586837 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:38.586583    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-console-config\") pod \"6c74ce7c-5ceb-4343-9d0f-c11efeeaa103\" (UID: \"6c74ce7c-5ceb-4343-9d0f-c11efeeaa103\") "
Jul 29 14:03:38.586837 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:38.586610    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-console-serving-cert\") pod \"6c74ce7c-5ceb-4343-9d0f-c11efeeaa103\" (UID: \"6c74ce7c-5ceb-4343-9d0f-c11efeeaa103\") "
Jul 29 14:03:38.587003 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:38.586976    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6c74ce7c-5ceb-4343-9d0f-c11efeeaa103" (UID: "6c74ce7c-5ceb-4343-9d0f-c11efeeaa103"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 14:03:38.587100 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:38.587052    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-service-ca" (OuterVolumeSpecName: "service-ca") pod "6c74ce7c-5ceb-4343-9d0f-c11efeeaa103" (UID: "6c74ce7c-5ceb-4343-9d0f-c11efeeaa103"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 14:03:38.587164 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:38.587113    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-console-config" (OuterVolumeSpecName: "console-config") pod "6c74ce7c-5ceb-4343-9d0f-c11efeeaa103" (UID: "6c74ce7c-5ceb-4343-9d0f-c11efeeaa103"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 14:03:38.588822 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:38.588791    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6c74ce7c-5ceb-4343-9d0f-c11efeeaa103" (UID: "6c74ce7c-5ceb-4343-9d0f-c11efeeaa103"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 14:03:38.588822 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:38.588806    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6c74ce7c-5ceb-4343-9d0f-c11efeeaa103" (UID: "6c74ce7c-5ceb-4343-9d0f-c11efeeaa103"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 14:03:38.588926 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:38.588894    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-kube-api-access-qp4kw" (OuterVolumeSpecName: "kube-api-access-qp4kw") pod "6c74ce7c-5ceb-4343-9d0f-c11efeeaa103" (UID: "6c74ce7c-5ceb-4343-9d0f-c11efeeaa103"). InnerVolumeSpecName "kube-api-access-qp4kw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:03:38.687823 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:38.687792    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qp4kw\" (UniqueName: \"kubernetes.io/projected/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-kube-api-access-qp4kw\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:03:38.687823 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:38.687818    2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-service-ca\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:03:38.687823 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:38.687827    2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-console-config\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:03:38.687998 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:38.687834    2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-console-serving-cert\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:03:38.687998 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:38.687843    2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-console-oauth-config\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:03:38.687998 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:38.687851    2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103-oauth-serving-cert\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:03:39.303369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:39.303332    2559 generic.go:356] "Generic (PLEG): container finished" podID="6c74ce7c-5ceb-4343-9d0f-c11efeeaa103" containerID="b954c3f9b28dcd986e2ccc9d71bd947feacb80de060d89815c0d151f8c5c5f9d" exitCode=0
Jul 29 14:03:39.303853 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:39.303382    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66d68d769d-8pp8r" event={"ID":"6c74ce7c-5ceb-4343-9d0f-c11efeeaa103","Type":"ContainerDied","Data":"b954c3f9b28dcd986e2ccc9d71bd947feacb80de060d89815c0d151f8c5c5f9d"}
Jul 29 14:03:39.303853 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:39.303386    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66d68d769d-8pp8r"
Jul 29 14:03:39.303853 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:39.303421    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66d68d769d-8pp8r" event={"ID":"6c74ce7c-5ceb-4343-9d0f-c11efeeaa103","Type":"ContainerDied","Data":"c1595cc70f525a8c1b62c89b18054e554eb4ce7aca9e080e35a8c92017b6e102"}
Jul 29 14:03:39.303853 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:39.303440    2559 scope.go:117] "RemoveContainer" containerID="b954c3f9b28dcd986e2ccc9d71bd947feacb80de060d89815c0d151f8c5c5f9d"
Jul 29 14:03:39.311104 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:39.311053    2559 scope.go:117] "RemoveContainer" containerID="b954c3f9b28dcd986e2ccc9d71bd947feacb80de060d89815c0d151f8c5c5f9d"
Jul 29 14:03:39.311362 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:03:39.311323    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b954c3f9b28dcd986e2ccc9d71bd947feacb80de060d89815c0d151f8c5c5f9d\": container with ID starting with b954c3f9b28dcd986e2ccc9d71bd947feacb80de060d89815c0d151f8c5c5f9d not found: ID does not exist" containerID="b954c3f9b28dcd986e2ccc9d71bd947feacb80de060d89815c0d151f8c5c5f9d"
Jul 29 14:03:39.311362 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:39.311352    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b954c3f9b28dcd986e2ccc9d71bd947feacb80de060d89815c0d151f8c5c5f9d"} err="failed to get container status \"b954c3f9b28dcd986e2ccc9d71bd947feacb80de060d89815c0d151f8c5c5f9d\": rpc error: code = NotFound desc = could not find container \"b954c3f9b28dcd986e2ccc9d71bd947feacb80de060d89815c0d151f8c5c5f9d\": container with ID starting with b954c3f9b28dcd986e2ccc9d71bd947feacb80de060d89815c0d151f8c5c5f9d not found: ID does not exist"
Jul 29 14:03:39.321627 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:39.321602    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66d68d769d-8pp8r"]
Jul 29 14:03:39.326843 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:39.326813    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-66d68d769d-8pp8r"]
Jul 29 14:03:40.635511 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:40.635478    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c74ce7c-5ceb-4343-9d0f-c11efeeaa103" path="/var/lib/kubelet/pods/6c74ce7c-5ceb-4343-9d0f-c11efeeaa103/volumes"
Jul 29 14:03:54.927274 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:54.927235    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:03:54.987345 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:54.987308    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:03:58.359155 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:58.359121    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6fbcfc5544-9kctz"]
Jul 29 14:03:58.359628 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:58.359479    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c74ce7c-5ceb-4343-9d0f-c11efeeaa103" containerName="console"
Jul 29 14:03:58.359628 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:58.359498    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c74ce7c-5ceb-4343-9d0f-c11efeeaa103" containerName="console"
Jul 29 14:03:58.359628 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:58.359561    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c74ce7c-5ceb-4343-9d0f-c11efeeaa103" containerName="console"
Jul 29 14:03:58.364098 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:58.364077    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fbcfc5544-9kctz"
Jul 29 14:03:58.378169 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:58.378100    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6fbcfc5544-9kctz"]
Jul 29 14:03:58.424936 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:58.424910    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:03:58.427880 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:58.427860    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/25ba9889-7adb-4d8d-94b0-c05fd96a1981-console-serving-cert\") pod \"console-6fbcfc5544-9kctz\" (UID: \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\") " pod="openshift-console/console-6fbcfc5544-9kctz"
Jul 29 14:03:58.427967 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:58.427893    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25ba9889-7adb-4d8d-94b0-c05fd96a1981-trusted-ca-bundle\") pod \"console-6fbcfc5544-9kctz\" (UID: \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\") " pod="openshift-console/console-6fbcfc5544-9kctz"
Jul 29 14:03:58.427967 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:58.427911    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/25ba9889-7adb-4d8d-94b0-c05fd96a1981-console-config\") pod \"console-6fbcfc5544-9kctz\" (UID: \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\") " pod="openshift-console/console-6fbcfc5544-9kctz"
Jul 29 14:03:58.428089 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:58.427962    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kxlv\" (UniqueName: \"kubernetes.io/projected/25ba9889-7adb-4d8d-94b0-c05fd96a1981-kube-api-access-9kxlv\") pod \"console-6fbcfc5544-9kctz\" (UID: \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\") " pod="openshift-console/console-6fbcfc5544-9kctz"
Jul 29 14:03:58.428089 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:58.428018    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/25ba9889-7adb-4d8d-94b0-c05fd96a1981-service-ca\") pod \"console-6fbcfc5544-9kctz\" (UID: \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\") " pod="openshift-console/console-6fbcfc5544-9kctz"
Jul 29 14:03:58.428187 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:58.428100    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/25ba9889-7adb-4d8d-94b0-c05fd96a1981-oauth-serving-cert\") pod \"console-6fbcfc5544-9kctz\" (UID: \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\") " pod="openshift-console/console-6fbcfc5544-9kctz"
Jul 29 14:03:58.428187 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:58.428128    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/25ba9889-7adb-4d8d-94b0-c05fd96a1981-console-oauth-config\") pod \"console-6fbcfc5544-9kctz\" (UID: \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\") " pod="openshift-console/console-6fbcfc5544-9kctz"
Jul 29 14:03:58.479098 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:58.479048    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:03:58.529051 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:58.529015    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/25ba9889-7adb-4d8d-94b0-c05fd96a1981-console-oauth-config\") pod \"console-6fbcfc5544-9kctz\" (UID: \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\") " pod="openshift-console/console-6fbcfc5544-9kctz"
Jul 29 14:03:58.529205 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:58.529077    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/25ba9889-7adb-4d8d-94b0-c05fd96a1981-console-serving-cert\") pod \"console-6fbcfc5544-9kctz\" (UID: \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\") " pod="openshift-console/console-6fbcfc5544-9kctz"
Jul 29 14:03:58.529205 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:58.529108    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25ba9889-7adb-4d8d-94b0-c05fd96a1981-trusted-ca-bundle\") pod \"console-6fbcfc5544-9kctz\" (UID: \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\") " pod="openshift-console/console-6fbcfc5544-9kctz"
Jul 29 14:03:58.529205 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:58.529127    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/25ba9889-7adb-4d8d-94b0-c05fd96a1981-console-config\") pod \"console-6fbcfc5544-9kctz\" (UID: \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\") " pod="openshift-console/console-6fbcfc5544-9kctz"
Jul 29 14:03:58.529205 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:58.529150    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kxlv\" (UniqueName: \"kubernetes.io/projected/25ba9889-7adb-4d8d-94b0-c05fd96a1981-kube-api-access-9kxlv\") pod \"console-6fbcfc5544-9kctz\" (UID: \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\") " pod="openshift-console/console-6fbcfc5544-9kctz"
Jul 29 14:03:58.529205 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:58.529186    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/25ba9889-7adb-4d8d-94b0-c05fd96a1981-service-ca\") pod \"console-6fbcfc5544-9kctz\" (UID: \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\") " pod="openshift-console/console-6fbcfc5544-9kctz"
Jul 29 14:03:58.529436 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:58.529335    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/25ba9889-7adb-4d8d-94b0-c05fd96a1981-oauth-serving-cert\") pod \"console-6fbcfc5544-9kctz\" (UID: \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\") " pod="openshift-console/console-6fbcfc5544-9kctz"
Jul 29 14:03:58.529878 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:58.529846    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/25ba9889-7adb-4d8d-94b0-c05fd96a1981-console-config\") pod \"console-6fbcfc5544-9kctz\" (UID: \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\") " pod="openshift-console/console-6fbcfc5544-9kctz"
Jul 29 14:03:58.529987 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:58.529955    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/25ba9889-7adb-4d8d-94b0-c05fd96a1981-service-ca\") pod \"console-6fbcfc5544-9kctz\" (UID: \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\") " pod="openshift-console/console-6fbcfc5544-9kctz"
Jul 29 14:03:58.530141 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:58.530118    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/25ba9889-7adb-4d8d-94b0-c05fd96a1981-oauth-serving-cert\") pod \"console-6fbcfc5544-9kctz\" (UID: \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\") " pod="openshift-console/console-6fbcfc5544-9kctz"
Jul 29 14:03:58.530224 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:58.530125    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25ba9889-7adb-4d8d-94b0-c05fd96a1981-trusted-ca-bundle\") pod \"console-6fbcfc5544-9kctz\" (UID: \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\") " pod="openshift-console/console-6fbcfc5544-9kctz"
Jul 29 14:03:58.531506 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:58.531487    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/25ba9889-7adb-4d8d-94b0-c05fd96a1981-console-oauth-config\") pod \"console-6fbcfc5544-9kctz\" (UID: \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\") " pod="openshift-console/console-6fbcfc5544-9kctz"
Jul 29 14:03:58.531631 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:58.531613    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/25ba9889-7adb-4d8d-94b0-c05fd96a1981-console-serving-cert\") pod \"console-6fbcfc5544-9kctz\" (UID: \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\") " pod="openshift-console/console-6fbcfc5544-9kctz"
Jul 29 14:03:58.541638 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:58.541616    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kxlv\" (UniqueName: \"kubernetes.io/projected/25ba9889-7adb-4d8d-94b0-c05fd96a1981-kube-api-access-9kxlv\") pod \"console-6fbcfc5544-9kctz\" (UID: \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\") " pod="openshift-console/console-6fbcfc5544-9kctz"
Jul 29 14:03:58.570916 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:58.570884    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:03:58.626497 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:58.626411    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:03:58.675951 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:58.675921    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fbcfc5544-9kctz"
Jul 29 14:03:58.802102 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:58.802043    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6fbcfc5544-9kctz"]
Jul 29 14:03:58.804945 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:03:58.804916    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25ba9889_7adb_4d8d_94b0_c05fd96a1981.slice/crio-4db8b35785140abdaaebf294eaf4cefe0866d4f700d77d35356c51063279d581 WatchSource:0}: Error finding container 4db8b35785140abdaaebf294eaf4cefe0866d4f700d77d35356c51063279d581: Status 404 returned error can't find the container with id 4db8b35785140abdaaebf294eaf4cefe0866d4f700d77d35356c51063279d581
Jul 29 14:03:59.160461 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:59.160422    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:03:59.223601 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:59.223564    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:03:59.370785 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:59.370750    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fbcfc5544-9kctz" event={"ID":"25ba9889-7adb-4d8d-94b0-c05fd96a1981","Type":"ContainerStarted","Data":"5a1cba4ca88d19b3c37df4c3d1b6357b2cd77889a3fd92cb25c63dd0a4cbf54f"}
Jul 29 14:03:59.371187 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:59.370797    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fbcfc5544-9kctz" event={"ID":"25ba9889-7adb-4d8d-94b0-c05fd96a1981","Type":"ContainerStarted","Data":"4db8b35785140abdaaebf294eaf4cefe0866d4f700d77d35356c51063279d581"}
Jul 29 14:03:59.392906 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:03:59.392854    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6fbcfc5544-9kctz" podStartSLOduration=1.392840441 podStartE2EDuration="1.392840441s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:03:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:03:59.391702226 +0000 UTC m=+169.282373561" watchObservedRunningTime="2026-07-29 14:03:59.392840441 +0000 UTC m=+169.283511776"
Jul 29 14:04:06.970363 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:06.970278    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:04:07.024073 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:07.024024    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:04:08.676706 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:08.676673    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6fbcfc5544-9kctz"
Jul 29 14:04:08.677131 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:08.676721    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6fbcfc5544-9kctz"
Jul 29 14:04:08.681636 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:08.681609    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6fbcfc5544-9kctz"
Jul 29 14:04:09.411018 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:09.410988    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6fbcfc5544-9kctz"
Jul 29 14:04:09.451726 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:09.451695    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9dcc87b6c-pqk2w"]
Jul 29 14:04:09.549190 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:09.549153    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:04:09.606370 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:09.606338    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:04:09.663405 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:09.663282    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:04:11.934756 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:11.934678    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:04:12.035667 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:12.035630    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:04:34.473430 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:34.473365    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-9dcc87b6c-pqk2w" podUID="6a3f5275-f909-4f04-a43b-8e49f8a38d9b" containerName="console" containerID="cri-o://85fc78cab5965d89ce975f997362fc354529dd5c33696bfe3ad92a5f8ef9e7d4" gracePeriod=15
Jul 29 14:04:34.713743 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:34.713715    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9dcc87b6c-pqk2w"
Jul 29 14:04:34.898398 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:34.898359    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-console-serving-cert\") pod \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\" (UID: \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\") "
Jul 29 14:04:34.898398 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:34.898408    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-oauth-serving-cert\") pod \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\" (UID: \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\") "
Jul 29 14:04:34.898648 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:34.898446    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-console-oauth-config\") pod \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\" (UID: \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\") "
Jul 29 14:04:34.898648 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:34.898469    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-console-config\") pod \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\" (UID: \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\") "
Jul 29 14:04:34.898648 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:34.898499    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4xsg\" (UniqueName: \"kubernetes.io/projected/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-kube-api-access-l4xsg\") pod \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\" (UID: \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\") "
Jul 29 14:04:34.898648 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:34.898521    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-service-ca\") pod \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\" (UID: \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\") "
Jul 29 14:04:34.898648 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:34.898597    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-trusted-ca-bundle\") pod \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\" (UID: \"6a3f5275-f909-4f04-a43b-8e49f8a38d9b\") "
Jul 29 14:04:34.899110 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:34.899033    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-console-config" (OuterVolumeSpecName: "console-config") pod "6a3f5275-f909-4f04-a43b-8e49f8a38d9b" (UID: "6a3f5275-f909-4f04-a43b-8e49f8a38d9b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 14:04:34.899110 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:34.898958    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6a3f5275-f909-4f04-a43b-8e49f8a38d9b" (UID: "6a3f5275-f909-4f04-a43b-8e49f8a38d9b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 14:04:34.899274 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:34.899135    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6a3f5275-f909-4f04-a43b-8e49f8a38d9b" (UID: "6a3f5275-f909-4f04-a43b-8e49f8a38d9b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 14:04:34.899274 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:34.899128    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-service-ca" (OuterVolumeSpecName: "service-ca") pod "6a3f5275-f909-4f04-a43b-8e49f8a38d9b" (UID: "6a3f5275-f909-4f04-a43b-8e49f8a38d9b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 14:04:34.900591 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:34.900572    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6a3f5275-f909-4f04-a43b-8e49f8a38d9b" (UID: "6a3f5275-f909-4f04-a43b-8e49f8a38d9b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 14:04:34.900782 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:34.900761    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-kube-api-access-l4xsg" (OuterVolumeSpecName: "kube-api-access-l4xsg") pod "6a3f5275-f909-4f04-a43b-8e49f8a38d9b" (UID: "6a3f5275-f909-4f04-a43b-8e49f8a38d9b"). InnerVolumeSpecName "kube-api-access-l4xsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:04:34.900842 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:34.900819    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6a3f5275-f909-4f04-a43b-8e49f8a38d9b" (UID: "6a3f5275-f909-4f04-a43b-8e49f8a38d9b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 14:04:34.999511 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:34.999475    2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-console-serving-cert\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:04:34.999511 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:34.999506    2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-oauth-serving-cert\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:04:34.999511 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:34.999518    2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-console-oauth-config\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:04:34.999731 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:34.999527    2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-console-config\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:04:34.999731 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:34.999535    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4xsg\" (UniqueName: \"kubernetes.io/projected/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-kube-api-access-l4xsg\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:04:34.999731 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:34.999544    2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-service-ca\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:04:34.999731 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:34.999552    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a3f5275-f909-4f04-a43b-8e49f8a38d9b-trusted-ca-bundle\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:04:35.493809 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:35.493774    2559 generic.go:356] "Generic (PLEG): container finished" podID="6a3f5275-f909-4f04-a43b-8e49f8a38d9b" containerID="85fc78cab5965d89ce975f997362fc354529dd5c33696bfe3ad92a5f8ef9e7d4" exitCode=0
Jul 29 14:04:35.494265 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:35.493825    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9dcc87b6c-pqk2w" event={"ID":"6a3f5275-f909-4f04-a43b-8e49f8a38d9b","Type":"ContainerDied","Data":"85fc78cab5965d89ce975f997362fc354529dd5c33696bfe3ad92a5f8ef9e7d4"}
Jul 29 14:04:35.494265 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:35.493838    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9dcc87b6c-pqk2w"
Jul 29 14:04:35.494265 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:35.493897    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9dcc87b6c-pqk2w" event={"ID":"6a3f5275-f909-4f04-a43b-8e49f8a38d9b","Type":"ContainerDied","Data":"84c0977df1ab8d9422082972adb39fda6dd78284abb2f663ad7f722b1ad3625d"}
Jul 29 14:04:35.494265 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:35.493919    2559 scope.go:117] "RemoveContainer" containerID="85fc78cab5965d89ce975f997362fc354529dd5c33696bfe3ad92a5f8ef9e7d4"
Jul 29 14:04:35.502264 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:35.502243    2559 scope.go:117] "RemoveContainer" containerID="85fc78cab5965d89ce975f997362fc354529dd5c33696bfe3ad92a5f8ef9e7d4"
Jul 29 14:04:35.502516 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:04:35.502496    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85fc78cab5965d89ce975f997362fc354529dd5c33696bfe3ad92a5f8ef9e7d4\": container with ID starting with 85fc78cab5965d89ce975f997362fc354529dd5c33696bfe3ad92a5f8ef9e7d4 not found: ID does not exist" containerID="85fc78cab5965d89ce975f997362fc354529dd5c33696bfe3ad92a5f8ef9e7d4"
Jul 29 14:04:35.502573 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:35.502525    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85fc78cab5965d89ce975f997362fc354529dd5c33696bfe3ad92a5f8ef9e7d4"} err="failed to get container status \"85fc78cab5965d89ce975f997362fc354529dd5c33696bfe3ad92a5f8ef9e7d4\": rpc error: code = NotFound desc = could not find container \"85fc78cab5965d89ce975f997362fc354529dd5c33696bfe3ad92a5f8ef9e7d4\": container with ID starting with 85fc78cab5965d89ce975f997362fc354529dd5c33696bfe3ad92a5f8ef9e7d4 not found: ID does not exist"
Jul 29 14:04:35.514297 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:35.514274    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9dcc87b6c-pqk2w"]
Jul 29 14:04:35.516928 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:35.516910    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-9dcc87b6c-pqk2w"]
Jul 29 14:04:36.635373 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:36.635339    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a3f5275-f909-4f04-a43b-8e49f8a38d9b" path="/var/lib/kubelet/pods/6a3f5275-f909-4f04-a43b-8e49f8a38d9b/volumes"
Jul 29 14:04:39.829751 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:39.829720    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:04:39.889824 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:04:39.889788    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:05:11.163536 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:11.163496    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:05:11.260965 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:11.260920    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:05:37.968478 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:37.968379    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:05:38.846794 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:38.846760    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:05:39.937496 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:39.937464    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-64qdf"]
Jul 29 14:05:39.937893 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:39.937721    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a3f5275-f909-4f04-a43b-8e49f8a38d9b" containerName="console"
Jul 29 14:05:39.937893 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:39.937735    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a3f5275-f909-4f04-a43b-8e49f8a38d9b" containerName="console"
Jul 29 14:05:39.937893 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:39.937774    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a3f5275-f909-4f04-a43b-8e49f8a38d9b" containerName="console"
Jul 29 14:05:39.940518 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:39.940495    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-64qdf"
Jul 29 14:05:39.944169 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:39.944145    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 29 14:05:39.944294 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:39.944210    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-qmmmz\""
Jul 29 14:05:39.944810 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:39.944794    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 29 14:05:39.956356 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:39.956334    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-64qdf"]
Jul 29 14:05:40.017478 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:40.017439    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7dc1af9-44c3-4277-9911-d41e6b6c3749-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-64qdf\" (UID: \"a7dc1af9-44c3-4277-9911-d41e6b6c3749\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-64qdf"
Jul 29 14:05:40.017645 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:40.017516    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swrcr\" (UniqueName: \"kubernetes.io/projected/a7dc1af9-44c3-4277-9911-d41e6b6c3749-kube-api-access-swrcr\") pod \"cert-manager-operator-controller-manager-55c444bf86-64qdf\" (UID: \"a7dc1af9-44c3-4277-9911-d41e6b6c3749\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-64qdf"
Jul 29 14:05:40.118346 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:40.118312    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swrcr\" (UniqueName: \"kubernetes.io/projected/a7dc1af9-44c3-4277-9911-d41e6b6c3749-kube-api-access-swrcr\") pod \"cert-manager-operator-controller-manager-55c444bf86-64qdf\" (UID: \"a7dc1af9-44c3-4277-9911-d41e6b6c3749\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-64qdf"
Jul 29 14:05:40.118495 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:40.118367    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7dc1af9-44c3-4277-9911-d41e6b6c3749-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-64qdf\" (UID: \"a7dc1af9-44c3-4277-9911-d41e6b6c3749\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-64qdf"
Jul 29 14:05:40.118696 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:40.118680    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7dc1af9-44c3-4277-9911-d41e6b6c3749-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-64qdf\" (UID: \"a7dc1af9-44c3-4277-9911-d41e6b6c3749\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-64qdf"
Jul 29 14:05:40.128938 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:40.128907    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swrcr\" (UniqueName: \"kubernetes.io/projected/a7dc1af9-44c3-4277-9911-d41e6b6c3749-kube-api-access-swrcr\") pod \"cert-manager-operator-controller-manager-55c444bf86-64qdf\" (UID: \"a7dc1af9-44c3-4277-9911-d41e6b6c3749\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-64qdf"
Jul 29 14:05:40.249493 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:40.249463    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-64qdf"
Jul 29 14:05:40.384509 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:40.384354    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-64qdf"]
Jul 29 14:05:40.387574 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:05:40.387548    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7dc1af9_44c3_4277_9911_d41e6b6c3749.slice/crio-18e1d0a38e0c886c13972bc381012ce3fa72a0f134819b026de228ef1b72234e WatchSource:0}: Error finding container 18e1d0a38e0c886c13972bc381012ce3fa72a0f134819b026de228ef1b72234e: Status 404 returned error can't find the container with id 18e1d0a38e0c886c13972bc381012ce3fa72a0f134819b026de228ef1b72234e
Jul 29 14:05:40.709011 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:40.708934    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-64qdf" event={"ID":"a7dc1af9-44c3-4277-9911-d41e6b6c3749","Type":"ContainerStarted","Data":"18e1d0a38e0c886c13972bc381012ce3fa72a0f134819b026de228ef1b72234e"}
Jul 29 14:05:41.777834 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:41.777793    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:05:41.847491 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:41.847415    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:05:44.354745 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:44.354704    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:05:44.548342 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:44.548308    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:05:44.723108 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:44.723007    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-64qdf" event={"ID":"a7dc1af9-44c3-4277-9911-d41e6b6c3749","Type":"ContainerStarted","Data":"9174ac6a23a73d1b6d2eaec5e35db8be713619a49fb199ad967cb4296fcdca43"}
Jul 29 14:05:45.222313 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:45.222281    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:05:45.339503 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:45.339473    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:05:46.757123 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:46.757080    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:05:46.965408 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:46.965370    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:05:47.215325 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:47.215230    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:05:47.437637 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:47.437606    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:05:51.135479 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:51.135422    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-64qdf" podStartSLOduration=8.828446775 podStartE2EDuration="12.135404233s" totalImagesPullingTime="3.306957458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:05:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:05:44.764220177 +0000 UTC m=+274.654891513" watchObservedRunningTime="2026-07-29 14:05:51.135404233 +0000 UTC m=+281.026075567"
Jul 29 14:05:51.136492 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:51.136471    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-mfxt6"]
Jul 29 14:05:51.139609 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:51.139591    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-mfxt6"
Jul 29 14:05:51.142315 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:51.142295    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 29 14:05:51.143533 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:51.143517    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 29 14:05:51.143629 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:51.143523    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-ftzlq\""
Jul 29 14:05:51.150317 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:51.150297    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-mfxt6"]
Jul 29 14:05:51.303091 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:51.303039    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/beeb67d7-d29e-49ec-96a2-623b407243f3-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-mfxt6\" (UID: \"beeb67d7-d29e-49ec-96a2-623b407243f3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mfxt6"
Jul 29 14:05:51.303274 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:51.303133    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqtvf\" (UniqueName: \"kubernetes.io/projected/beeb67d7-d29e-49ec-96a2-623b407243f3-kube-api-access-mqtvf\") pod \"cert-manager-webhook-6486ffd997-mfxt6\" (UID: \"beeb67d7-d29e-49ec-96a2-623b407243f3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mfxt6"
Jul 29 14:05:51.404038 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:51.403950    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/beeb67d7-d29e-49ec-96a2-623b407243f3-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-mfxt6\" (UID: \"beeb67d7-d29e-49ec-96a2-623b407243f3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mfxt6"
Jul 29 14:05:51.404038 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:51.404012    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqtvf\" (UniqueName: \"kubernetes.io/projected/beeb67d7-d29e-49ec-96a2-623b407243f3-kube-api-access-mqtvf\") pod \"cert-manager-webhook-6486ffd997-mfxt6\" (UID: \"beeb67d7-d29e-49ec-96a2-623b407243f3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mfxt6"
Jul 29 14:05:51.412206 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:51.412176    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/beeb67d7-d29e-49ec-96a2-623b407243f3-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-mfxt6\" (UID: \"beeb67d7-d29e-49ec-96a2-623b407243f3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mfxt6"
Jul 29 14:05:51.412317 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:51.412237    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqtvf\" (UniqueName: \"kubernetes.io/projected/beeb67d7-d29e-49ec-96a2-623b407243f3-kube-api-access-mqtvf\") pod \"cert-manager-webhook-6486ffd997-mfxt6\" (UID: \"beeb67d7-d29e-49ec-96a2-623b407243f3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mfxt6"
Jul 29 14:05:51.447764 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:51.447739    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-mfxt6"
Jul 29 14:05:51.568831 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:51.568739    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-mfxt6"]
Jul 29 14:05:51.571397 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:05:51.571360    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbeeb67d7_d29e_49ec_96a2_623b407243f3.slice/crio-5445979469c9a059e270c76d830cc9976f810e4d63dc53e64fe3f6ec459bbedf WatchSource:0}: Error finding container 5445979469c9a059e270c76d830cc9976f810e4d63dc53e64fe3f6ec459bbedf: Status 404 returned error can't find the container with id 5445979469c9a059e270c76d830cc9976f810e4d63dc53e64fe3f6ec459bbedf
Jul 29 14:05:51.746565 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:51.746526    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-mfxt6" event={"ID":"beeb67d7-d29e-49ec-96a2-623b407243f3","Type":"ContainerStarted","Data":"5445979469c9a059e270c76d830cc9976f810e4d63dc53e64fe3f6ec459bbedf"}
Jul 29 14:05:53.782818 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:53.782761    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:05:53.987184 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:53.987147    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:05:54.758931 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:54.758897    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-mfxt6" event={"ID":"beeb67d7-d29e-49ec-96a2-623b407243f3","Type":"ContainerStarted","Data":"6e0931ee38d45046598a470c1b13bfddc32ed99b990b7fa600eaf10256fb16b4"}
Jul 29 14:05:54.759132 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:54.758984    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-mfxt6"
Jul 29 14:05:54.791474 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:54.791425    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-mfxt6" podStartSLOduration=0.787491824 podStartE2EDuration="3.791411174s" totalImagesPullingTime="3.00391935s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:05:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:05:54.790678338 +0000 UTC m=+284.681349676" watchObservedRunningTime="2026-07-29 14:05:54.791411174 +0000 UTC m=+284.682082508"
Jul 29 14:05:55.027069 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:55.027023    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-6jrm4"]
Jul 29 14:05:55.030128 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:55.030110    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-6jrm4"
Jul 29 14:05:55.032892 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:55.032867    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-dt8tk\""
Jul 29 14:05:55.042817 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:55.042794    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-6jrm4"]
Jul 29 14:05:55.133402 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:55.133370    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dps4m\" (UniqueName: \"kubernetes.io/projected/07840239-bc96-460a-ae50-0193c893d184-kube-api-access-dps4m\") pod \"cert-manager-55b4d954d5-6jrm4\" (UID: \"07840239-bc96-460a-ae50-0193c893d184\") " pod="cert-manager/cert-manager-55b4d954d5-6jrm4"
Jul 29 14:05:55.133578 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:55.133421    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/07840239-bc96-460a-ae50-0193c893d184-bound-sa-token\") pod \"cert-manager-55b4d954d5-6jrm4\" (UID: \"07840239-bc96-460a-ae50-0193c893d184\") " pod="cert-manager/cert-manager-55b4d954d5-6jrm4"
Jul 29 14:05:55.234850 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:55.234815    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/07840239-bc96-460a-ae50-0193c893d184-bound-sa-token\") pod \"cert-manager-55b4d954d5-6jrm4\" (UID: \"07840239-bc96-460a-ae50-0193c893d184\") " pod="cert-manager/cert-manager-55b4d954d5-6jrm4"
Jul 29 14:05:55.235011 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:55.234888    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dps4m\" (UniqueName: \"kubernetes.io/projected/07840239-bc96-460a-ae50-0193c893d184-kube-api-access-dps4m\") pod \"cert-manager-55b4d954d5-6jrm4\" (UID: \"07840239-bc96-460a-ae50-0193c893d184\") " pod="cert-manager/cert-manager-55b4d954d5-6jrm4"
Jul 29 14:05:55.245017 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:55.244990    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/07840239-bc96-460a-ae50-0193c893d184-bound-sa-token\") pod \"cert-manager-55b4d954d5-6jrm4\" (UID: \"07840239-bc96-460a-ae50-0193c893d184\") " pod="cert-manager/cert-manager-55b4d954d5-6jrm4"
Jul 29 14:05:55.245174 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:55.245153    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dps4m\" (UniqueName: \"kubernetes.io/projected/07840239-bc96-460a-ae50-0193c893d184-kube-api-access-dps4m\") pod \"cert-manager-55b4d954d5-6jrm4\" (UID: \"07840239-bc96-460a-ae50-0193c893d184\") " pod="cert-manager/cert-manager-55b4d954d5-6jrm4"
Jul 29 14:05:55.338336 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:55.338256    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-6jrm4"
Jul 29 14:05:55.464037 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:55.464014    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-6jrm4"]
Jul 29 14:05:55.466249 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:05:55.466227    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07840239_bc96_460a_ae50_0193c893d184.slice/crio-b35cf94a7e186b0226405949ec925f2f3c52dedd983f2c52c742b15ac6ff7fe4 WatchSource:0}: Error finding container b35cf94a7e186b0226405949ec925f2f3c52dedd983f2c52c742b15ac6ff7fe4: Status 404 returned error can't find the container with id b35cf94a7e186b0226405949ec925f2f3c52dedd983f2c52c742b15ac6ff7fe4
Jul 29 14:05:55.763612 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:55.763579    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-6jrm4" event={"ID":"07840239-bc96-460a-ae50-0193c893d184","Type":"ContainerStarted","Data":"9d707e4fb28f581548f5c33560237b88f0b5a61400e1aead37b799f89b931427"}
Jul 29 14:05:55.763766 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:55.763618    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-6jrm4" event={"ID":"07840239-bc96-460a-ae50-0193c893d184","Type":"ContainerStarted","Data":"b35cf94a7e186b0226405949ec925f2f3c52dedd983f2c52c742b15ac6ff7fe4"}
Jul 29 14:05:55.782137 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:55.782088    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-6jrm4" podStartSLOduration=0.782052588 podStartE2EDuration="782.052588ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:05:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:05:55.781205995 +0000 UTC m=+285.671877342" watchObservedRunningTime="2026-07-29 14:05:55.782052588 +0000 UTC m=+285.672723924"
Jul 29 14:05:57.071811 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:57.071776    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:05:57.190930 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:05:57.190892    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:06:00.766441 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:00.766413    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-mfxt6"
Jul 29 14:06:02.493309 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:02.493268    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:06:02.595464 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:02.595422    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:06:04.597377 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:04.597333    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:06:10.539387 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:10.539356    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdw8l_b93f6faa-c8bf-43e9-94e0-b3dcd97b7296/ovn-acl-logging/0.log"
Jul 29 14:06:10.544239 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:10.544208    2559 kubelet.go:1648] "Image garbage collection succeeded"
Jul 29 14:06:10.545764 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:10.545747    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdw8l_b93f6faa-c8bf-43e9-94e0-b3dcd97b7296/ovn-acl-logging/0.log"
Jul 29 14:06:20.608188 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:20.608130    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:06:21.577458 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:21.577421    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49"]
Jul 29 14:06:21.580380 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:21.580365    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49"
Jul 29 14:06:21.585147 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:21.585113    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 29 14:06:21.585147 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:21.585143    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 29 14:06:21.585338 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:21.585260    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 29 14:06:21.585338 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:21.585260    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-2g49j\""
Jul 29 14:06:21.585441 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:21.585386    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 29 14:06:21.595294 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:21.595268    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49"]
Jul 29 14:06:21.619856 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:21.619832    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9b5129c9-2f62-466a-87a5-6076b6c6fa30-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49\" (UID: \"9b5129c9-2f62-466a-87a5-6076b6c6fa30\") " pod="opendatahub/opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49"
Jul 29 14:06:21.620235 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:21.619861    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9b5129c9-2f62-466a-87a5-6076b6c6fa30-manifests\") pod \"opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49\" (UID: \"9b5129c9-2f62-466a-87a5-6076b6c6fa30\") " pod="opendatahub/opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49"
Jul 29 14:06:21.620235 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:21.619893    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9b5129c9-2f62-466a-87a5-6076b6c6fa30-webhook-cert\") pod \"opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49\" (UID: \"9b5129c9-2f62-466a-87a5-6076b6c6fa30\") " pod="opendatahub/opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49"
Jul 29 14:06:21.620235 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:21.619912    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9b5129c9-2f62-466a-87a5-6076b6c6fa30-tmp\") pod \"opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49\" (UID: \"9b5129c9-2f62-466a-87a5-6076b6c6fa30\") " pod="opendatahub/opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49"
Jul 29 14:06:21.620235 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:21.620014    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9x7k\" (UniqueName: \"kubernetes.io/projected/9b5129c9-2f62-466a-87a5-6076b6c6fa30-kube-api-access-n9x7k\") pod \"opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49\" (UID: \"9b5129c9-2f62-466a-87a5-6076b6c6fa30\") " pod="opendatahub/opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49"
Jul 29 14:06:21.720664 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:21.720631    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9b5129c9-2f62-466a-87a5-6076b6c6fa30-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49\" (UID: \"9b5129c9-2f62-466a-87a5-6076b6c6fa30\") " pod="opendatahub/opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49"
Jul 29 14:06:21.720664 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:21.720667    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9b5129c9-2f62-466a-87a5-6076b6c6fa30-manifests\") pod \"opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49\" (UID: \"9b5129c9-2f62-466a-87a5-6076b6c6fa30\") " pod="opendatahub/opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49"
Jul 29 14:06:21.720847 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:21.720709    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9b5129c9-2f62-466a-87a5-6076b6c6fa30-webhook-cert\") pod \"opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49\" (UID: \"9b5129c9-2f62-466a-87a5-6076b6c6fa30\") " pod="opendatahub/opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49"
Jul 29 14:06:21.720847 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:21.720733    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9b5129c9-2f62-466a-87a5-6076b6c6fa30-tmp\") pod \"opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49\" (UID: \"9b5129c9-2f62-466a-87a5-6076b6c6fa30\") " pod="opendatahub/opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49"
Jul 29 14:06:21.720847 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:21.720775    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9x7k\" (UniqueName: \"kubernetes.io/projected/9b5129c9-2f62-466a-87a5-6076b6c6fa30-kube-api-access-n9x7k\") pod \"opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49\" (UID: \"9b5129c9-2f62-466a-87a5-6076b6c6fa30\") " pod="opendatahub/opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49"
Jul 29 14:06:21.721291 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:21.721266    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9b5129c9-2f62-466a-87a5-6076b6c6fa30-manifests\") pod \"opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49\" (UID: \"9b5129c9-2f62-466a-87a5-6076b6c6fa30\") " pod="opendatahub/opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49"
Jul 29 14:06:21.723128 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:21.723098    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9b5129c9-2f62-466a-87a5-6076b6c6fa30-tmp\") pod \"opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49\" (UID: \"9b5129c9-2f62-466a-87a5-6076b6c6fa30\") " pod="opendatahub/opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49"
Jul 29 14:06:21.723307 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:21.723288    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9b5129c9-2f62-466a-87a5-6076b6c6fa30-webhook-cert\") pod \"opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49\" (UID: \"9b5129c9-2f62-466a-87a5-6076b6c6fa30\") " pod="opendatahub/opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49"
Jul 29 14:06:21.723390 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:21.723288    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9b5129c9-2f62-466a-87a5-6076b6c6fa30-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49\" (UID: \"9b5129c9-2f62-466a-87a5-6076b6c6fa30\") " pod="opendatahub/opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49"
Jul 29 14:06:21.729672 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:21.729648    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9x7k\" (UniqueName: \"kubernetes.io/projected/9b5129c9-2f62-466a-87a5-6076b6c6fa30-kube-api-access-n9x7k\") pod \"opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49\" (UID: \"9b5129c9-2f62-466a-87a5-6076b6c6fa30\") " pod="opendatahub/opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49"
Jul 29 14:06:21.890363 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:21.890276    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49"
Jul 29 14:06:22.024407 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:22.024374    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49"]
Jul 29 14:06:22.027509 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:06:22.027481    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b5129c9_2f62_466a_87a5_6076b6c6fa30.slice/crio-2234c65558179611707d8298d92d1e44fbb89ff5f0e9f5e4f7d158992e4d8927 WatchSource:0}: Error finding container 2234c65558179611707d8298d92d1e44fbb89ff5f0e9f5e4f7d158992e4d8927: Status 404 returned error can't find the container with id 2234c65558179611707d8298d92d1e44fbb89ff5f0e9f5e4f7d158992e4d8927
Jul 29 14:06:22.029211 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:22.029192    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 14:06:22.876791 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:22.876746    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49" event={"ID":"9b5129c9-2f62-466a-87a5-6076b6c6fa30","Type":"ContainerStarted","Data":"2234c65558179611707d8298d92d1e44fbb89ff5f0e9f5e4f7d158992e4d8927"}
Jul 29 14:06:23.382527 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:23.382490    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:06:25.890828 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:25.890791    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49" event={"ID":"9b5129c9-2f62-466a-87a5-6076b6c6fa30","Type":"ContainerStarted","Data":"48a9919e9e9ca0a8330b3534247d774d68ccfb956b88ba638d34680a32150192"}
Jul 29 14:06:26.896672 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:26.896635    2559 generic.go:356] "Generic (PLEG): container finished" podID="9b5129c9-2f62-466a-87a5-6076b6c6fa30" containerID="48a9919e9e9ca0a8330b3534247d774d68ccfb956b88ba638d34680a32150192" exitCode=0
Jul 29 14:06:26.897094 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:26.896731    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49" event={"ID":"9b5129c9-2f62-466a-87a5-6076b6c6fa30","Type":"ContainerDied","Data":"48a9919e9e9ca0a8330b3534247d774d68ccfb956b88ba638d34680a32150192"}
Jul 29 14:06:27.902815 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:27.902780    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49" event={"ID":"9b5129c9-2f62-466a-87a5-6076b6c6fa30","Type":"ContainerStarted","Data":"3ad735320f664100c57049da1fbae74a877ac18156a497e44f2b8c10d25ae6bf"}
Jul 29 14:06:27.903217 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:27.902903    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49"
Jul 29 14:06:27.928306 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:27.928258    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49" podStartSLOduration=2.511588447 podStartE2EDuration="6.928243595s" totalImagesPullingTime="3.374231667s" totalInitContainerRuntime="1.042423481s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:06:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:06:27.925655964 +0000 UTC m=+317.816327299" watchObservedRunningTime="2026-07-29 14:06:27.928243595 +0000 UTC m=+317.818914930"
Jul 29 14:06:28.874358 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:28.874322    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:06:29.025487 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:29.025449    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:06:33.153078 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:33.153024    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:06:33.268714 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:33.268684    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:06:38.908844 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:38.908811    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49"
Jul 29 14:06:41.683874 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:41.683846    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6b4bcbd496-4w54r"]
Jul 29 14:06:41.687180 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:41.687154    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6b4bcbd496-4w54r"
Jul 29 14:06:41.691258 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:41.691233    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 29 14:06:41.692998 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:41.692973    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 29 14:06:41.699882 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:41.699860    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-mps5c\""
Jul 29 14:06:41.708589 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:41.708573    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 29 14:06:41.710330 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:41.710317    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 29 14:06:41.716362 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:41.716339    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6b4bcbd496-4w54r"]
Jul 29 14:06:41.769327 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:41.769296    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4nzn\" (UniqueName: \"kubernetes.io/projected/d28ff993-6a2b-42e8-bced-e9c5d34e0ff8-kube-api-access-n4nzn\") pod \"kube-auth-proxy-6b4bcbd496-4w54r\" (UID: \"d28ff993-6a2b-42e8-bced-e9c5d34e0ff8\") " pod="openshift-ingress/kube-auth-proxy-6b4bcbd496-4w54r"
Jul 29 14:06:41.769327 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:41.769331    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d28ff993-6a2b-42e8-bced-e9c5d34e0ff8-tmp\") pod \"kube-auth-proxy-6b4bcbd496-4w54r\" (UID: \"d28ff993-6a2b-42e8-bced-e9c5d34e0ff8\") " pod="openshift-ingress/kube-auth-proxy-6b4bcbd496-4w54r"
Jul 29 14:06:41.769503 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:41.769352    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d28ff993-6a2b-42e8-bced-e9c5d34e0ff8-tls-certs\") pod \"kube-auth-proxy-6b4bcbd496-4w54r\" (UID: \"d28ff993-6a2b-42e8-bced-e9c5d34e0ff8\") " pod="openshift-ingress/kube-auth-proxy-6b4bcbd496-4w54r"
Jul 29 14:06:41.825216 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:41.825188    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:06:41.870172 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:41.870141    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d28ff993-6a2b-42e8-bced-e9c5d34e0ff8-tls-certs\") pod \"kube-auth-proxy-6b4bcbd496-4w54r\" (UID: \"d28ff993-6a2b-42e8-bced-e9c5d34e0ff8\") " pod="openshift-ingress/kube-auth-proxy-6b4bcbd496-4w54r"
Jul 29 14:06:41.870291 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:41.870211    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4nzn\" (UniqueName: \"kubernetes.io/projected/d28ff993-6a2b-42e8-bced-e9c5d34e0ff8-kube-api-access-n4nzn\") pod \"kube-auth-proxy-6b4bcbd496-4w54r\" (UID: \"d28ff993-6a2b-42e8-bced-e9c5d34e0ff8\") " pod="openshift-ingress/kube-auth-proxy-6b4bcbd496-4w54r"
Jul 29 14:06:41.870291 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:41.870246    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d28ff993-6a2b-42e8-bced-e9c5d34e0ff8-tmp\") pod \"kube-auth-proxy-6b4bcbd496-4w54r\" (UID: \"d28ff993-6a2b-42e8-bced-e9c5d34e0ff8\") " pod="openshift-ingress/kube-auth-proxy-6b4bcbd496-4w54r"
Jul 29 14:06:41.872404 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:41.872379    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d28ff993-6a2b-42e8-bced-e9c5d34e0ff8-tmp\") pod \"kube-auth-proxy-6b4bcbd496-4w54r\" (UID: \"d28ff993-6a2b-42e8-bced-e9c5d34e0ff8\") " pod="openshift-ingress/kube-auth-proxy-6b4bcbd496-4w54r"
Jul 29 14:06:41.872549 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:41.872533    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d28ff993-6a2b-42e8-bced-e9c5d34e0ff8-tls-certs\") pod \"kube-auth-proxy-6b4bcbd496-4w54r\" (UID: \"d28ff993-6a2b-42e8-bced-e9c5d34e0ff8\") " pod="openshift-ingress/kube-auth-proxy-6b4bcbd496-4w54r"
Jul 29 14:06:41.887485 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:41.887457    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4nzn\" (UniqueName: \"kubernetes.io/projected/d28ff993-6a2b-42e8-bced-e9c5d34e0ff8-kube-api-access-n4nzn\") pod \"kube-auth-proxy-6b4bcbd496-4w54r\" (UID: \"d28ff993-6a2b-42e8-bced-e9c5d34e0ff8\") " pod="openshift-ingress/kube-auth-proxy-6b4bcbd496-4w54r"
Jul 29 14:06:41.973074 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:41.972979    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:06:41.996879 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:41.996854    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6b4bcbd496-4w54r"
Jul 29 14:06:42.134152 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:42.134100    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:06:42.134551 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:42.134489    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6b4bcbd496-4w54r"]
Jul 29 14:06:42.136712 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:06:42.136684    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd28ff993_6a2b_42e8_bced_e9c5d34e0ff8.slice/crio-88223f20aa4cd67b1863413858e9b37b220f66f5a49a6c01a6f90abf680a8b71 WatchSource:0}: Error finding container 88223f20aa4cd67b1863413858e9b37b220f66f5a49a6c01a6f90abf680a8b71: Status 404 returned error can't find the container with id 88223f20aa4cd67b1863413858e9b37b220f66f5a49a6c01a6f90abf680a8b71
Jul 29 14:06:42.223121 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:42.223033    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:06:42.340234 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:42.340198    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:06:42.848643 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:42.848587    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:06:42.965195 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:42.965157    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6b4bcbd496-4w54r" event={"ID":"d28ff993-6a2b-42e8-bced-e9c5d34e0ff8","Type":"ContainerStarted","Data":"88223f20aa4cd67b1863413858e9b37b220f66f5a49a6c01a6f90abf680a8b71"}
Jul 29 14:06:43.084270 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:43.084221    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:06:43.088935 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:43.088900    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-pqrns"]
Jul 29 14:06:43.092439 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:43.092415    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-pqrns"
Jul 29 14:06:43.097105 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:43.097083    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-z68zc\""
Jul 29 14:06:43.097407 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:43.097384    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 29 14:06:43.104451 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:43.104426    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-pqrns"]
Jul 29 14:06:43.179371 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:43.179327    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:06:43.182477 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:43.182445    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gq6mg\" (UniqueName: \"kubernetes.io/projected/1ff31e21-bbe1-41c8-801f-e02d452b7718-kube-api-access-gq6mg\") pod \"model-serving-api-54d89bd5fc-pqrns\" (UID: \"1ff31e21-bbe1-41c8-801f-e02d452b7718\") " pod="opendatahub/model-serving-api-54d89bd5fc-pqrns"
Jul 29 14:06:43.182615 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:43.182503    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ff31e21-bbe1-41c8-801f-e02d452b7718-tls-certs\") pod \"model-serving-api-54d89bd5fc-pqrns\" (UID: \"1ff31e21-bbe1-41c8-801f-e02d452b7718\") " pod="opendatahub/model-serving-api-54d89bd5fc-pqrns"
Jul 29 14:06:43.283755 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:43.283719    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ff31e21-bbe1-41c8-801f-e02d452b7718-tls-certs\") pod \"model-serving-api-54d89bd5fc-pqrns\" (UID: \"1ff31e21-bbe1-41c8-801f-e02d452b7718\") " pod="opendatahub/model-serving-api-54d89bd5fc-pqrns"
Jul 29 14:06:43.283922 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:43.283790    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gq6mg\" (UniqueName: \"kubernetes.io/projected/1ff31e21-bbe1-41c8-801f-e02d452b7718-kube-api-access-gq6mg\") pod \"model-serving-api-54d89bd5fc-pqrns\" (UID: \"1ff31e21-bbe1-41c8-801f-e02d452b7718\") " pod="opendatahub/model-serving-api-54d89bd5fc-pqrns"
Jul 29 14:06:43.283922 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:06:43.283883    2559 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 29 14:06:43.283995 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:06:43.283951    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ff31e21-bbe1-41c8-801f-e02d452b7718-tls-certs podName:1ff31e21-bbe1-41c8-801f-e02d452b7718 nodeName:}" failed. No retries permitted until 2026-07-29 14:06:43.783935019 +0000 UTC m=+333.674606333 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/1ff31e21-bbe1-41c8-801f-e02d452b7718-tls-certs") pod "model-serving-api-54d89bd5fc-pqrns" (UID: "1ff31e21-bbe1-41c8-801f-e02d452b7718") : secret "model-serving-api-tls" not found
Jul 29 14:06:43.302863 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:43.302836    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gq6mg\" (UniqueName: \"kubernetes.io/projected/1ff31e21-bbe1-41c8-801f-e02d452b7718-kube-api-access-gq6mg\") pod \"model-serving-api-54d89bd5fc-pqrns\" (UID: \"1ff31e21-bbe1-41c8-801f-e02d452b7718\") " pod="opendatahub/model-serving-api-54d89bd5fc-pqrns"
Jul 29 14:06:43.346354 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:43.346311    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:06:43.486330 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:43.486245    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:06:43.788297 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:43.788262    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ff31e21-bbe1-41c8-801f-e02d452b7718-tls-certs\") pod \"model-serving-api-54d89bd5fc-pqrns\" (UID: \"1ff31e21-bbe1-41c8-801f-e02d452b7718\") " pod="opendatahub/model-serving-api-54d89bd5fc-pqrns"
Jul 29 14:06:43.791006 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:43.790985    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ff31e21-bbe1-41c8-801f-e02d452b7718-tls-certs\") pod \"model-serving-api-54d89bd5fc-pqrns\" (UID: \"1ff31e21-bbe1-41c8-801f-e02d452b7718\") " pod="opendatahub/model-serving-api-54d89bd5fc-pqrns"
Jul 29 14:06:44.006783 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:44.006739    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-pqrns"
Jul 29 14:06:44.147030 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:44.146999    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-pqrns"]
Jul 29 14:06:44.149483 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:06:44.149443    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ff31e21_bbe1_41c8_801f_e02d452b7718.slice/crio-2944d36845f2c65f665e4446c1abf4263d249032b143804c53bfa6f24045d9b0 WatchSource:0}: Error finding container 2944d36845f2c65f665e4446c1abf4263d249032b143804c53bfa6f24045d9b0: Status 404 returned error can't find the container with id 2944d36845f2c65f665e4446c1abf4263d249032b143804c53bfa6f24045d9b0
Jul 29 14:06:44.285835 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:44.285799    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:06:44.974387 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:44.974353    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6b4bcbd496-4w54r" event={"ID":"d28ff993-6a2b-42e8-bced-e9c5d34e0ff8","Type":"ContainerStarted","Data":"b34c59a1bc1c5a89ef6ddbb071d1f615ef5a54ebeb500698d0b1577a6f37e561"}
Jul 29 14:06:44.975468 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:44.975443    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-pqrns" event={"ID":"1ff31e21-bbe1-41c8-801f-e02d452b7718","Type":"ContainerStarted","Data":"2944d36845f2c65f665e4446c1abf4263d249032b143804c53bfa6f24045d9b0"}
Jul 29 14:06:44.992882 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:44.992221    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6b4bcbd496-4w54r" podStartSLOduration=1.295789585 podStartE2EDuration="3.992202242s" totalImagesPullingTime="2.696412657s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:06:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:06:44.990552782 +0000 UTC m=+334.881224116" watchObservedRunningTime="2026-07-29 14:06:44.992202242 +0000 UTC m=+334.882873577"
Jul 29 14:06:46.987949 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:46.987906    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-pqrns" event={"ID":"1ff31e21-bbe1-41c8-801f-e02d452b7718","Type":"ContainerStarted","Data":"fa770a89d131c908d0c0485419d3348d3da574dd143b38398f4f1948ecf33b59"}
Jul 29 14:06:46.988386 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:46.988089    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-pqrns"
Jul 29 14:06:47.004831 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:47.004783    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-pqrns" podStartSLOduration=2.162486422 podStartE2EDuration="4.004770816s" totalImagesPullingTime="1.842284394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:06:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:06:47.00361489 +0000 UTC m=+336.894286224" watchObservedRunningTime="2026-07-29 14:06:47.004770816 +0000 UTC m=+336.895442150"
Jul 29 14:06:49.010398 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.010350    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:06:49.088873 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.088840    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:06:49.699991 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.699959    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"]
Jul 29 14:06:49.703278 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.703260    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"
Jul 29 14:06:49.707414 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.707385    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-dscgr\""
Jul 29 14:06:49.707519 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.707441    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 29 14:06:49.742914 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.742881    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"]
Jul 29 14:06:49.769654 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.769628    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:06:49.840109 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.840075    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27d584d0-3cea-4000-910c-5d089ec8360e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl\" (UID: \"27d584d0-3cea-4000-910c-5d089ec8360e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"
Jul 29 14:06:49.840251 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.840133    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27d584d0-3cea-4000-910c-5d089ec8360e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl\" (UID: \"27d584d0-3cea-4000-910c-5d089ec8360e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"
Jul 29 14:06:49.840251 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.840153    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27d584d0-3cea-4000-910c-5d089ec8360e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl\" (UID: \"27d584d0-3cea-4000-910c-5d089ec8360e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"
Jul 29 14:06:49.840251 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.840171    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27d584d0-3cea-4000-910c-5d089ec8360e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl\" (UID: \"27d584d0-3cea-4000-910c-5d089ec8360e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"
Jul 29 14:06:49.840251 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.840197    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27d584d0-3cea-4000-910c-5d089ec8360e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl\" (UID: \"27d584d0-3cea-4000-910c-5d089ec8360e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"
Jul 29 14:06:49.840251 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.840247    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27d584d0-3cea-4000-910c-5d089ec8360e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl\" (UID: \"27d584d0-3cea-4000-910c-5d089ec8360e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"
Jul 29 14:06:49.840410 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.840264    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27d584d0-3cea-4000-910c-5d089ec8360e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl\" (UID: \"27d584d0-3cea-4000-910c-5d089ec8360e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"
Jul 29 14:06:49.840410 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.840314    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27d584d0-3cea-4000-910c-5d089ec8360e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl\" (UID: \"27d584d0-3cea-4000-910c-5d089ec8360e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"
Jul 29 14:06:49.840410 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.840342    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dv7j5\" (UniqueName: \"kubernetes.io/projected/27d584d0-3cea-4000-910c-5d089ec8360e-kube-api-access-dv7j5\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl\" (UID: \"27d584d0-3cea-4000-910c-5d089ec8360e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"
Jul 29 14:06:49.905965 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.905933    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:06:49.940930 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.940902    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27d584d0-3cea-4000-910c-5d089ec8360e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl\" (UID: \"27d584d0-3cea-4000-910c-5d089ec8360e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"
Jul 29 14:06:49.941049 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.940935    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27d584d0-3cea-4000-910c-5d089ec8360e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl\" (UID: \"27d584d0-3cea-4000-910c-5d089ec8360e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"
Jul 29 14:06:49.941049 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.940954    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27d584d0-3cea-4000-910c-5d089ec8360e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl\" (UID: \"27d584d0-3cea-4000-910c-5d089ec8360e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"
Jul 29 14:06:49.941049 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.940979    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27d584d0-3cea-4000-910c-5d089ec8360e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl\" (UID: \"27d584d0-3cea-4000-910c-5d089ec8360e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"
Jul 29 14:06:49.941049 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.940997    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dv7j5\" (UniqueName: \"kubernetes.io/projected/27d584d0-3cea-4000-910c-5d089ec8360e-kube-api-access-dv7j5\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl\" (UID: \"27d584d0-3cea-4000-910c-5d089ec8360e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"
Jul 29 14:06:49.941049 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.941041    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27d584d0-3cea-4000-910c-5d089ec8360e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl\" (UID: \"27d584d0-3cea-4000-910c-5d089ec8360e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"
Jul 29 14:06:49.941338 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.941101    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27d584d0-3cea-4000-910c-5d089ec8360e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl\" (UID: \"27d584d0-3cea-4000-910c-5d089ec8360e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"
Jul 29 14:06:49.941338 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.941128    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27d584d0-3cea-4000-910c-5d089ec8360e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl\" (UID: \"27d584d0-3cea-4000-910c-5d089ec8360e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"
Jul 29 14:06:49.941338 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.941156    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27d584d0-3cea-4000-910c-5d089ec8360e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl\" (UID: \"27d584d0-3cea-4000-910c-5d089ec8360e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"
Jul 29 14:06:49.941488 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.941399    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27d584d0-3cea-4000-910c-5d089ec8360e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl\" (UID: \"27d584d0-3cea-4000-910c-5d089ec8360e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"
Jul 29 14:06:49.941488 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.941435    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27d584d0-3cea-4000-910c-5d089ec8360e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl\" (UID: \"27d584d0-3cea-4000-910c-5d089ec8360e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"
Jul 29 14:06:49.941562 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.941513    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27d584d0-3cea-4000-910c-5d089ec8360e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl\" (UID: \"27d584d0-3cea-4000-910c-5d089ec8360e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"
Jul 29 14:06:49.941623 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.941599    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27d584d0-3cea-4000-910c-5d089ec8360e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl\" (UID: \"27d584d0-3cea-4000-910c-5d089ec8360e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"
Jul 29 14:06:49.941796 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.941776    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27d584d0-3cea-4000-910c-5d089ec8360e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl\" (UID: \"27d584d0-3cea-4000-910c-5d089ec8360e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"
Jul 29 14:06:49.943378 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.943344    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27d584d0-3cea-4000-910c-5d089ec8360e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl\" (UID: \"27d584d0-3cea-4000-910c-5d089ec8360e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"
Jul 29 14:06:49.943607 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.943585    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27d584d0-3cea-4000-910c-5d089ec8360e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl\" (UID: \"27d584d0-3cea-4000-910c-5d089ec8360e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"
Jul 29 14:06:49.954793 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.954733    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dv7j5\" (UniqueName: \"kubernetes.io/projected/27d584d0-3cea-4000-910c-5d089ec8360e-kube-api-access-dv7j5\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl\" (UID: \"27d584d0-3cea-4000-910c-5d089ec8360e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"
Jul 29 14:06:49.955618 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:49.955588    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27d584d0-3cea-4000-910c-5d089ec8360e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl\" (UID: \"27d584d0-3cea-4000-910c-5d089ec8360e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"
Jul 29 14:06:50.013048 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:50.013022    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"
Jul 29 14:06:50.085203 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:50.085159    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:06:50.157856 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:50.157694    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"]
Jul 29 14:06:50.160269 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:06:50.160246    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27d584d0_3cea_4000_910c_5d089ec8360e.slice/crio-f41a66836e8301b0f7db3af87798564119eb0ab775d0e3cb904335da839ae12f WatchSource:0}: Error finding container f41a66836e8301b0f7db3af87798564119eb0ab775d0e3cb904335da839ae12f: Status 404 returned error can't find the container with id f41a66836e8301b0f7db3af87798564119eb0ab775d0e3cb904335da839ae12f
Jul 29 14:06:51.007214 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:51.007177    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl" event={"ID":"27d584d0-3cea-4000-910c-5d089ec8360e","Type":"ContainerStarted","Data":"f41a66836e8301b0f7db3af87798564119eb0ab775d0e3cb904335da839ae12f"}
Jul 29 14:06:51.750938 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:51.750901    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-f9f5fc96c-qvnsk"]
Jul 29 14:06:51.755254 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:51.755230    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-f9f5fc96c-qvnsk"
Jul 29 14:06:51.758212 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:51.758190    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 29 14:06:51.758342 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:51.758228    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 29 14:06:51.759661 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:51.759640    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 29 14:06:51.759762 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:51.759728    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 29 14:06:51.759801 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:51.759779    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-kpql8\""
Jul 29 14:06:51.759847 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:51.759833    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 29 14:06:51.763921 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:51.763898    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-f9f5fc96c-qvnsk"]
Jul 29 14:06:51.856562 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:51.854956    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6a5c29f8-366e-4e4e-ad77-e406d4213703-metrics-cert\") pod \"lws-controller-manager-f9f5fc96c-qvnsk\" (UID: \"6a5c29f8-366e-4e4e-ad77-e406d4213703\") " pod="openshift-lws-operator/lws-controller-manager-f9f5fc96c-qvnsk"
Jul 29 14:06:51.856562 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:51.855011    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6a5c29f8-366e-4e4e-ad77-e406d4213703-cert\") pod \"lws-controller-manager-f9f5fc96c-qvnsk\" (UID: \"6a5c29f8-366e-4e4e-ad77-e406d4213703\") " pod="openshift-lws-operator/lws-controller-manager-f9f5fc96c-qvnsk"
Jul 29 14:06:51.856562 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:51.855172    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6a5c29f8-366e-4e4e-ad77-e406d4213703-manager-config\") pod \"lws-controller-manager-f9f5fc96c-qvnsk\" (UID: \"6a5c29f8-366e-4e4e-ad77-e406d4213703\") " pod="openshift-lws-operator/lws-controller-manager-f9f5fc96c-qvnsk"
Jul 29 14:06:51.856562 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:51.855248    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6bjp\" (UniqueName: \"kubernetes.io/projected/6a5c29f8-366e-4e4e-ad77-e406d4213703-kube-api-access-w6bjp\") pod \"lws-controller-manager-f9f5fc96c-qvnsk\" (UID: \"6a5c29f8-366e-4e4e-ad77-e406d4213703\") " pod="openshift-lws-operator/lws-controller-manager-f9f5fc96c-qvnsk"
Jul 29 14:06:51.956102 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:51.956054    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6a5c29f8-366e-4e4e-ad77-e406d4213703-manager-config\") pod \"lws-controller-manager-f9f5fc96c-qvnsk\" (UID: \"6a5c29f8-366e-4e4e-ad77-e406d4213703\") " pod="openshift-lws-operator/lws-controller-manager-f9f5fc96c-qvnsk"
Jul 29 14:06:51.956260 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:51.956120    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6bjp\" (UniqueName: \"kubernetes.io/projected/6a5c29f8-366e-4e4e-ad77-e406d4213703-kube-api-access-w6bjp\") pod \"lws-controller-manager-f9f5fc96c-qvnsk\" (UID: \"6a5c29f8-366e-4e4e-ad77-e406d4213703\") " pod="openshift-lws-operator/lws-controller-manager-f9f5fc96c-qvnsk"
Jul 29 14:06:51.956260 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:51.956160    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6a5c29f8-366e-4e4e-ad77-e406d4213703-metrics-cert\") pod \"lws-controller-manager-f9f5fc96c-qvnsk\" (UID: \"6a5c29f8-366e-4e4e-ad77-e406d4213703\") " pod="openshift-lws-operator/lws-controller-manager-f9f5fc96c-qvnsk"
Jul 29 14:06:51.956260 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:51.956176    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6a5c29f8-366e-4e4e-ad77-e406d4213703-cert\") pod \"lws-controller-manager-f9f5fc96c-qvnsk\" (UID: \"6a5c29f8-366e-4e4e-ad77-e406d4213703\") " pod="openshift-lws-operator/lws-controller-manager-f9f5fc96c-qvnsk"
Jul 29 14:06:51.956643 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:51.956621    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6a5c29f8-366e-4e4e-ad77-e406d4213703-manager-config\") pod \"lws-controller-manager-f9f5fc96c-qvnsk\" (UID: \"6a5c29f8-366e-4e4e-ad77-e406d4213703\") " pod="openshift-lws-operator/lws-controller-manager-f9f5fc96c-qvnsk"
Jul 29 14:06:51.958496 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:51.958474    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6a5c29f8-366e-4e4e-ad77-e406d4213703-metrics-cert\") pod \"lws-controller-manager-f9f5fc96c-qvnsk\" (UID: \"6a5c29f8-366e-4e4e-ad77-e406d4213703\") " pod="openshift-lws-operator/lws-controller-manager-f9f5fc96c-qvnsk"
Jul 29 14:06:51.958665 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:51.958645    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6a5c29f8-366e-4e4e-ad77-e406d4213703-cert\") pod \"lws-controller-manager-f9f5fc96c-qvnsk\" (UID: \"6a5c29f8-366e-4e4e-ad77-e406d4213703\") " pod="openshift-lws-operator/lws-controller-manager-f9f5fc96c-qvnsk"
Jul 29 14:06:51.964224 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:51.964201    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6bjp\" (UniqueName: \"kubernetes.io/projected/6a5c29f8-366e-4e4e-ad77-e406d4213703-kube-api-access-w6bjp\") pod \"lws-controller-manager-f9f5fc96c-qvnsk\" (UID: \"6a5c29f8-366e-4e4e-ad77-e406d4213703\") " pod="openshift-lws-operator/lws-controller-manager-f9f5fc96c-qvnsk"
Jul 29 14:06:52.065919 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:52.065839    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-f9f5fc96c-qvnsk"
Jul 29 14:06:52.195983 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:52.195935    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-f9f5fc96c-qvnsk"]
Jul 29 14:06:52.198042 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:06:52.198015    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a5c29f8_366e_4e4e_ad77_e406d4213703.slice/crio-330fbcc2b34243a59865a912e21b6ef617181484f09539638dba4cc7b2c47a19 WatchSource:0}: Error finding container 330fbcc2b34243a59865a912e21b6ef617181484f09539638dba4cc7b2c47a19: Status 404 returned error can't find the container with id 330fbcc2b34243a59865a912e21b6ef617181484f09539638dba4cc7b2c47a19
Jul 29 14:06:53.017448 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:53.017409    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-f9f5fc96c-qvnsk" event={"ID":"6a5c29f8-366e-4e4e-ad77-e406d4213703","Type":"ContainerStarted","Data":"330fbcc2b34243a59865a912e21b6ef617181484f09539638dba4cc7b2c47a19"}
Jul 29 14:06:53.516560 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:53.516314    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:06:53.676720 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:53.676676    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:06:53.759684 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:53.759629    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:06:53.878049 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:53.877934    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:06:53.978469 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:53.978432    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:06:54.113337 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:54.113297    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:06:54.217744 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:54.217663    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:06:55.355827 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:55.355780    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:06:55.356176 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:55.355865    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:06:55.356176 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:55.355895    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:06:56.032624 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:56.032588    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl" event={"ID":"27d584d0-3cea-4000-910c-5d089ec8360e","Type":"ContainerStarted","Data":"a6c4da4bd38cd49fbc0f478261b4a79443a3f93f0f3c9e5312ef645ca14755ff"}
Jul 29 14:06:56.034096 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:56.034046    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-f9f5fc96c-qvnsk" event={"ID":"6a5c29f8-366e-4e4e-ad77-e406d4213703","Type":"ContainerStarted","Data":"ceb204d020e69743ca538f53fa46cd2f6f63cefeb1608e2a815b46407b2cf884"}
Jul 29 14:06:56.034224 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:56.034112    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-f9f5fc96c-qvnsk"
Jul 29 14:06:56.083401 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:56.083339    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl" podStartSLOduration=1.889975904 podStartE2EDuration="7.083319306s" totalImagesPullingTime="5.193343402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:06:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:06:56.081683373 +0000 UTC m=+345.972354708" watchObservedRunningTime="2026-07-29 14:06:56.083319306 +0000 UTC m=+345.973990642"
Jul 29 14:06:56.128932 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:56.128864    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-f9f5fc96c-qvnsk" podStartSLOduration=1.919765862 podStartE2EDuration="5.128843961s" totalImagesPullingTime="3.209078099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:06:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:06:56.128768345 +0000 UTC m=+346.019439680" watchObservedRunningTime="2026-07-29 14:06:56.128843961 +0000 UTC m=+346.019515298"
Jul 29 14:06:57.013299 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:57.013260    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"
Jul 29 14:06:57.017777 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:57.017753    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"
Jul 29 14:06:57.041730 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:57.041696    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"
Jul 29 14:06:57.042650 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:57.042630    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl"
Jul 29 14:06:57.998770 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:06:57.998741    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-pqrns"
Jul 29 14:07:07.044020 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:07:07.043946    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-f9f5fc96c-qvnsk"
Jul 29 14:07:42.463907 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:07:42.463857    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:07:42.632969 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:07:42.632923    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:07:42.725463 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:07:42.725374    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:07:43.404963 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:07:43.404927    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:07:43.621270 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:07:43.621220    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:12.352376 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:12.352337    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:15.745607 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:15.745562    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:16.558241 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:16.558199    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:17.240721 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:17.240680    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:17.397961 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:17.397919    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:20.165105 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:20.165068    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:20.459614 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:20.459522    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:20.567554 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:20.567516    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:22.231707 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:22.231674    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:22.343274 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:22.343236    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:22.744638 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:22.744592    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:23.529299 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:23.529262    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:25.423543 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:25.423505    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:27.672588 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:27.672553    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6fbcfc5544-9kctz"]
Jul 29 14:08:27.786878 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:27.786844    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:27.887147 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:27.887112    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:27.990395 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:27.990361    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:28.135589 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:28.135553    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:28.381129 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:28.381021    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:28.481408 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:28.481369    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:28.581028 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:28.580983    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:33.772185 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:33.772090    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:33.872435 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:33.872384    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:34.185599 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:34.185513    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:34.318887 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:34.318848    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:35.765130 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:35.765085    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:36.007007 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:36.006967    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:37.562642 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:37.562607    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:37.662237 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:37.662201    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:40.113415 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:40.113381    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:40.211993 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:40.211948    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:42.300008 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:42.299968    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:42.366456 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:42.366426    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:46.755993 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:46.755953    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:46.857394 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:46.857358    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:52.146872 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:52.146830    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:52.245245 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:52.245205    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:52.695186 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:52.695145    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6fbcfc5544-9kctz" podUID="25ba9889-7adb-4d8d-94b0-c05fd96a1981" containerName="console" containerID="cri-o://5a1cba4ca88d19b3c37df4c3d1b6357b2cd77889a3fd92cb25c63dd0a4cbf54f" gracePeriod=15
Jul 29 14:08:52.741605 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:52.741575    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:52.841430 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:52.841391    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:52.945181 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:52.945159    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fbcfc5544-9kctz"
Jul 29 14:08:53.042612 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:53.042583    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/25ba9889-7adb-4d8d-94b0-c05fd96a1981-console-serving-cert\") pod \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\" (UID: \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\") "
Jul 29 14:08:53.042986 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:53.042627    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/25ba9889-7adb-4d8d-94b0-c05fd96a1981-console-config\") pod \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\" (UID: \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\") "
Jul 29 14:08:53.042986 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:53.042657    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9kxlv\" (UniqueName: \"kubernetes.io/projected/25ba9889-7adb-4d8d-94b0-c05fd96a1981-kube-api-access-9kxlv\") pod \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\" (UID: \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\") "
Jul 29 14:08:53.042986 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:53.042691    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/25ba9889-7adb-4d8d-94b0-c05fd96a1981-service-ca\") pod \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\" (UID: \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\") "
Jul 29 14:08:53.042986 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:53.042708    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/25ba9889-7adb-4d8d-94b0-c05fd96a1981-oauth-serving-cert\") pod \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\" (UID: \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\") "
Jul 29 14:08:53.042986 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:53.042733    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/25ba9889-7adb-4d8d-94b0-c05fd96a1981-console-oauth-config\") pod \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\" (UID: \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\") "
Jul 29 14:08:53.042986 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:53.042770    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25ba9889-7adb-4d8d-94b0-c05fd96a1981-trusted-ca-bundle\") pod \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\" (UID: \"25ba9889-7adb-4d8d-94b0-c05fd96a1981\") "
Jul 29 14:08:53.043355 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:53.043152    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25ba9889-7adb-4d8d-94b0-c05fd96a1981-service-ca" (OuterVolumeSpecName: "service-ca") pod "25ba9889-7adb-4d8d-94b0-c05fd96a1981" (UID: "25ba9889-7adb-4d8d-94b0-c05fd96a1981"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 14:08:53.043405 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:53.043347    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25ba9889-7adb-4d8d-94b0-c05fd96a1981-console-config" (OuterVolumeSpecName: "console-config") pod "25ba9889-7adb-4d8d-94b0-c05fd96a1981" (UID: "25ba9889-7adb-4d8d-94b0-c05fd96a1981"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 14:08:53.043664 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:53.043640    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25ba9889-7adb-4d8d-94b0-c05fd96a1981-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "25ba9889-7adb-4d8d-94b0-c05fd96a1981" (UID: "25ba9889-7adb-4d8d-94b0-c05fd96a1981"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 14:08:53.043783 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:53.043757    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25ba9889-7adb-4d8d-94b0-c05fd96a1981-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "25ba9889-7adb-4d8d-94b0-c05fd96a1981" (UID: "25ba9889-7adb-4d8d-94b0-c05fd96a1981"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 14:08:53.044862 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:53.044829    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25ba9889-7adb-4d8d-94b0-c05fd96a1981-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "25ba9889-7adb-4d8d-94b0-c05fd96a1981" (UID: "25ba9889-7adb-4d8d-94b0-c05fd96a1981"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 14:08:53.044965 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:53.044843    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25ba9889-7adb-4d8d-94b0-c05fd96a1981-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "25ba9889-7adb-4d8d-94b0-c05fd96a1981" (UID: "25ba9889-7adb-4d8d-94b0-c05fd96a1981"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 14:08:53.045046 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:53.045031    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25ba9889-7adb-4d8d-94b0-c05fd96a1981-kube-api-access-9kxlv" (OuterVolumeSpecName: "kube-api-access-9kxlv") pod "25ba9889-7adb-4d8d-94b0-c05fd96a1981" (UID: "25ba9889-7adb-4d8d-94b0-c05fd96a1981"). InnerVolumeSpecName "kube-api-access-9kxlv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:08:53.143601 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:53.143569    2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/25ba9889-7adb-4d8d-94b0-c05fd96a1981-console-serving-cert\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:08:53.143601 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:53.143595    2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/25ba9889-7adb-4d8d-94b0-c05fd96a1981-console-config\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:08:53.143601 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:53.143605    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9kxlv\" (UniqueName: \"kubernetes.io/projected/25ba9889-7adb-4d8d-94b0-c05fd96a1981-kube-api-access-9kxlv\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:08:53.143813 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:53.143613    2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/25ba9889-7adb-4d8d-94b0-c05fd96a1981-service-ca\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:08:53.143813 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:53.143623    2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/25ba9889-7adb-4d8d-94b0-c05fd96a1981-oauth-serving-cert\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:08:53.143813 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:53.143632    2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/25ba9889-7adb-4d8d-94b0-c05fd96a1981-console-oauth-config\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:08:53.143813 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:53.143640    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25ba9889-7adb-4d8d-94b0-c05fd96a1981-trusted-ca-bundle\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:08:53.510438 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:53.510409    2559 generic.go:356] "Generic (PLEG): container finished" podID="25ba9889-7adb-4d8d-94b0-c05fd96a1981" containerID="5a1cba4ca88d19b3c37df4c3d1b6357b2cd77889a3fd92cb25c63dd0a4cbf54f" exitCode=0
Jul 29 14:08:53.510876 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:53.510452    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fbcfc5544-9kctz"
Jul 29 14:08:53.510876 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:53.510465    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fbcfc5544-9kctz" event={"ID":"25ba9889-7adb-4d8d-94b0-c05fd96a1981","Type":"ContainerDied","Data":"5a1cba4ca88d19b3c37df4c3d1b6357b2cd77889a3fd92cb25c63dd0a4cbf54f"}
Jul 29 14:08:53.510876 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:53.510503    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fbcfc5544-9kctz" event={"ID":"25ba9889-7adb-4d8d-94b0-c05fd96a1981","Type":"ContainerDied","Data":"4db8b35785140abdaaebf294eaf4cefe0866d4f700d77d35356c51063279d581"}
Jul 29 14:08:53.510876 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:53.510522    2559 scope.go:117] "RemoveContainer" containerID="5a1cba4ca88d19b3c37df4c3d1b6357b2cd77889a3fd92cb25c63dd0a4cbf54f"
Jul 29 14:08:53.519198 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:53.519182    2559 scope.go:117] "RemoveContainer" containerID="5a1cba4ca88d19b3c37df4c3d1b6357b2cd77889a3fd92cb25c63dd0a4cbf54f"
Jul 29 14:08:53.519429 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:08:53.519411    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a1cba4ca88d19b3c37df4c3d1b6357b2cd77889a3fd92cb25c63dd0a4cbf54f\": container with ID starting with 5a1cba4ca88d19b3c37df4c3d1b6357b2cd77889a3fd92cb25c63dd0a4cbf54f not found: ID does not exist" containerID="5a1cba4ca88d19b3c37df4c3d1b6357b2cd77889a3fd92cb25c63dd0a4cbf54f"
Jul 29 14:08:53.519493 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:53.519437    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a1cba4ca88d19b3c37df4c3d1b6357b2cd77889a3fd92cb25c63dd0a4cbf54f"} err="failed to get container status \"5a1cba4ca88d19b3c37df4c3d1b6357b2cd77889a3fd92cb25c63dd0a4cbf54f\": rpc error: code = NotFound desc = could not find container \"5a1cba4ca88d19b3c37df4c3d1b6357b2cd77889a3fd92cb25c63dd0a4cbf54f\": container with ID starting with 5a1cba4ca88d19b3c37df4c3d1b6357b2cd77889a3fd92cb25c63dd0a4cbf54f not found: ID does not exist"
Jul 29 14:08:53.535098 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:53.535075    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6fbcfc5544-9kctz"]
Jul 29 14:08:53.541920 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:53.541899    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6fbcfc5544-9kctz"]
Jul 29 14:08:54.636024 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:54.635991    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25ba9889-7adb-4d8d-94b0-c05fd96a1981" path="/var/lib/kubelet/pods/25ba9889-7adb-4d8d-94b0-c05fd96a1981/volumes"
Jul 29 14:08:56.543660 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:56.543629    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:56.647698 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:56.647658    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:08:59.960604 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:08:59.960565    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:09:00.061609 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:00.061576    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:09:27.321863 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:27.321825    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:09:27.521166 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:27.521130    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:09:41.201599 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:41.201563    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:09:41.309364 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:41.309321    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:09:41.392205 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:41.392166    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:09:41.989894 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:41.989856    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-dndct"]
Jul 29 14:09:41.990190 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:41.990176    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25ba9889-7adb-4d8d-94b0-c05fd96a1981" containerName="console"
Jul 29 14:09:41.990248 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:41.990192    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="25ba9889-7adb-4d8d-94b0-c05fd96a1981" containerName="console"
Jul 29 14:09:41.990282 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:41.990251    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="25ba9889-7adb-4d8d-94b0-c05fd96a1981" containerName="console"
Jul 29 14:09:41.993112 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:41.993092    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-dndct"
Jul 29 14:09:41.993520 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:41.993498    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nznhf\" (UniqueName: \"kubernetes.io/projected/7d14b172-bbe2-42db-9bd8-78eb4027574c-kube-api-access-nznhf\") pod \"authorino-58cd955675-dndct\" (UID: \"7d14b172-bbe2-42db-9bd8-78eb4027574c\") " pod="kuadrant-system/authorino-58cd955675-dndct"
Jul 29 14:09:41.995786 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:41.995769    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 29 14:09:41.996994 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:41.996977    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-7hsb5\""
Jul 29 14:09:41.997132 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:41.997012    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 29 14:09:41.999855 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:41.999803    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-dndct"]
Jul 29 14:09:42.022220 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:42.022184    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:09:42.079122 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:42.079090    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:09:42.094380 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:42.094349    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nznhf\" (UniqueName: \"kubernetes.io/projected/7d14b172-bbe2-42db-9bd8-78eb4027574c-kube-api-access-nznhf\") pod \"authorino-58cd955675-dndct\" (UID: \"7d14b172-bbe2-42db-9bd8-78eb4027574c\") " pod="kuadrant-system/authorino-58cd955675-dndct"
Jul 29 14:09:42.102939 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:42.102911    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nznhf\" (UniqueName: \"kubernetes.io/projected/7d14b172-bbe2-42db-9bd8-78eb4027574c-kube-api-access-nznhf\") pod \"authorino-58cd955675-dndct\" (UID: \"7d14b172-bbe2-42db-9bd8-78eb4027574c\") " pod="kuadrant-system/authorino-58cd955675-dndct"
Jul 29 14:09:42.175132 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:42.175099    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-gmxd7"]
Jul 29 14:09:42.178265 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:42.178246    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-gmxd7"
Jul 29 14:09:42.183965 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:42.183932    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-gmxd7"]
Jul 29 14:09:42.195038 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:42.195013    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfdpt\" (UniqueName: \"kubernetes.io/projected/ad157779-d32c-4796-976a-9da20c19032c-kube-api-access-vfdpt\") pod \"authorino-75d556bd77-gmxd7\" (UID: \"ad157779-d32c-4796-976a-9da20c19032c\") " pod="kuadrant-system/authorino-75d556bd77-gmxd7"
Jul 29 14:09:42.295501 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:42.295418    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfdpt\" (UniqueName: \"kubernetes.io/projected/ad157779-d32c-4796-976a-9da20c19032c-kube-api-access-vfdpt\") pod \"authorino-75d556bd77-gmxd7\" (UID: \"ad157779-d32c-4796-976a-9da20c19032c\") " pod="kuadrant-system/authorino-75d556bd77-gmxd7"
Jul 29 14:09:42.302024 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:42.302004    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-dndct"
Jul 29 14:09:42.304036 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:42.304013    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfdpt\" (UniqueName: \"kubernetes.io/projected/ad157779-d32c-4796-976a-9da20c19032c-kube-api-access-vfdpt\") pod \"authorino-75d556bd77-gmxd7\" (UID: \"ad157779-d32c-4796-976a-9da20c19032c\") " pod="kuadrant-system/authorino-75d556bd77-gmxd7"
Jul 29 14:09:42.425811 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:42.425778    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-dndct"]
Jul 29 14:09:42.430135 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:09:42.430103    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7d14b172_bbe2_42db_9bd8_78eb4027574c.slice/crio-5ab6276bfb32c1174f4c76384f3e74d899466864d278e132be9f301c671fa6f3 WatchSource:0}: Error finding container 5ab6276bfb32c1174f4c76384f3e74d899466864d278e132be9f301c671fa6f3: Status 404 returned error can't find the container with id 5ab6276bfb32c1174f4c76384f3e74d899466864d278e132be9f301c671fa6f3
Jul 29 14:09:42.488185 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:42.488146    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-gmxd7"
Jul 29 14:09:42.612837 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:42.612802    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-gmxd7"]
Jul 29 14:09:42.615747 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:09:42.615716    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad157779_d32c_4796_976a_9da20c19032c.slice/crio-14d2ebd9a02b34ec4089c2b6cd79f9b3236a71737b6d7b15ad9b422559388754 WatchSource:0}: Error finding container 14d2ebd9a02b34ec4089c2b6cd79f9b3236a71737b6d7b15ad9b422559388754: Status 404 returned error can't find the container with id 14d2ebd9a02b34ec4089c2b6cd79f9b3236a71737b6d7b15ad9b422559388754
Jul 29 14:09:42.706593 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:42.706555    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-gmxd7" event={"ID":"ad157779-d32c-4796-976a-9da20c19032c","Type":"ContainerStarted","Data":"14d2ebd9a02b34ec4089c2b6cd79f9b3236a71737b6d7b15ad9b422559388754"}
Jul 29 14:09:42.707617 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:42.707590    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-dndct" event={"ID":"7d14b172-bbe2-42db-9bd8-78eb4027574c","Type":"ContainerStarted","Data":"5ab6276bfb32c1174f4c76384f3e74d899466864d278e132be9f301c671fa6f3"}
Jul 29 14:09:46.732943 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:46.732904    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-gmxd7" event={"ID":"ad157779-d32c-4796-976a-9da20c19032c","Type":"ContainerStarted","Data":"8105c6ede976e448bc186f20b75a43e9a54c8ebff2a358200875aae3b5cf7775"}
Jul 29 14:09:46.734466 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:46.734440    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-dndct" event={"ID":"7d14b172-bbe2-42db-9bd8-78eb4027574c","Type":"ContainerStarted","Data":"6b235de4aa733f2b700e10ed397453fccedbccf29934f0708613a5b3b2962f3e"}
Jul 29 14:09:46.747571 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:46.747522    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-gmxd7" podStartSLOduration=1.2501759080000001 podStartE2EDuration="4.747508151s" totalImagesPullingTime="3.497332243s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:09:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:09:46.74618266 +0000 UTC m=+516.636853993" watchObservedRunningTime="2026-07-29 14:09:46.747508151 +0000 UTC m=+516.638179486"
Jul 29 14:09:46.760809 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:46.760760    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-dndct" podStartSLOduration=2.0884326619999998 podStartE2EDuration="5.760747738s" totalImagesPullingTime="3.672315076s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:09:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:09:46.759225289 +0000 UTC m=+516.649896625" watchObservedRunningTime="2026-07-29 14:09:46.760747738 +0000 UTC m=+516.651419108"
Jul 29 14:09:46.778108 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:46.778051    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-dndct"]
Jul 29 14:09:48.742305 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:48.742269    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-dndct" podUID="7d14b172-bbe2-42db-9bd8-78eb4027574c" containerName="authorino" containerID="cri-o://6b235de4aa733f2b700e10ed397453fccedbccf29934f0708613a5b3b2962f3e" gracePeriod=30
Jul 29 14:09:48.981823 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:48.981793    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-dndct"
Jul 29 14:09:49.053581 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:49.053507    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nznhf\" (UniqueName: \"kubernetes.io/projected/7d14b172-bbe2-42db-9bd8-78eb4027574c-kube-api-access-nznhf\") pod \"7d14b172-bbe2-42db-9bd8-78eb4027574c\" (UID: \"7d14b172-bbe2-42db-9bd8-78eb4027574c\") "
Jul 29 14:09:49.055589 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:49.055561    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d14b172-bbe2-42db-9bd8-78eb4027574c-kube-api-access-nznhf" (OuterVolumeSpecName: "kube-api-access-nznhf") pod "7d14b172-bbe2-42db-9bd8-78eb4027574c" (UID: "7d14b172-bbe2-42db-9bd8-78eb4027574c"). InnerVolumeSpecName "kube-api-access-nznhf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:09:49.154596 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:49.154566    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nznhf\" (UniqueName: \"kubernetes.io/projected/7d14b172-bbe2-42db-9bd8-78eb4027574c-kube-api-access-nznhf\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:09:49.752399 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:49.752361    2559 generic.go:356] "Generic (PLEG): container finished" podID="7d14b172-bbe2-42db-9bd8-78eb4027574c" containerID="6b235de4aa733f2b700e10ed397453fccedbccf29934f0708613a5b3b2962f3e" exitCode=0
Jul 29 14:09:49.752399 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:49.752403    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-dndct" event={"ID":"7d14b172-bbe2-42db-9bd8-78eb4027574c","Type":"ContainerDied","Data":"6b235de4aa733f2b700e10ed397453fccedbccf29934f0708613a5b3b2962f3e"}
Jul 29 14:09:49.752861 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:49.752414    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-dndct"
Jul 29 14:09:49.752861 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:49.752429    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-dndct" event={"ID":"7d14b172-bbe2-42db-9bd8-78eb4027574c","Type":"ContainerDied","Data":"5ab6276bfb32c1174f4c76384f3e74d899466864d278e132be9f301c671fa6f3"}
Jul 29 14:09:49.752861 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:49.752445    2559 scope.go:117] "RemoveContainer" containerID="6b235de4aa733f2b700e10ed397453fccedbccf29934f0708613a5b3b2962f3e"
Jul 29 14:09:49.761019 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:49.760998    2559 scope.go:117] "RemoveContainer" containerID="6b235de4aa733f2b700e10ed397453fccedbccf29934f0708613a5b3b2962f3e"
Jul 29 14:09:49.761308 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:09:49.761284    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b235de4aa733f2b700e10ed397453fccedbccf29934f0708613a5b3b2962f3e\": container with ID starting with 6b235de4aa733f2b700e10ed397453fccedbccf29934f0708613a5b3b2962f3e not found: ID does not exist" containerID="6b235de4aa733f2b700e10ed397453fccedbccf29934f0708613a5b3b2962f3e"
Jul 29 14:09:49.761406 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:49.761313    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b235de4aa733f2b700e10ed397453fccedbccf29934f0708613a5b3b2962f3e"} err="failed to get container status \"6b235de4aa733f2b700e10ed397453fccedbccf29934f0708613a5b3b2962f3e\": rpc error: code = NotFound desc = could not find container \"6b235de4aa733f2b700e10ed397453fccedbccf29934f0708613a5b3b2962f3e\": container with ID starting with 6b235de4aa733f2b700e10ed397453fccedbccf29934f0708613a5b3b2962f3e not found: ID does not exist"
Jul 29 14:09:49.775612 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:49.775590    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-dndct"]
Jul 29 14:09:49.780992 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:49.780968    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-dndct"]
Jul 29 14:09:50.636274 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:50.636244    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d14b172-bbe2-42db-9bd8-78eb4027574c" path="/var/lib/kubelet/pods/7d14b172-bbe2-42db-9bd8-78eb4027574c/volumes"
Jul 29 14:09:57.509307 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:57.509273    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:09:57.573010 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:57.572974    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:09:58.456251 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:58.456216    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:09:58.551570 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:58.551525    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:09:58.969801 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:09:58.969760    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:10:11.631414 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:11.631380    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:10:11.734360 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:11.734322    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:10:11.865974 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:11.865943    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tl2xr"]
Jul 29 14:10:11.866235 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:11.866223    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d14b172-bbe2-42db-9bd8-78eb4027574c" containerName="authorino"
Jul 29 14:10:11.866286 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:11.866236    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d14b172-bbe2-42db-9bd8-78eb4027574c" containerName="authorino"
Jul 29 14:10:11.866320 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:11.866292    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d14b172-bbe2-42db-9bd8-78eb4027574c" containerName="authorino"
Jul 29 14:10:11.870221 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:11.870205    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tl2xr"
Jul 29 14:10:11.877248 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:11.877223    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tl2xr"]
Jul 29 14:10:11.933008 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:11.932919    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrj4k\" (UniqueName: \"kubernetes.io/projected/da99698b-1aa3-4040-b522-4e0a4db6a10c-kube-api-access-mrj4k\") pod \"authorino-5d4b67c446-tl2xr\" (UID: \"da99698b-1aa3-4040-b522-4e0a4db6a10c\") " pod="kuadrant-system/authorino-5d4b67c446-tl2xr"
Jul 29 14:10:12.034296 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:12.034259    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrj4k\" (UniqueName: \"kubernetes.io/projected/da99698b-1aa3-4040-b522-4e0a4db6a10c-kube-api-access-mrj4k\") pod \"authorino-5d4b67c446-tl2xr\" (UID: \"da99698b-1aa3-4040-b522-4e0a4db6a10c\") " pod="kuadrant-system/authorino-5d4b67c446-tl2xr"
Jul 29 14:10:12.042211 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:12.042186    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrj4k\" (UniqueName: \"kubernetes.io/projected/da99698b-1aa3-4040-b522-4e0a4db6a10c-kube-api-access-mrj4k\") pod \"authorino-5d4b67c446-tl2xr\" (UID: \"da99698b-1aa3-4040-b522-4e0a4db6a10c\") " pod="kuadrant-system/authorino-5d4b67c446-tl2xr"
Jul 29 14:10:12.099559 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:12.099525    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tl2xr"]
Jul 29 14:10:12.099754 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:12.099741    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tl2xr"
Jul 29 14:10:12.125119 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:12.125075    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-67d8bf4898-72j9p"]
Jul 29 14:10:12.128362 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:12.128343    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67d8bf4898-72j9p"
Jul 29 14:10:12.135407 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:12.135349    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-67d8bf4898-72j9p"]
Jul 29 14:10:12.173573 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:12.173544    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-67d8bf4898-72j9p"]
Jul 29 14:10:12.173797 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:10:12.173776    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-sbg8n], unattached volumes=[], failed to process volumes=[kube-api-access-sbg8n]: context canceled" pod="kuadrant-system/authorino-67d8bf4898-72j9p" podUID="0cce857b-3157-4625-afcc-3b7bc24671f1"
Jul 29 14:10:12.233228 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:12.233157    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tl2xr"]
Jul 29 14:10:12.235605 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:12.235579    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbg8n\" (UniqueName: \"kubernetes.io/projected/0cce857b-3157-4625-afcc-3b7bc24671f1-kube-api-access-sbg8n\") pod \"authorino-67d8bf4898-72j9p\" (UID: \"0cce857b-3157-4625-afcc-3b7bc24671f1\") " pod="kuadrant-system/authorino-67d8bf4898-72j9p"
Jul 29 14:10:12.235718 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:10:12.235613    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podda99698b_1aa3_4040_b522_4e0a4db6a10c.slice/crio-9e8b823ef7583824320d2cfb723be035237aeec799a7fc1b831dd838e69426f2 WatchSource:0}: Error finding container 9e8b823ef7583824320d2cfb723be035237aeec799a7fc1b831dd838e69426f2: Status 404 returned error can't find the container with id 9e8b823ef7583824320d2cfb723be035237aeec799a7fc1b831dd838e69426f2
Jul 29 14:10:12.335999 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:12.335967    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbg8n\" (UniqueName: \"kubernetes.io/projected/0cce857b-3157-4625-afcc-3b7bc24671f1-kube-api-access-sbg8n\") pod \"authorino-67d8bf4898-72j9p\" (UID: \"0cce857b-3157-4625-afcc-3b7bc24671f1\") " pod="kuadrant-system/authorino-67d8bf4898-72j9p"
Jul 29 14:10:12.344629 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:12.344598    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbg8n\" (UniqueName: \"kubernetes.io/projected/0cce857b-3157-4625-afcc-3b7bc24671f1-kube-api-access-sbg8n\") pod \"authorino-67d8bf4898-72j9p\" (UID: \"0cce857b-3157-4625-afcc-3b7bc24671f1\") " pod="kuadrant-system/authorino-67d8bf4898-72j9p"
Jul 29 14:10:12.847673 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:12.847643    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67d8bf4898-72j9p"
Jul 29 14:10:12.847673 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:12.847669    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-tl2xr" event={"ID":"da99698b-1aa3-4040-b522-4e0a4db6a10c","Type":"ContainerStarted","Data":"9e8b823ef7583824320d2cfb723be035237aeec799a7fc1b831dd838e69426f2"}
Jul 29 14:10:12.854720 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:12.854697    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67d8bf4898-72j9p"
Jul 29 14:10:12.941797 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:12.941774    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbg8n\" (UniqueName: \"kubernetes.io/projected/0cce857b-3157-4625-afcc-3b7bc24671f1-kube-api-access-sbg8n\") pod \"0cce857b-3157-4625-afcc-3b7bc24671f1\" (UID: \"0cce857b-3157-4625-afcc-3b7bc24671f1\") "
Jul 29 14:10:12.943822 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:12.943797    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cce857b-3157-4625-afcc-3b7bc24671f1-kube-api-access-sbg8n" (OuterVolumeSpecName: "kube-api-access-sbg8n") pod "0cce857b-3157-4625-afcc-3b7bc24671f1" (UID: "0cce857b-3157-4625-afcc-3b7bc24671f1"). InnerVolumeSpecName "kube-api-access-sbg8n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:10:13.042787 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:13.042747    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbg8n\" (UniqueName: \"kubernetes.io/projected/0cce857b-3157-4625-afcc-3b7bc24671f1-kube-api-access-sbg8n\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:10:13.853137 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:13.853097    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-tl2xr" event={"ID":"da99698b-1aa3-4040-b522-4e0a4db6a10c","Type":"ContainerStarted","Data":"6270d91e66aec5aa94d8f7cd2a51edeec7d0559e062e7254911aff28faff6c35"}
Jul 29 14:10:13.853137 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:13.853121    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67d8bf4898-72j9p"
Jul 29 14:10:13.853663 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:13.853132    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-tl2xr" podUID="da99698b-1aa3-4040-b522-4e0a4db6a10c" containerName="authorino" containerID="cri-o://6270d91e66aec5aa94d8f7cd2a51edeec7d0559e062e7254911aff28faff6c35" gracePeriod=30
Jul 29 14:10:13.869569 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:13.869524    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-tl2xr" podStartSLOduration=2.19129029 podStartE2EDuration="2.869508031s" totalImagesPullingTime="678.217741ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:10:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:10:13.868671334 +0000 UTC m=+543.759342670" watchObservedRunningTime="2026-07-29 14:10:13.869508031 +0000 UTC m=+543.760179366"
Jul 29 14:10:13.895025 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:13.895001    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-67d8bf4898-72j9p"]
Jul 29 14:10:13.900950 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:13.900927    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-67d8bf4898-72j9p"]
Jul 29 14:10:14.086152 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:14.086133    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tl2xr"
Jul 29 14:10:14.152608 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:14.152531    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mrj4k\" (UniqueName: \"kubernetes.io/projected/da99698b-1aa3-4040-b522-4e0a4db6a10c-kube-api-access-mrj4k\") pod \"da99698b-1aa3-4040-b522-4e0a4db6a10c\" (UID: \"da99698b-1aa3-4040-b522-4e0a4db6a10c\") "
Jul 29 14:10:14.154534 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:14.154509    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da99698b-1aa3-4040-b522-4e0a4db6a10c-kube-api-access-mrj4k" (OuterVolumeSpecName: "kube-api-access-mrj4k") pod "da99698b-1aa3-4040-b522-4e0a4db6a10c" (UID: "da99698b-1aa3-4040-b522-4e0a4db6a10c"). InnerVolumeSpecName "kube-api-access-mrj4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:10:14.254109 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:14.254077    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrj4k\" (UniqueName: \"kubernetes.io/projected/da99698b-1aa3-4040-b522-4e0a4db6a10c-kube-api-access-mrj4k\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:10:14.635695 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:14.635661    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cce857b-3157-4625-afcc-3b7bc24671f1" path="/var/lib/kubelet/pods/0cce857b-3157-4625-afcc-3b7bc24671f1/volumes"
Jul 29 14:10:14.858117 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:14.858081    2559 generic.go:356] "Generic (PLEG): container finished" podID="da99698b-1aa3-4040-b522-4e0a4db6a10c" containerID="6270d91e66aec5aa94d8f7cd2a51edeec7d0559e062e7254911aff28faff6c35" exitCode=0
Jul 29 14:10:14.858492 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:14.858132    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tl2xr"
Jul 29 14:10:14.858492 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:14.858152    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-tl2xr" event={"ID":"da99698b-1aa3-4040-b522-4e0a4db6a10c","Type":"ContainerDied","Data":"6270d91e66aec5aa94d8f7cd2a51edeec7d0559e062e7254911aff28faff6c35"}
Jul 29 14:10:14.858492 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:14.858183    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-tl2xr" event={"ID":"da99698b-1aa3-4040-b522-4e0a4db6a10c","Type":"ContainerDied","Data":"9e8b823ef7583824320d2cfb723be035237aeec799a7fc1b831dd838e69426f2"}
Jul 29 14:10:14.858492 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:14.858200    2559 scope.go:117] "RemoveContainer" containerID="6270d91e66aec5aa94d8f7cd2a51edeec7d0559e062e7254911aff28faff6c35"
Jul 29 14:10:14.865768 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:14.865751    2559 scope.go:117] "RemoveContainer" containerID="6270d91e66aec5aa94d8f7cd2a51edeec7d0559e062e7254911aff28faff6c35"
Jul 29 14:10:14.866024 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:10:14.866006    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6270d91e66aec5aa94d8f7cd2a51edeec7d0559e062e7254911aff28faff6c35\": container with ID starting with 6270d91e66aec5aa94d8f7cd2a51edeec7d0559e062e7254911aff28faff6c35 not found: ID does not exist" containerID="6270d91e66aec5aa94d8f7cd2a51edeec7d0559e062e7254911aff28faff6c35"
Jul 29 14:10:14.866083 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:14.866032    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6270d91e66aec5aa94d8f7cd2a51edeec7d0559e062e7254911aff28faff6c35"} err="failed to get container status \"6270d91e66aec5aa94d8f7cd2a51edeec7d0559e062e7254911aff28faff6c35\": rpc error: code = NotFound desc = could not find container \"6270d91e66aec5aa94d8f7cd2a51edeec7d0559e062e7254911aff28faff6c35\": container with ID starting with 6270d91e66aec5aa94d8f7cd2a51edeec7d0559e062e7254911aff28faff6c35 not found: ID does not exist"
Jul 29 14:10:14.875849 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:14.875828    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tl2xr"]
Jul 29 14:10:14.879974 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:14.879955    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tl2xr"]
Jul 29 14:10:16.410999 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:16.410965    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-gmxd7"]
Jul 29 14:10:16.411381 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:16.411167    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-gmxd7" podUID="ad157779-d32c-4796-976a-9da20c19032c" containerName="authorino" containerID="cri-o://8105c6ede976e448bc186f20b75a43e9a54c8ebff2a358200875aae3b5cf7775" gracePeriod=30
Jul 29 14:10:16.636152 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:16.636122    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da99698b-1aa3-4040-b522-4e0a4db6a10c" path="/var/lib/kubelet/pods/da99698b-1aa3-4040-b522-4e0a4db6a10c/volumes"
Jul 29 14:10:16.656937 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:16.656915    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-gmxd7"
Jul 29 14:10:16.773884 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:16.773857    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfdpt\" (UniqueName: \"kubernetes.io/projected/ad157779-d32c-4796-976a-9da20c19032c-kube-api-access-vfdpt\") pod \"ad157779-d32c-4796-976a-9da20c19032c\" (UID: \"ad157779-d32c-4796-976a-9da20c19032c\") "
Jul 29 14:10:16.775833 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:16.775806    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad157779-d32c-4796-976a-9da20c19032c-kube-api-access-vfdpt" (OuterVolumeSpecName: "kube-api-access-vfdpt") pod "ad157779-d32c-4796-976a-9da20c19032c" (UID: "ad157779-d32c-4796-976a-9da20c19032c"). InnerVolumeSpecName "kube-api-access-vfdpt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:10:16.869497 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:16.869466    2559 generic.go:356] "Generic (PLEG): container finished" podID="ad157779-d32c-4796-976a-9da20c19032c" containerID="8105c6ede976e448bc186f20b75a43e9a54c8ebff2a358200875aae3b5cf7775" exitCode=0
Jul 29 14:10:16.869655 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:16.869507    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-gmxd7"
Jul 29 14:10:16.869655 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:16.869527    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-gmxd7" event={"ID":"ad157779-d32c-4796-976a-9da20c19032c","Type":"ContainerDied","Data":"8105c6ede976e448bc186f20b75a43e9a54c8ebff2a358200875aae3b5cf7775"}
Jul 29 14:10:16.869655 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:16.869557    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-gmxd7" event={"ID":"ad157779-d32c-4796-976a-9da20c19032c","Type":"ContainerDied","Data":"14d2ebd9a02b34ec4089c2b6cd79f9b3236a71737b6d7b15ad9b422559388754"}
Jul 29 14:10:16.869655 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:16.869573    2559 scope.go:117] "RemoveContainer" containerID="8105c6ede976e448bc186f20b75a43e9a54c8ebff2a358200875aae3b5cf7775"
Jul 29 14:10:16.874553 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:16.874530    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfdpt\" (UniqueName: \"kubernetes.io/projected/ad157779-d32c-4796-976a-9da20c19032c-kube-api-access-vfdpt\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:10:16.877891 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:16.877876    2559 scope.go:117] "RemoveContainer" containerID="8105c6ede976e448bc186f20b75a43e9a54c8ebff2a358200875aae3b5cf7775"
Jul 29 14:10:16.878176 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:10:16.878158    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8105c6ede976e448bc186f20b75a43e9a54c8ebff2a358200875aae3b5cf7775\": container with ID starting with 8105c6ede976e448bc186f20b75a43e9a54c8ebff2a358200875aae3b5cf7775 not found: ID does not exist" containerID="8105c6ede976e448bc186f20b75a43e9a54c8ebff2a358200875aae3b5cf7775"
Jul 29 14:10:16.878235 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:16.878182    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8105c6ede976e448bc186f20b75a43e9a54c8ebff2a358200875aae3b5cf7775"} err="failed to get container status \"8105c6ede976e448bc186f20b75a43e9a54c8ebff2a358200875aae3b5cf7775\": rpc error: code = NotFound desc = could not find container \"8105c6ede976e448bc186f20b75a43e9a54c8ebff2a358200875aae3b5cf7775\": container with ID starting with 8105c6ede976e448bc186f20b75a43e9a54c8ebff2a358200875aae3b5cf7775 not found: ID does not exist"
Jul 29 14:10:16.895339 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:16.895320    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-gmxd7"]
Jul 29 14:10:16.899445 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:16.899423    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-gmxd7"]
Jul 29 14:10:18.635898 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:18.635868    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad157779-d32c-4796-976a-9da20c19032c" path="/var/lib/kubelet/pods/ad157779-d32c-4796-976a-9da20c19032c/volumes"
Jul 29 14:10:21.193780 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:21.193743    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:10:21.264792 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:21.264758    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:10:21.360389 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:21.360354    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:10:21.461521 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:21.461435    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:10:24.265794 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:24.265751    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:10:24.368132 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:24.368085    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:10:43.378337 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:43.378300    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:10:43.478282 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:43.478229    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:10:55.869724 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:55.869682    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:10:55.962053 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:55.962019    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:10:56.596213 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:56.596177    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-968d69cc5-m82rk"]
Jul 29 14:10:56.596478 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:56.596466    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad157779-d32c-4796-976a-9da20c19032c" containerName="authorino"
Jul 29 14:10:56.596521 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:56.596480    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad157779-d32c-4796-976a-9da20c19032c" containerName="authorino"
Jul 29 14:10:56.596521 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:56.596497    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da99698b-1aa3-4040-b522-4e0a4db6a10c" containerName="authorino"
Jul 29 14:10:56.596521 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:56.596502    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="da99698b-1aa3-4040-b522-4e0a4db6a10c" containerName="authorino"
Jul 29 14:10:56.596625 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:56.596553    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad157779-d32c-4796-976a-9da20c19032c" containerName="authorino"
Jul 29 14:10:56.596625 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:56.596562    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="da99698b-1aa3-4040-b522-4e0a4db6a10c" containerName="authorino"
Jul 29 14:10:56.599437 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:56.599421    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-968d69cc5-m82rk"
Jul 29 14:10:56.602987 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:56.602960    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 29 14:10:56.603154 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:56.603094    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 29 14:10:56.603154 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:56.603131    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 14:10:56.610951 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:56.610930    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-968d69cc5-m82rk"]
Jul 29 14:10:56.684925 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:56.684889    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8gmp\" (UniqueName: \"kubernetes.io/projected/360e93a9-d4c3-463f-9f5a-859e2d222e96-kube-api-access-l8gmp\") pod \"maas-api-968d69cc5-m82rk\" (UID: \"360e93a9-d4c3-463f-9f5a-859e2d222e96\") " pod="odh-ai-gateway-infra/maas-api-968d69cc5-m82rk"
Jul 29 14:10:56.684925 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:56.684925    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/360e93a9-d4c3-463f-9f5a-859e2d222e96-maas-api-tls\") pod \"maas-api-968d69cc5-m82rk\" (UID: \"360e93a9-d4c3-463f-9f5a-859e2d222e96\") " pod="odh-ai-gateway-infra/maas-api-968d69cc5-m82rk"
Jul 29 14:10:56.685156 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:56.684944    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/360e93a9-d4c3-463f-9f5a-859e2d222e96-openshift-service-ca\") pod \"maas-api-968d69cc5-m82rk\" (UID: \"360e93a9-d4c3-463f-9f5a-859e2d222e96\") " pod="odh-ai-gateway-infra/maas-api-968d69cc5-m82rk"
Jul 29 14:10:56.786278 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:56.786242    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8gmp\" (UniqueName: \"kubernetes.io/projected/360e93a9-d4c3-463f-9f5a-859e2d222e96-kube-api-access-l8gmp\") pod \"maas-api-968d69cc5-m82rk\" (UID: \"360e93a9-d4c3-463f-9f5a-859e2d222e96\") " pod="odh-ai-gateway-infra/maas-api-968d69cc5-m82rk"
Jul 29 14:10:56.786441 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:56.786283    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/360e93a9-d4c3-463f-9f5a-859e2d222e96-maas-api-tls\") pod \"maas-api-968d69cc5-m82rk\" (UID: \"360e93a9-d4c3-463f-9f5a-859e2d222e96\") " pod="odh-ai-gateway-infra/maas-api-968d69cc5-m82rk"
Jul 29 14:10:56.786441 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:56.786308    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/360e93a9-d4c3-463f-9f5a-859e2d222e96-openshift-service-ca\") pod \"maas-api-968d69cc5-m82rk\" (UID: \"360e93a9-d4c3-463f-9f5a-859e2d222e96\") " pod="odh-ai-gateway-infra/maas-api-968d69cc5-m82rk"
Jul 29 14:10:56.786998 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:56.786970    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/360e93a9-d4c3-463f-9f5a-859e2d222e96-openshift-service-ca\") pod \"maas-api-968d69cc5-m82rk\" (UID: \"360e93a9-d4c3-463f-9f5a-859e2d222e96\") " pod="odh-ai-gateway-infra/maas-api-968d69cc5-m82rk"
Jul 29 14:10:56.788638 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:56.788619    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/360e93a9-d4c3-463f-9f5a-859e2d222e96-maas-api-tls\") pod \"maas-api-968d69cc5-m82rk\" (UID: \"360e93a9-d4c3-463f-9f5a-859e2d222e96\") " pod="odh-ai-gateway-infra/maas-api-968d69cc5-m82rk"
Jul 29 14:10:56.794331 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:56.794307    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:10:56.795389 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:56.795367    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8gmp\" (UniqueName: \"kubernetes.io/projected/360e93a9-d4c3-463f-9f5a-859e2d222e96-kube-api-access-l8gmp\") pod \"maas-api-968d69cc5-m82rk\" (UID: \"360e93a9-d4c3-463f-9f5a-859e2d222e96\") " pod="odh-ai-gateway-infra/maas-api-968d69cc5-m82rk"
Jul 29 14:10:56.895618 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:56.895523    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:10:56.909484 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:56.909455    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-968d69cc5-m82rk"
Jul 29 14:10:57.042963 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:57.042934    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-968d69cc5-m82rk"]
Jul 29 14:10:57.044891 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:10:57.044866    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod360e93a9_d4c3_463f_9f5a_859e2d222e96.slice/crio-a7428e84888301b1884431aafeef4e2d6a83a67f0ca340ef43d48e62e45d1d16 WatchSource:0}: Error finding container a7428e84888301b1884431aafeef4e2d6a83a67f0ca340ef43d48e62e45d1d16: Status 404 returned error can't find the container with id a7428e84888301b1884431aafeef4e2d6a83a67f0ca340ef43d48e62e45d1d16
Jul 29 14:10:57.143110 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:57.143046    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:10:57.244671 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:57.244638    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:10:58.039516 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:10:58.039478    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-968d69cc5-m82rk" event={"ID":"360e93a9-d4c3-463f-9f5a-859e2d222e96","Type":"ContainerStarted","Data":"a7428e84888301b1884431aafeef4e2d6a83a67f0ca340ef43d48e62e45d1d16"}
Jul 29 14:11:00.055993 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:00.055958    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-968d69cc5-m82rk" event={"ID":"360e93a9-d4c3-463f-9f5a-859e2d222e96","Type":"ContainerStarted","Data":"269d721c32141d3ef7fc6f18848f34525bb5cdb13944ff91ce3e23333ceb1c75"}
Jul 29 14:11:00.056403 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:00.056051    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-968d69cc5-m82rk"
Jul 29 14:11:00.076563 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:00.076500    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-968d69cc5-m82rk" podStartSLOduration=1.179958325 podStartE2EDuration="4.076485891s" totalImagesPullingTime="2.896527566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:10:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:11:00.074969951 +0000 UTC m=+589.965641286" watchObservedRunningTime="2026-07-29 14:11:00.076485891 +0000 UTC m=+589.967157247"
Jul 29 14:11:00.899232 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:00.899177    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:11:00.999773 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:00.999704    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:11:06.066765 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:06.066729    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-968d69cc5-m82rk"
Jul 29 14:11:07.363485 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:07.363448    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:11:07.465687 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:07.465648    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:11:10.564203 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:10.564175    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdw8l_b93f6faa-c8bf-43e9-94e0-b3dcd97b7296/ovn-acl-logging/0.log"
Jul 29 14:11:10.573475 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:10.573452    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdw8l_b93f6faa-c8bf-43e9-94e0-b3dcd97b7296/ovn-acl-logging/0.log"
Jul 29 14:11:15.956687 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:15.956652    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw"]
Jul 29 14:11:15.960940 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:15.960920    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw"
Jul 29 14:11:15.963868 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:15.963841    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-nccq4\""
Jul 29 14:11:15.963868 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:15.963865    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 29 14:11:15.964023 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:15.963884    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 29 14:11:15.965165 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:15.965149    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 29 14:11:15.970295 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:15.970274    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw"]
Jul 29 14:11:16.013970 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:16.013932    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:11:16.044123 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:16.044090    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33d14efa-fc8e-473a-8252-18f6308f0514-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw\" (UID: \"33d14efa-fc8e-473a-8252-18f6308f0514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw"
Jul 29 14:11:16.044123 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:16.044125    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33d14efa-fc8e-473a-8252-18f6308f0514-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw\" (UID: \"33d14efa-fc8e-473a-8252-18f6308f0514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw"
Jul 29 14:11:16.044281 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:16.044156    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33d14efa-fc8e-473a-8252-18f6308f0514-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw\" (UID: \"33d14efa-fc8e-473a-8252-18f6308f0514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw"
Jul 29 14:11:16.044281 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:16.044207    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33d14efa-fc8e-473a-8252-18f6308f0514-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw\" (UID: \"33d14efa-fc8e-473a-8252-18f6308f0514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw"
Jul 29 14:11:16.044281 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:16.044242    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33d14efa-fc8e-473a-8252-18f6308f0514-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw\" (UID: \"33d14efa-fc8e-473a-8252-18f6308f0514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw"
Jul 29 14:11:16.044281 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:16.044272    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhcwm\" (UniqueName: \"kubernetes.io/projected/33d14efa-fc8e-473a-8252-18f6308f0514-kube-api-access-bhcwm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw\" (UID: \"33d14efa-fc8e-473a-8252-18f6308f0514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw"
Jul 29 14:11:16.144803 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:16.144763    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33d14efa-fc8e-473a-8252-18f6308f0514-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw\" (UID: \"33d14efa-fc8e-473a-8252-18f6308f0514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw"
Jul 29 14:11:16.144995 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:16.144813    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33d14efa-fc8e-473a-8252-18f6308f0514-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw\" (UID: \"33d14efa-fc8e-473a-8252-18f6308f0514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw"
Jul 29 14:11:16.144995 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:16.144846    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33d14efa-fc8e-473a-8252-18f6308f0514-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw\" (UID: \"33d14efa-fc8e-473a-8252-18f6308f0514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw"
Jul 29 14:11:16.144995 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:16.144879    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhcwm\" (UniqueName: \"kubernetes.io/projected/33d14efa-fc8e-473a-8252-18f6308f0514-kube-api-access-bhcwm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw\" (UID: \"33d14efa-fc8e-473a-8252-18f6308f0514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw"
Jul 29 14:11:16.144995 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:16.144945    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33d14efa-fc8e-473a-8252-18f6308f0514-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw\" (UID: \"33d14efa-fc8e-473a-8252-18f6308f0514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw"
Jul 29 14:11:16.144995 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:16.144966    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33d14efa-fc8e-473a-8252-18f6308f0514-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw\" (UID: \"33d14efa-fc8e-473a-8252-18f6308f0514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw"
Jul 29 14:11:16.145354 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:16.145321    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33d14efa-fc8e-473a-8252-18f6308f0514-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw\" (UID: \"33d14efa-fc8e-473a-8252-18f6308f0514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw"
Jul 29 14:11:16.145491 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:16.145443    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33d14efa-fc8e-473a-8252-18f6308f0514-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw\" (UID: \"33d14efa-fc8e-473a-8252-18f6308f0514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw"
Jul 29 14:11:16.145543 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:16.145528    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33d14efa-fc8e-473a-8252-18f6308f0514-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw\" (UID: \"33d14efa-fc8e-473a-8252-18f6308f0514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw"
Jul 29 14:11:16.147214 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:16.147197    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33d14efa-fc8e-473a-8252-18f6308f0514-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw\" (UID: \"33d14efa-fc8e-473a-8252-18f6308f0514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw"
Jul 29 14:11:16.147376 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:16.147358    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33d14efa-fc8e-473a-8252-18f6308f0514-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw\" (UID: \"33d14efa-fc8e-473a-8252-18f6308f0514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw"
Jul 29 14:11:16.154028 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:16.154006    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhcwm\" (UniqueName: \"kubernetes.io/projected/33d14efa-fc8e-473a-8252-18f6308f0514-kube-api-access-bhcwm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw\" (UID: \"33d14efa-fc8e-473a-8252-18f6308f0514\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw"
Jul 29 14:11:16.271676 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:16.271623    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw"
Jul 29 14:11:16.400978 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:16.400916    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw"]
Jul 29 14:11:16.403240 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:11:16.403203    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33d14efa_fc8e_473a_8252_18f6308f0514.slice/crio-da2155292765cb52e5dddccbf3a6c8d4d04d0737041a1e63c7bdc8fb90a979aa WatchSource:0}: Error finding container da2155292765cb52e5dddccbf3a6c8d4d04d0737041a1e63c7bdc8fb90a979aa: Status 404 returned error can't find the container with id da2155292765cb52e5dddccbf3a6c8d4d04d0737041a1e63c7bdc8fb90a979aa
Jul 29 14:11:17.125521 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:17.125485    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw" event={"ID":"33d14efa-fc8e-473a-8252-18f6308f0514","Type":"ContainerStarted","Data":"da2155292765cb52e5dddccbf3a6c8d4d04d0737041a1e63c7bdc8fb90a979aa"}
Jul 29 14:11:18.131163 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:18.131119    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw" event={"ID":"33d14efa-fc8e-473a-8252-18f6308f0514","Type":"ContainerStarted","Data":"f761e54ef9a455e943a0982c70e0568e7f39c57ae24a6b05d5f7bdb655f12e8b"}
Jul 29 14:11:18.149176 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:18.149130    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw" podStartSLOduration=2.053382412 podStartE2EDuration="3.149115903s" totalImagesPullingTime="1.095733491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:11:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:11:18.147811087 +0000 UTC m=+608.038482423" watchObservedRunningTime="2026-07-29 14:11:18.149115903 +0000 UTC m=+608.039787280"
Jul 29 14:11:25.171286 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:25.171233    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:11:26.272610 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:26.272574    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw"
Jul 29 14:11:26.272988 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:26.272670    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw"
Jul 29 14:11:26.285135 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:26.285117    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw"
Jul 29 14:11:27.179638 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:27.179603    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw"
Jul 29 14:11:35.991703 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:35.991613    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:11:51.794978 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:51.794940    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:11:57.875127 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:11:57.875088    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:12:18.929489 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:18.929450    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:12:29.051668 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:29.051638    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-r2qs7"]
Jul 29 14:12:29.054878 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:29.054860    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r2qs7"
Jul 29 14:12:29.057976 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:29.057954    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 29 14:12:29.065643 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:29.065618    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-r2qs7"]
Jul 29 14:12:29.143765 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:29.143729    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:12:29.250685 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:29.250651    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9df8bffa-4c17-4e30-8770-38ca42f4e2c3-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r2qs7\" (UID: \"9df8bffa-4c17-4e30-8770-38ca42f4e2c3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r2qs7"
Jul 29 14:12:29.250685 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:29.250688    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9df8bffa-4c17-4e30-8770-38ca42f4e2c3-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r2qs7\" (UID: \"9df8bffa-4c17-4e30-8770-38ca42f4e2c3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r2qs7"
Jul 29 14:12:29.250934 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:29.250719    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9df8bffa-4c17-4e30-8770-38ca42f4e2c3-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r2qs7\" (UID: \"9df8bffa-4c17-4e30-8770-38ca42f4e2c3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r2qs7"
Jul 29 14:12:29.250934 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:29.250819    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9df8bffa-4c17-4e30-8770-38ca42f4e2c3-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r2qs7\" (UID: \"9df8bffa-4c17-4e30-8770-38ca42f4e2c3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r2qs7"
Jul 29 14:12:29.250934 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:29.250871    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9df8bffa-4c17-4e30-8770-38ca42f4e2c3-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r2qs7\" (UID: \"9df8bffa-4c17-4e30-8770-38ca42f4e2c3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r2qs7"
Jul 29 14:12:29.250934 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:29.250928    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gb6tk\" (UniqueName: \"kubernetes.io/projected/9df8bffa-4c17-4e30-8770-38ca42f4e2c3-kube-api-access-gb6tk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r2qs7\" (UID: \"9df8bffa-4c17-4e30-8770-38ca42f4e2c3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r2qs7"
Jul 29 14:12:29.352085 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:29.351981    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9df8bffa-4c17-4e30-8770-38ca42f4e2c3-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r2qs7\" (UID: \"9df8bffa-4c17-4e30-8770-38ca42f4e2c3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r2qs7"
Jul 29 14:12:29.352085 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:29.352026    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9df8bffa-4c17-4e30-8770-38ca42f4e2c3-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r2qs7\" (UID: \"9df8bffa-4c17-4e30-8770-38ca42f4e2c3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r2qs7"
Jul 29 14:12:29.352085 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:29.352073    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9df8bffa-4c17-4e30-8770-38ca42f4e2c3-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r2qs7\" (UID: \"9df8bffa-4c17-4e30-8770-38ca42f4e2c3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r2qs7"
Jul 29 14:12:29.352357 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:29.352104    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb6tk\" (UniqueName: \"kubernetes.io/projected/9df8bffa-4c17-4e30-8770-38ca42f4e2c3-kube-api-access-gb6tk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r2qs7\" (UID: \"9df8bffa-4c17-4e30-8770-38ca42f4e2c3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r2qs7"
Jul 29 14:12:29.352357 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:29.352166    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9df8bffa-4c17-4e30-8770-38ca42f4e2c3-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r2qs7\" (UID: \"9df8bffa-4c17-4e30-8770-38ca42f4e2c3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r2qs7"
Jul 29 14:12:29.352357 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:29.352194    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9df8bffa-4c17-4e30-8770-38ca42f4e2c3-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r2qs7\" (UID: \"9df8bffa-4c17-4e30-8770-38ca42f4e2c3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r2qs7"
Jul 29 14:12:29.352537 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:29.352512    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9df8bffa-4c17-4e30-8770-38ca42f4e2c3-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r2qs7\" (UID: \"9df8bffa-4c17-4e30-8770-38ca42f4e2c3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r2qs7"
Jul 29 14:12:29.352585 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:29.352540    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9df8bffa-4c17-4e30-8770-38ca42f4e2c3-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r2qs7\" (UID: \"9df8bffa-4c17-4e30-8770-38ca42f4e2c3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r2qs7"
Jul 29 14:12:29.352585 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:29.352576    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9df8bffa-4c17-4e30-8770-38ca42f4e2c3-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r2qs7\" (UID: \"9df8bffa-4c17-4e30-8770-38ca42f4e2c3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r2qs7"
Jul 29 14:12:29.354428 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:29.354403    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9df8bffa-4c17-4e30-8770-38ca42f4e2c3-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r2qs7\" (UID: \"9df8bffa-4c17-4e30-8770-38ca42f4e2c3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r2qs7"
Jul 29 14:12:29.354579 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:29.354558    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9df8bffa-4c17-4e30-8770-38ca42f4e2c3-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r2qs7\" (UID: \"9df8bffa-4c17-4e30-8770-38ca42f4e2c3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r2qs7"
Jul 29 14:12:29.360951 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:29.360928    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gb6tk\" (UniqueName: \"kubernetes.io/projected/9df8bffa-4c17-4e30-8770-38ca42f4e2c3-kube-api-access-gb6tk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-r2qs7\" (UID: \"9df8bffa-4c17-4e30-8770-38ca42f4e2c3\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r2qs7"
Jul 29 14:12:29.364611 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:29.364586    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r2qs7"
Jul 29 14:12:29.500262 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:29.500235    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-r2qs7"]
Jul 29 14:12:29.502840 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:12:29.502812    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9df8bffa_4c17_4e30_8770_38ca42f4e2c3.slice/crio-e1ef00f7108c36df53085645196c09283e5c881681fa144dea89f8f49a0029b2 WatchSource:0}: Error finding container e1ef00f7108c36df53085645196c09283e5c881681fa144dea89f8f49a0029b2: Status 404 returned error can't find the container with id e1ef00f7108c36df53085645196c09283e5c881681fa144dea89f8f49a0029b2
Jul 29 14:12:29.504119 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:29.504100    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 14:12:30.424501 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:30.424410    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r2qs7" event={"ID":"9df8bffa-4c17-4e30-8770-38ca42f4e2c3","Type":"ContainerStarted","Data":"da3862e8c4ee6b2ed3ddf02229b71d9d015eb6afb35c5d25ed31ae75a12e7909"}
Jul 29 14:12:30.424501 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:30.424475    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r2qs7" event={"ID":"9df8bffa-4c17-4e30-8770-38ca42f4e2c3","Type":"ContainerStarted","Data":"e1ef00f7108c36df53085645196c09283e5c881681fa144dea89f8f49a0029b2"}
Jul 29 14:12:30.443927 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:30.443879    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r2qs7" podStartSLOduration=0.952849646 podStartE2EDuration="1.443860669s" totalImagesPullingTime="491.011023ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:12:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:12:30.442841362 +0000 UTC m=+680.333512696" watchObservedRunningTime="2026-07-29 14:12:30.443860669 +0000 UTC m=+680.334532005"
Jul 29 14:12:36.046672 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:36.046635    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7"]
Jul 29 14:12:36.048845 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:36.048821    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7"
Jul 29 14:12:36.052505 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:36.052486    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 29 14:12:36.065634 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:36.065607    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7"]
Jul 29 14:12:36.087533 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:36.087506    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:12:36.103659 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:36.103631    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fd71469-d266-469f-b7f6-972bec639c9d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7\" (UID: \"6fd71469-d266-469f-b7f6-972bec639c9d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7"
Jul 29 14:12:36.103753 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:36.103672    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bz4x\" (UniqueName: \"kubernetes.io/projected/6fd71469-d266-469f-b7f6-972bec639c9d-kube-api-access-7bz4x\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7\" (UID: \"6fd71469-d266-469f-b7f6-972bec639c9d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7"
Jul 29 14:12:36.103753 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:36.103717    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6fd71469-d266-469f-b7f6-972bec639c9d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7\" (UID: \"6fd71469-d266-469f-b7f6-972bec639c9d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7"
Jul 29 14:12:36.103826 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:36.103769    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6fd71469-d266-469f-b7f6-972bec639c9d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7\" (UID: \"6fd71469-d266-469f-b7f6-972bec639c9d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7"
Jul 29 14:12:36.103826 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:36.103792    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fd71469-d266-469f-b7f6-972bec639c9d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7\" (UID: \"6fd71469-d266-469f-b7f6-972bec639c9d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7"
Jul 29 14:12:36.103826 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:36.103811    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6fd71469-d266-469f-b7f6-972bec639c9d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7\" (UID: \"6fd71469-d266-469f-b7f6-972bec639c9d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7"
Jul 29 14:12:36.204677 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:36.204644    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fd71469-d266-469f-b7f6-972bec639c9d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7\" (UID: \"6fd71469-d266-469f-b7f6-972bec639c9d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7"
Jul 29 14:12:36.204864 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:36.204685    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6fd71469-d266-469f-b7f6-972bec639c9d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7\" (UID: \"6fd71469-d266-469f-b7f6-972bec639c9d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7"
Jul 29 14:12:36.204864 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:36.204740    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fd71469-d266-469f-b7f6-972bec639c9d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7\" (UID: \"6fd71469-d266-469f-b7f6-972bec639c9d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7"
Jul 29 14:12:36.204864 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:36.204761    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bz4x\" (UniqueName: \"kubernetes.io/projected/6fd71469-d266-469f-b7f6-972bec639c9d-kube-api-access-7bz4x\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7\" (UID: \"6fd71469-d266-469f-b7f6-972bec639c9d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7"
Jul 29 14:12:36.204864 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:36.204796    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6fd71469-d266-469f-b7f6-972bec639c9d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7\" (UID: \"6fd71469-d266-469f-b7f6-972bec639c9d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7"
Jul 29 14:12:36.204864 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:36.204817    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6fd71469-d266-469f-b7f6-972bec639c9d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7\" (UID: \"6fd71469-d266-469f-b7f6-972bec639c9d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7"
Jul 29 14:12:36.205229 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:36.205201    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6fd71469-d266-469f-b7f6-972bec639c9d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7\" (UID: \"6fd71469-d266-469f-b7f6-972bec639c9d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7"
Jul 29 14:12:36.205303 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:36.205254    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fd71469-d266-469f-b7f6-972bec639c9d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7\" (UID: \"6fd71469-d266-469f-b7f6-972bec639c9d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7"
Jul 29 14:12:36.205303 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:36.205277    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6fd71469-d266-469f-b7f6-972bec639c9d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7\" (UID: \"6fd71469-d266-469f-b7f6-972bec639c9d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7"
Jul 29 14:12:36.207126 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:36.207093    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6fd71469-d266-469f-b7f6-972bec639c9d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7\" (UID: \"6fd71469-d266-469f-b7f6-972bec639c9d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7"
Jul 29 14:12:36.207808 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:36.207787    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fd71469-d266-469f-b7f6-972bec639c9d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7\" (UID: \"6fd71469-d266-469f-b7f6-972bec639c9d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7"
Jul 29 14:12:36.212733 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:36.212711    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bz4x\" (UniqueName: \"kubernetes.io/projected/6fd71469-d266-469f-b7f6-972bec639c9d-kube-api-access-7bz4x\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7\" (UID: \"6fd71469-d266-469f-b7f6-972bec639c9d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7"
Jul 29 14:12:36.365759 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:36.365672    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7"
Jul 29 14:12:36.491048 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:36.491026    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7"]
Jul 29 14:12:36.493322 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:12:36.493286    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6fd71469_d266_469f_b7f6_972bec639c9d.slice/crio-993f5ca7ad726c177515cc608062b7369a148b95a5c1f18d3810588dbe950a37 WatchSource:0}: Error finding container 993f5ca7ad726c177515cc608062b7369a148b95a5c1f18d3810588dbe950a37: Status 404 returned error can't find the container with id 993f5ca7ad726c177515cc608062b7369a148b95a5c1f18d3810588dbe950a37
Jul 29 14:12:37.456397 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:37.456354    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7" event={"ID":"6fd71469-d266-469f-b7f6-972bec639c9d","Type":"ContainerStarted","Data":"9012c624fb653cb9e75f5829d490273f9917a83fd2aa44edced612e1dc68b53b"}
Jul 29 14:12:37.456397 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:37.456402    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7" event={"ID":"6fd71469-d266-469f-b7f6-972bec639c9d","Type":"ContainerStarted","Data":"993f5ca7ad726c177515cc608062b7369a148b95a5c1f18d3810588dbe950a37"}
Jul 29 14:12:37.475995 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:37.475947    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7" podStartSLOduration=1.254256572 podStartE2EDuration="1.475932274s" totalImagesPullingTime="221.675702ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:12:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:12:37.474331074 +0000 UTC m=+687.365002408" watchObservedRunningTime="2026-07-29 14:12:37.475932274 +0000 UTC m=+687.366603609"
Jul 29 14:12:39.365134 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:39.365091    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r2qs7"
Jul 29 14:12:39.365134 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:39.365156    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r2qs7"
Jul 29 14:12:39.377544 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:39.377519    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r2qs7"
Jul 29 14:12:39.475983 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:39.475957    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-r2qs7"
Jul 29 14:12:46.366051 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:46.366013    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7"
Jul 29 14:12:46.366646 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:46.366082    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7"
Jul 29 14:12:46.378352 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:46.378326    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7"
Jul 29 14:12:46.505581 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:12:46.505548    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7"
Jul 29 14:13:08.715934 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:13:08.715835    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:13:13.957324 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:13:13.957284    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:13:14.104900 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:13:14.104866    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:13:15.085810 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:13:15.085768    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:13:15.189106 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:13:15.189043    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:15:24.498201 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:15:24.498168    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:15:25.732120 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:15:25.732087    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:16:01.334409 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:16:01.334374    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:16:01.434416 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:16:01.434382    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:16:01.570970 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:16:01.570935    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:16:07.225796 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:16:07.225713    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:16:07.331721 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:16:07.331689    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:16:07.427154 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:16:07.427113    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:16:07.580577 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:16:07.580535    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:16:10.597245 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:16:10.597217    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdw8l_b93f6faa-c8bf-43e9-94e0-b3dcd97b7296/ovn-acl-logging/0.log"
Jul 29 14:16:10.597942 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:16:10.597920    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdw8l_b93f6faa-c8bf-43e9-94e0-b3dcd97b7296/ovn-acl-logging/0.log"
Jul 29 14:16:13.380829 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:16:13.380794    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:16:13.481962 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:16:13.481930    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:17:12.893168 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:17:12.893113    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:17:15.724624 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:17:15.724590    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:18:07.833532 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:18:07.833439    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:18:07.935258 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:18:07.935220    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:21:08.193448 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:21:08.193401    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:21:10.622317 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:21:10.622290    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdw8l_b93f6faa-c8bf-43e9-94e0-b3dcd97b7296/ovn-acl-logging/0.log"
Jul 29 14:21:10.623867 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:21:10.623845    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdw8l_b93f6faa-c8bf-43e9-94e0-b3dcd97b7296/ovn-acl-logging/0.log"
Jul 29 14:23:04.963110 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:23:04.963052    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:24:01.638946 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:01.638864    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:24:13.857261 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:13.857228    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v"]
Jul 29 14:24:13.860398 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:13.860377    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v"
Jul 29 14:24:13.863072 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:13.863038    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-d531b1ac-kserve-self-signed-certs\""
Jul 29 14:24:13.876086 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:13.876043    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v"]
Jul 29 14:24:13.912400 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:13.912372    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-dshm\") pod \"e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v\" (UID: \"8cfdfb85-2456-41bb-9fe7-eebdb5213f99\") " pod="llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v"
Jul 29 14:24:13.912536 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:13.912404    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-home\") pod \"e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v\" (UID: \"8cfdfb85-2456-41bb-9fe7-eebdb5213f99\") " pod="llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v"
Jul 29 14:24:13.912536 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:13.912434    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-model-cache\") pod \"e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v\" (UID: \"8cfdfb85-2456-41bb-9fe7-eebdb5213f99\") " pod="llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v"
Jul 29 14:24:13.912536 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:13.912521    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r4hc\" (UniqueName: \"kubernetes.io/projected/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-kube-api-access-5r4hc\") pod \"e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v\" (UID: \"8cfdfb85-2456-41bb-9fe7-eebdb5213f99\") " pod="llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v"
Jul 29 14:24:13.912657 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:13.912556    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-tls-certs\") pod \"e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v\" (UID: \"8cfdfb85-2456-41bb-9fe7-eebdb5213f99\") " pod="llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v"
Jul 29 14:24:13.912657 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:13.912596    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-tmp-dir\") pod \"e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v\" (UID: \"8cfdfb85-2456-41bb-9fe7-eebdb5213f99\") " pod="llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v"
Jul 29 14:24:13.972414 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:13.972383    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:24:14.013156 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:14.013126    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-tmp-dir\") pod \"e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v\" (UID: \"8cfdfb85-2456-41bb-9fe7-eebdb5213f99\") " pod="llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v"
Jul 29 14:24:14.013269 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:14.013162    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-dshm\") pod \"e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v\" (UID: \"8cfdfb85-2456-41bb-9fe7-eebdb5213f99\") " pod="llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v"
Jul 29 14:24:14.013269 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:14.013182    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-home\") pod \"e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v\" (UID: \"8cfdfb85-2456-41bb-9fe7-eebdb5213f99\") " pod="llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v"
Jul 29 14:24:14.013363 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:14.013264    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-model-cache\") pod \"e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v\" (UID: \"8cfdfb85-2456-41bb-9fe7-eebdb5213f99\") " pod="llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v"
Jul 29 14:24:14.013363 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:14.013319    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r4hc\" (UniqueName: \"kubernetes.io/projected/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-kube-api-access-5r4hc\") pod \"e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v\" (UID: \"8cfdfb85-2456-41bb-9fe7-eebdb5213f99\") " pod="llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v"
Jul 29 14:24:14.013363 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:14.013360    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-tls-certs\") pod \"e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v\" (UID: \"8cfdfb85-2456-41bb-9fe7-eebdb5213f99\") " pod="llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v"
Jul 29 14:24:14.013528 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:14.013488    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-tmp-dir\") pod \"e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v\" (UID: \"8cfdfb85-2456-41bb-9fe7-eebdb5213f99\") " pod="llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v"
Jul 29 14:24:14.013528 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:14.013519    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-home\") pod \"e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v\" (UID: \"8cfdfb85-2456-41bb-9fe7-eebdb5213f99\") " pod="llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v"
Jul 29 14:24:14.013671 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:14.013650    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-model-cache\") pod \"e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v\" (UID: \"8cfdfb85-2456-41bb-9fe7-eebdb5213f99\") " pod="llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v"
Jul 29 14:24:14.015478 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:14.015463    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-dshm\") pod \"e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v\" (UID: \"8cfdfb85-2456-41bb-9fe7-eebdb5213f99\") " pod="llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v"
Jul 29 14:24:14.015740 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:14.015718    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-tls-certs\") pod \"e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v\" (UID: \"8cfdfb85-2456-41bb-9fe7-eebdb5213f99\") " pod="llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v"
Jul 29 14:24:14.022112 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:14.022086    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r4hc\" (UniqueName: \"kubernetes.io/projected/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-kube-api-access-5r4hc\") pod \"e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v\" (UID: \"8cfdfb85-2456-41bb-9fe7-eebdb5213f99\") " pod="llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v"
Jul 29 14:24:14.169949 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:14.169861    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v"
Jul 29 14:24:14.302993 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:14.302957    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v"]
Jul 29 14:24:14.323187 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:14.323160    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v" event={"ID":"8cfdfb85-2456-41bb-9fe7-eebdb5213f99","Type":"ContainerStarted","Data":"24a4e1a989b6df542d42d5884904916b3d760f83e0380d26a0cfdf685bf307b2"}
Jul 29 14:24:15.328541 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:15.328497    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v" event={"ID":"8cfdfb85-2456-41bb-9fe7-eebdb5213f99","Type":"ContainerStarted","Data":"d94eb28fcec72143e6defb0208008dd0ed2dea05e0dc7be04de7aece36ba4bba"}
Jul 29 14:24:15.345116 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:15.345049    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v" podStartSLOduration=2.345031942 podStartE2EDuration="2.345031942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:24:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:24:15.34443085 +0000 UTC m=+1385.235102185" watchObservedRunningTime="2026-07-29 14:24:15.345031942 +0000 UTC m=+1385.235703286"
Jul 29 14:24:16.859634 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:16.859597    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv"]
Jul 29 14:24:16.862895 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:16.862877    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv"
Jul 29 14:24:16.866032 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:16.866014    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-d531b1ac-kserve-self-signed-certs\""
Jul 29 14:24:16.880959 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:16.880932    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv"]
Jul 29 14:24:16.923088 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:16.923000    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:24:16.937408 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:16.937382    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/441fcbd8-c7ee-4171-9fed-c9a7674c166f-model-cache\") pod \"e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv\" (UID: \"441fcbd8-c7ee-4171-9fed-c9a7674c166f\") " pod="llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv"
Jul 29 14:24:16.937514 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:16.937412    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/441fcbd8-c7ee-4171-9fed-c9a7674c166f-dshm\") pod \"e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv\" (UID: \"441fcbd8-c7ee-4171-9fed-c9a7674c166f\") " pod="llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv"
Jul 29 14:24:16.937514 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:16.937436    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/441fcbd8-c7ee-4171-9fed-c9a7674c166f-home\") pod \"e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv\" (UID: \"441fcbd8-c7ee-4171-9fed-c9a7674c166f\") " pod="llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv"
Jul 29 14:24:16.937514 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:16.937462    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/441fcbd8-c7ee-4171-9fed-c9a7674c166f-tmp-dir\") pod \"e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv\" (UID: \"441fcbd8-c7ee-4171-9fed-c9a7674c166f\") " pod="llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv"
Jul 29 14:24:16.937514 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:16.937494    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfxkt\" (UniqueName: \"kubernetes.io/projected/441fcbd8-c7ee-4171-9fed-c9a7674c166f-kube-api-access-wfxkt\") pod \"e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv\" (UID: \"441fcbd8-c7ee-4171-9fed-c9a7674c166f\") " pod="llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv"
Jul 29 14:24:16.937652 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:16.937525    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/441fcbd8-c7ee-4171-9fed-c9a7674c166f-tls-certs\") pod \"e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv\" (UID: \"441fcbd8-c7ee-4171-9fed-c9a7674c166f\") " pod="llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv"
Jul 29 14:24:17.038480 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:17.038434    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/441fcbd8-c7ee-4171-9fed-c9a7674c166f-dshm\") pod \"e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv\" (UID: \"441fcbd8-c7ee-4171-9fed-c9a7674c166f\") " pod="llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv"
Jul 29 14:24:17.038480 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:17.038488    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/441fcbd8-c7ee-4171-9fed-c9a7674c166f-home\") pod \"e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv\" (UID: \"441fcbd8-c7ee-4171-9fed-c9a7674c166f\") " pod="llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv"
Jul 29 14:24:17.038707 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:17.038523    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/441fcbd8-c7ee-4171-9fed-c9a7674c166f-tmp-dir\") pod \"e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv\" (UID: \"441fcbd8-c7ee-4171-9fed-c9a7674c166f\") " pod="llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv"
Jul 29 14:24:17.038707 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:17.038543    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfxkt\" (UniqueName: \"kubernetes.io/projected/441fcbd8-c7ee-4171-9fed-c9a7674c166f-kube-api-access-wfxkt\") pod \"e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv\" (UID: \"441fcbd8-c7ee-4171-9fed-c9a7674c166f\") " pod="llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv"
Jul 29 14:24:17.038707 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:17.038577    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/441fcbd8-c7ee-4171-9fed-c9a7674c166f-tls-certs\") pod \"e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv\" (UID: \"441fcbd8-c7ee-4171-9fed-c9a7674c166f\") " pod="llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv"
Jul 29 14:24:17.038707 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:17.038634    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/441fcbd8-c7ee-4171-9fed-c9a7674c166f-model-cache\") pod \"e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv\" (UID: \"441fcbd8-c7ee-4171-9fed-c9a7674c166f\") " pod="llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv"
Jul 29 14:24:17.038957 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:17.038934    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/441fcbd8-c7ee-4171-9fed-c9a7674c166f-home\") pod \"e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv\" (UID: \"441fcbd8-c7ee-4171-9fed-c9a7674c166f\") " pod="llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv"
Jul 29 14:24:17.039023 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:17.038958    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/441fcbd8-c7ee-4171-9fed-c9a7674c166f-model-cache\") pod \"e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv\" (UID: \"441fcbd8-c7ee-4171-9fed-c9a7674c166f\") " pod="llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv"
Jul 29 14:24:17.039023 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:17.039000    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/441fcbd8-c7ee-4171-9fed-c9a7674c166f-tmp-dir\") pod \"e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv\" (UID: \"441fcbd8-c7ee-4171-9fed-c9a7674c166f\") " pod="llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv"
Jul 29 14:24:17.040630 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:17.040612    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/441fcbd8-c7ee-4171-9fed-c9a7674c166f-dshm\") pod \"e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv\" (UID: \"441fcbd8-c7ee-4171-9fed-c9a7674c166f\") " pod="llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv"
Jul 29 14:24:17.041449 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:17.041431    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/441fcbd8-c7ee-4171-9fed-c9a7674c166f-tls-certs\") pod \"e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv\" (UID: \"441fcbd8-c7ee-4171-9fed-c9a7674c166f\") " pod="llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv"
Jul 29 14:24:17.056407 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:17.056379    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfxkt\" (UniqueName: \"kubernetes.io/projected/441fcbd8-c7ee-4171-9fed-c9a7674c166f-kube-api-access-wfxkt\") pod \"e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv\" (UID: \"441fcbd8-c7ee-4171-9fed-c9a7674c166f\") " pod="llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv"
Jul 29 14:24:17.172169 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:17.172082    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv"
Jul 29 14:24:17.299970 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:17.299935    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv"]
Jul 29 14:24:17.303011 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:24:17.302984    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod441fcbd8_c7ee_4171_9fed_c9a7674c166f.slice/crio-9f25e42e80e4b585d3ecb8ac6529535cb3dc77ced7cb4baf1d3d2bcfafa0e980 WatchSource:0}: Error finding container 9f25e42e80e4b585d3ecb8ac6529535cb3dc77ced7cb4baf1d3d2bcfafa0e980: Status 404 returned error can't find the container with id 9f25e42e80e4b585d3ecb8ac6529535cb3dc77ced7cb4baf1d3d2bcfafa0e980
Jul 29 14:24:17.338378 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:17.338353    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv" event={"ID":"441fcbd8-c7ee-4171-9fed-c9a7674c166f","Type":"ContainerStarted","Data":"9f25e42e80e4b585d3ecb8ac6529535cb3dc77ced7cb4baf1d3d2bcfafa0e980"}
Jul 29 14:24:18.343975 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:18.343938    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv" event={"ID":"441fcbd8-c7ee-4171-9fed-c9a7674c166f","Type":"ContainerStarted","Data":"34196abb8a1cabd6b54034bd13c0463f5ca9b3c75b020f88b5c9c0e47579ea5a"}
Jul 29 14:24:18.362342 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:18.362292    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv" podStartSLOduration=2.362276838 podStartE2EDuration="2.362276838s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:24:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:24:18.361253679 +0000 UTC m=+1388.251925015" watchObservedRunningTime="2026-07-29 14:24:18.362276838 +0000 UTC m=+1388.252948204"
Jul 29 14:24:24.170809 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:24.170703    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v"
Jul 29 14:24:24.170809 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:24.170768    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v"
Jul 29 14:24:24.183455 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:24.183430    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v"
Jul 29 14:24:24.382372 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:24.382337    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v"
Jul 29 14:24:27.172700 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:27.172659    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv"
Jul 29 14:24:27.173204 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:27.172716    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv"
Jul 29 14:24:27.185440 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:27.185415    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv"
Jul 29 14:24:27.394130 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:27.394102    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv"
Jul 29 14:24:57.334852 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:57.334819    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v"]
Jul 29 14:24:57.436482 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:57.436449    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:24:58.470137 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:58.470106    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv"]
Jul 29 14:24:58.494627 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:24:58.494597    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:25:12.353206 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:12.353140    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v" podUID="8cfdfb85-2456-41bb-9fe7-eebdb5213f99" containerName="main" containerID="cri-o://d94eb28fcec72143e6defb0208008dd0ed2dea05e0dc7be04de7aece36ba4bba" gracePeriod=45
Jul 29 14:25:12.578634 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:12.578605    2559 generic.go:356] "Generic (PLEG): container finished" podID="8cfdfb85-2456-41bb-9fe7-eebdb5213f99" containerID="d94eb28fcec72143e6defb0208008dd0ed2dea05e0dc7be04de7aece36ba4bba" exitCode=0
Jul 29 14:25:12.578762 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:12.578653    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v" event={"ID":"8cfdfb85-2456-41bb-9fe7-eebdb5213f99","Type":"ContainerDied","Data":"d94eb28fcec72143e6defb0208008dd0ed2dea05e0dc7be04de7aece36ba4bba"}
Jul 29 14:25:12.594372 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:12.594349    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v"
Jul 29 14:25:12.690333 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:12.690241    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-home\") pod \"8cfdfb85-2456-41bb-9fe7-eebdb5213f99\" (UID: \"8cfdfb85-2456-41bb-9fe7-eebdb5213f99\") "
Jul 29 14:25:12.690333 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:12.690298    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5r4hc\" (UniqueName: \"kubernetes.io/projected/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-kube-api-access-5r4hc\") pod \"8cfdfb85-2456-41bb-9fe7-eebdb5213f99\" (UID: \"8cfdfb85-2456-41bb-9fe7-eebdb5213f99\") "
Jul 29 14:25:12.690333 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:12.690317    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-model-cache\") pod \"8cfdfb85-2456-41bb-9fe7-eebdb5213f99\" (UID: \"8cfdfb85-2456-41bb-9fe7-eebdb5213f99\") "
Jul 29 14:25:12.690561 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:12.690379    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-tmp-dir\") pod \"8cfdfb85-2456-41bb-9fe7-eebdb5213f99\" (UID: \"8cfdfb85-2456-41bb-9fe7-eebdb5213f99\") "
Jul 29 14:25:12.690561 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:12.690414    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-tls-certs\") pod \"8cfdfb85-2456-41bb-9fe7-eebdb5213f99\" (UID: \"8cfdfb85-2456-41bb-9fe7-eebdb5213f99\") "
Jul 29 14:25:12.690561 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:12.690448    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-dshm\") pod \"8cfdfb85-2456-41bb-9fe7-eebdb5213f99\" (UID: \"8cfdfb85-2456-41bb-9fe7-eebdb5213f99\") "
Jul 29 14:25:12.690693 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:12.690677    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-model-cache" (OuterVolumeSpecName: "model-cache") pod "8cfdfb85-2456-41bb-9fe7-eebdb5213f99" (UID: "8cfdfb85-2456-41bb-9fe7-eebdb5213f99"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:25:12.690760 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:12.690692    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8cfdfb85-2456-41bb-9fe7-eebdb5213f99" (UID: "8cfdfb85-2456-41bb-9fe7-eebdb5213f99"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:25:12.690760 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:12.690620    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-home" (OuterVolumeSpecName: "home") pod "8cfdfb85-2456-41bb-9fe7-eebdb5213f99" (UID: "8cfdfb85-2456-41bb-9fe7-eebdb5213f99"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:25:12.692748 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:12.692688    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-dshm" (OuterVolumeSpecName: "dshm") pod "8cfdfb85-2456-41bb-9fe7-eebdb5213f99" (UID: "8cfdfb85-2456-41bb-9fe7-eebdb5213f99"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:25:12.692748 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:12.692717    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8cfdfb85-2456-41bb-9fe7-eebdb5213f99" (UID: "8cfdfb85-2456-41bb-9fe7-eebdb5213f99"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 14:25:12.692748 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:12.692740    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-kube-api-access-5r4hc" (OuterVolumeSpecName: "kube-api-access-5r4hc") pod "8cfdfb85-2456-41bb-9fe7-eebdb5213f99" (UID: "8cfdfb85-2456-41bb-9fe7-eebdb5213f99"). InnerVolumeSpecName "kube-api-access-5r4hc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:25:12.791343 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:12.791309    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5r4hc\" (UniqueName: \"kubernetes.io/projected/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-kube-api-access-5r4hc\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:25:12.791343 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:12.791339    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-model-cache\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:25:12.791343 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:12.791351    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-tmp-dir\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:25:12.791564 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:12.791359    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-tls-certs\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:25:12.791564 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:12.791367    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-dshm\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:25:12.791564 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:12.791374    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8cfdfb85-2456-41bb-9fe7-eebdb5213f99-home\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:25:13.488466 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:13.488394    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv" podUID="441fcbd8-c7ee-4171-9fed-c9a7674c166f" containerName="main" containerID="cri-o://34196abb8a1cabd6b54034bd13c0463f5ca9b3c75b020f88b5c9c0e47579ea5a" gracePeriod=45
Jul 29 14:25:13.584257 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:13.584232    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v"
Jul 29 14:25:13.584383 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:13.584237    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v" event={"ID":"8cfdfb85-2456-41bb-9fe7-eebdb5213f99","Type":"ContainerDied","Data":"24a4e1a989b6df542d42d5884904916b3d760f83e0380d26a0cfdf685bf307b2"}
Jul 29 14:25:13.584383 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:13.584349    2559 scope.go:117] "RemoveContainer" containerID="d94eb28fcec72143e6defb0208008dd0ed2dea05e0dc7be04de7aece36ba4bba"
Jul 29 14:25:13.605294 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:13.605272    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v"]
Jul 29 14:25:13.607121 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:13.607097    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-d531b1ac-kserve-556b8b95c5-7sb8v"]
Jul 29 14:25:13.741965 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:13.741935    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv"
Jul 29 14:25:13.800338 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:13.800310    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/441fcbd8-c7ee-4171-9fed-c9a7674c166f-model-cache\") pod \"441fcbd8-c7ee-4171-9fed-c9a7674c166f\" (UID: \"441fcbd8-c7ee-4171-9fed-c9a7674c166f\") "
Jul 29 14:25:13.800477 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:13.800346    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/441fcbd8-c7ee-4171-9fed-c9a7674c166f-dshm\") pod \"441fcbd8-c7ee-4171-9fed-c9a7674c166f\" (UID: \"441fcbd8-c7ee-4171-9fed-c9a7674c166f\") "
Jul 29 14:25:13.800477 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:13.800377    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfxkt\" (UniqueName: \"kubernetes.io/projected/441fcbd8-c7ee-4171-9fed-c9a7674c166f-kube-api-access-wfxkt\") pod \"441fcbd8-c7ee-4171-9fed-c9a7674c166f\" (UID: \"441fcbd8-c7ee-4171-9fed-c9a7674c166f\") "
Jul 29 14:25:13.800477 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:13.800413    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/441fcbd8-c7ee-4171-9fed-c9a7674c166f-tls-certs\") pod \"441fcbd8-c7ee-4171-9fed-c9a7674c166f\" (UID: \"441fcbd8-c7ee-4171-9fed-c9a7674c166f\") "
Jul 29 14:25:13.800477 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:13.800450    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/441fcbd8-c7ee-4171-9fed-c9a7674c166f-home\") pod \"441fcbd8-c7ee-4171-9fed-c9a7674c166f\" (UID: \"441fcbd8-c7ee-4171-9fed-c9a7674c166f\") "
Jul 29 14:25:13.800701 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:13.800550    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/441fcbd8-c7ee-4171-9fed-c9a7674c166f-tmp-dir\") pod \"441fcbd8-c7ee-4171-9fed-c9a7674c166f\" (UID: \"441fcbd8-c7ee-4171-9fed-c9a7674c166f\") "
Jul 29 14:25:13.800701 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:13.800621    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/441fcbd8-c7ee-4171-9fed-c9a7674c166f-model-cache" (OuterVolumeSpecName: "model-cache") pod "441fcbd8-c7ee-4171-9fed-c9a7674c166f" (UID: "441fcbd8-c7ee-4171-9fed-c9a7674c166f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:25:13.800813 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:13.800760    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/441fcbd8-c7ee-4171-9fed-c9a7674c166f-home" (OuterVolumeSpecName: "home") pod "441fcbd8-c7ee-4171-9fed-c9a7674c166f" (UID: "441fcbd8-c7ee-4171-9fed-c9a7674c166f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:25:13.800813 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:13.800780    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/441fcbd8-c7ee-4171-9fed-c9a7674c166f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "441fcbd8-c7ee-4171-9fed-c9a7674c166f" (UID: "441fcbd8-c7ee-4171-9fed-c9a7674c166f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:25:13.800813 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:13.800799    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/441fcbd8-c7ee-4171-9fed-c9a7674c166f-model-cache\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:25:13.802440 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:13.802415    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/441fcbd8-c7ee-4171-9fed-c9a7674c166f-dshm" (OuterVolumeSpecName: "dshm") pod "441fcbd8-c7ee-4171-9fed-c9a7674c166f" (UID: "441fcbd8-c7ee-4171-9fed-c9a7674c166f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:25:13.802535 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:13.802437    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/441fcbd8-c7ee-4171-9fed-c9a7674c166f-kube-api-access-wfxkt" (OuterVolumeSpecName: "kube-api-access-wfxkt") pod "441fcbd8-c7ee-4171-9fed-c9a7674c166f" (UID: "441fcbd8-c7ee-4171-9fed-c9a7674c166f"). InnerVolumeSpecName "kube-api-access-wfxkt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:25:13.802535 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:13.802505    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/441fcbd8-c7ee-4171-9fed-c9a7674c166f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "441fcbd8-c7ee-4171-9fed-c9a7674c166f" (UID: "441fcbd8-c7ee-4171-9fed-c9a7674c166f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 14:25:13.902121 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:13.902087    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/441fcbd8-c7ee-4171-9fed-c9a7674c166f-tmp-dir\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:25:13.902121 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:13.902114    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/441fcbd8-c7ee-4171-9fed-c9a7674c166f-dshm\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:25:13.902121 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:13.902123    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfxkt\" (UniqueName: \"kubernetes.io/projected/441fcbd8-c7ee-4171-9fed-c9a7674c166f-kube-api-access-wfxkt\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:25:13.902323 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:13.902132    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/441fcbd8-c7ee-4171-9fed-c9a7674c166f-tls-certs\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:25:13.902323 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:13.902141    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/441fcbd8-c7ee-4171-9fed-c9a7674c166f-home\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:25:14.590511 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:14.590476    2559 generic.go:356] "Generic (PLEG): container finished" podID="441fcbd8-c7ee-4171-9fed-c9a7674c166f" containerID="34196abb8a1cabd6b54034bd13c0463f5ca9b3c75b020f88b5c9c0e47579ea5a" exitCode=0
Jul 29 14:25:14.590916 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:14.590519    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv"
Jul 29 14:25:14.590916 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:14.590530    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv" event={"ID":"441fcbd8-c7ee-4171-9fed-c9a7674c166f","Type":"ContainerDied","Data":"34196abb8a1cabd6b54034bd13c0463f5ca9b3c75b020f88b5c9c0e47579ea5a"}
Jul 29 14:25:14.590916 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:14.590570    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv" event={"ID":"441fcbd8-c7ee-4171-9fed-c9a7674c166f","Type":"ContainerDied","Data":"9f25e42e80e4b585d3ecb8ac6529535cb3dc77ced7cb4baf1d3d2bcfafa0e980"}
Jul 29 14:25:14.590916 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:14.590590    2559 scope.go:117] "RemoveContainer" containerID="34196abb8a1cabd6b54034bd13c0463f5ca9b3c75b020f88b5c9c0e47579ea5a"
Jul 29 14:25:14.599477 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:14.599459    2559 scope.go:117] "RemoveContainer" containerID="34196abb8a1cabd6b54034bd13c0463f5ca9b3c75b020f88b5c9c0e47579ea5a"
Jul 29 14:25:14.599756 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:25:14.599735    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34196abb8a1cabd6b54034bd13c0463f5ca9b3c75b020f88b5c9c0e47579ea5a\": container with ID starting with 34196abb8a1cabd6b54034bd13c0463f5ca9b3c75b020f88b5c9c0e47579ea5a not found: ID does not exist" containerID="34196abb8a1cabd6b54034bd13c0463f5ca9b3c75b020f88b5c9c0e47579ea5a"
Jul 29 14:25:14.599820 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:14.599765    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34196abb8a1cabd6b54034bd13c0463f5ca9b3c75b020f88b5c9c0e47579ea5a"} err="failed to get container status \"34196abb8a1cabd6b54034bd13c0463f5ca9b3c75b020f88b5c9c0e47579ea5a\": rpc error: code = NotFound desc = could not find container \"34196abb8a1cabd6b54034bd13c0463f5ca9b3c75b020f88b5c9c0e47579ea5a\": container with ID starting with 34196abb8a1cabd6b54034bd13c0463f5ca9b3c75b020f88b5c9c0e47579ea5a not found: ID does not exist"
Jul 29 14:25:14.612945 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:14.612921    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv"]
Jul 29 14:25:14.616986 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:14.616963    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-d531b1ac-kserve-75cfb4fb7f-fvrgv"]
Jul 29 14:25:14.635973 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:14.635942    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="441fcbd8-c7ee-4171-9fed-c9a7674c166f" path="/var/lib/kubelet/pods/441fcbd8-c7ee-4171-9fed-c9a7674c166f/volumes"
Jul 29 14:25:14.636390 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:25:14.636375    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cfdfb85-2456-41bb-9fe7-eebdb5213f99" path="/var/lib/kubelet/pods/8cfdfb85-2456-41bb-9fe7-eebdb5213f99/volumes"
Jul 29 14:26:10.647185 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:26:10.647148    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdw8l_b93f6faa-c8bf-43e9-94e0-b3dcd97b7296/ovn-acl-logging/0.log"
Jul 29 14:26:10.651417 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:26:10.651396    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdw8l_b93f6faa-c8bf-43e9-94e0-b3dcd97b7296/ovn-acl-logging/0.log"
Jul 29 14:26:34.492511 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:26:34.492402    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:26:34.579753 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:26:34.579723    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:28:24.902878 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:24.902844    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:28:40.330045 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:40.330013    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:28:46.602198 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:46.602156    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:28:49.684309 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:49.684270    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:28:49.780801 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:49.780769    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:28:50.147339 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:50.147293    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:28:55.931860 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:55.931823    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:28:55.999010 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:55.998975    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:28:56.153245 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:56.153211    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:28:56.473929 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:56.473889    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:28:56.857459 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:56.857423    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:28:58.397280 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:58.397251    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:28:58.492790 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:58.492756    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s"]
Jul 29 14:28:58.493097 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:58.493075    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="441fcbd8-c7ee-4171-9fed-c9a7674c166f" containerName="main"
Jul 29 14:28:58.493097 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:58.493089    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="441fcbd8-c7ee-4171-9fed-c9a7674c166f" containerName="main"
Jul 29 14:28:58.493178 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:58.493107    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cfdfb85-2456-41bb-9fe7-eebdb5213f99" containerName="main"
Jul 29 14:28:58.493178 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:58.493113    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cfdfb85-2456-41bb-9fe7-eebdb5213f99" containerName="main"
Jul 29 14:28:58.493178 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:58.493157    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="441fcbd8-c7ee-4171-9fed-c9a7674c166f" containerName="main"
Jul 29 14:28:58.493178 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:58.493166    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cfdfb85-2456-41bb-9fe7-eebdb5213f99" containerName="main"
Jul 29 14:28:58.495921 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:58.495901    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s"
Jul 29 14:28:58.500762 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:58.500734    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-d8b7adbd-serving-cert\""
Jul 29 14:28:58.500856 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:58.500775    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7jqfw\""
Jul 29 14:28:58.513661 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:58.513635    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s"]
Jul 29 14:28:58.596526 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:58.596482    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jph8x\" (UniqueName: \"kubernetes.io/projected/1c652359-c971-479d-9b55-93d41fa537ef-kube-api-access-jph8x\") pod \"maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s\" (UID: \"1c652359-c971-479d-9b55-93d41fa537ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s"
Jul 29 14:28:58.596879 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:58.596530    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c652359-c971-479d-9b55-93d41fa537ef-openshift-service-ca\") pod \"maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s\" (UID: \"1c652359-c971-479d-9b55-93d41fa537ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s"
Jul 29 14:28:58.596879 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:58.596604    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c652359-c971-479d-9b55-93d41fa537ef-maas-api-tls\") pod \"maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s\" (UID: \"1c652359-c971-479d-9b55-93d41fa537ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s"
Jul 29 14:28:58.697721 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:58.697641    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c652359-c971-479d-9b55-93d41fa537ef-maas-api-tls\") pod \"maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s\" (UID: \"1c652359-c971-479d-9b55-93d41fa537ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s"
Jul 29 14:28:58.697884 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:58.697822    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jph8x\" (UniqueName: \"kubernetes.io/projected/1c652359-c971-479d-9b55-93d41fa537ef-kube-api-access-jph8x\") pod \"maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s\" (UID: \"1c652359-c971-479d-9b55-93d41fa537ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s"
Jul 29 14:28:58.697884 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:58.697875    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c652359-c971-479d-9b55-93d41fa537ef-openshift-service-ca\") pod \"maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s\" (UID: \"1c652359-c971-479d-9b55-93d41fa537ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s"
Jul 29 14:28:58.698544 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:58.698516    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c652359-c971-479d-9b55-93d41fa537ef-openshift-service-ca\") pod \"maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s\" (UID: \"1c652359-c971-479d-9b55-93d41fa537ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s"
Jul 29 14:28:58.700068 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:58.700034    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c652359-c971-479d-9b55-93d41fa537ef-maas-api-tls\") pod \"maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s\" (UID: \"1c652359-c971-479d-9b55-93d41fa537ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s"
Jul 29 14:28:58.707497 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:58.707477    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jph8x\" (UniqueName: \"kubernetes.io/projected/1c652359-c971-479d-9b55-93d41fa537ef-kube-api-access-jph8x\") pod \"maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s\" (UID: \"1c652359-c971-479d-9b55-93d41fa537ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s"
Jul 29 14:28:58.728592 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:58.728572    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:28:58.806891 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:58.806866    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s"
Jul 29 14:28:58.831980 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:58.831949    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:28:58.956452 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:58.956330    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s"]
Jul 29 14:28:58.959115 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:28:58.959088    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c652359_c971_479d_9b55_93d41fa537ef.slice/crio-1bd6d4e46db897ea522f1257a33e5942b317b858df538ee3e34d33435ad7227f WatchSource:0}: Error finding container 1bd6d4e46db897ea522f1257a33e5942b317b858df538ee3e34d33435ad7227f: Status 404 returned error can't find the container with id 1bd6d4e46db897ea522f1257a33e5942b317b858df538ee3e34d33435ad7227f
Jul 29 14:28:58.960380 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:58.960363    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 14:28:59.473379 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:59.473344    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:28:59.519394 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:59.519364    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s" event={"ID":"1c652359-c971-479d-9b55-93d41fa537ef","Type":"ContainerStarted","Data":"1bd6d4e46db897ea522f1257a33e5942b317b858df538ee3e34d33435ad7227f"}
Jul 29 14:28:59.607676 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:28:59.607646    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:29:00.525263 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:00.525223    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s" event={"ID":"1c652359-c971-479d-9b55-93d41fa537ef","Type":"ContainerStarted","Data":"c15c311fdebabed101a81be0810d11434e225a5c868b089ac5ced146fc6b2077"}
Jul 29 14:29:00.525655 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:00.525329    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s"
Jul 29 14:29:00.544504 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:00.544447    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s" podStartSLOduration=1.156422263 podStartE2EDuration="2.544430842s" totalImagesPullingTime="1.388008579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:28:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:29:00.543110714 +0000 UTC m=+1670.433782052" watchObservedRunningTime="2026-07-29 14:29:00.544430842 +0000 UTC m=+1670.435102176"
Jul 29 14:29:06.534981 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:06.534948    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s"
Jul 29 14:29:29.144226 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:29.144185    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s"]
Jul 29 14:29:29.144729 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:29.144443    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s" podUID="1c652359-c971-479d-9b55-93d41fa537ef" containerName="maas-api" containerID="cri-o://c15c311fdebabed101a81be0810d11434e225a5c868b089ac5ced146fc6b2077" gracePeriod=30
Jul 29 14:29:29.380848 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:29.380819    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s"
Jul 29 14:29:29.446274 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:29.446195    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jph8x\" (UniqueName: \"kubernetes.io/projected/1c652359-c971-479d-9b55-93d41fa537ef-kube-api-access-jph8x\") pod \"1c652359-c971-479d-9b55-93d41fa537ef\" (UID: \"1c652359-c971-479d-9b55-93d41fa537ef\") "
Jul 29 14:29:29.446274 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:29.446232    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c652359-c971-479d-9b55-93d41fa537ef-maas-api-tls\") pod \"1c652359-c971-479d-9b55-93d41fa537ef\" (UID: \"1c652359-c971-479d-9b55-93d41fa537ef\") "
Jul 29 14:29:29.446458 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:29.446314    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c652359-c971-479d-9b55-93d41fa537ef-openshift-service-ca\") pod \"1c652359-c971-479d-9b55-93d41fa537ef\" (UID: \"1c652359-c971-479d-9b55-93d41fa537ef\") "
Jul 29 14:29:29.446703 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:29.446679    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c652359-c971-479d-9b55-93d41fa537ef-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1c652359-c971-479d-9b55-93d41fa537ef" (UID: "1c652359-c971-479d-9b55-93d41fa537ef"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 14:29:29.448426 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:29.448403    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c652359-c971-479d-9b55-93d41fa537ef-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1c652359-c971-479d-9b55-93d41fa537ef" (UID: "1c652359-c971-479d-9b55-93d41fa537ef"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 14:29:29.448572 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:29.448454    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c652359-c971-479d-9b55-93d41fa537ef-kube-api-access-jph8x" (OuterVolumeSpecName: "kube-api-access-jph8x") pod "1c652359-c971-479d-9b55-93d41fa537ef" (UID: "1c652359-c971-479d-9b55-93d41fa537ef"). InnerVolumeSpecName "kube-api-access-jph8x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:29:29.546958 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:29.546919    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c652359-c971-479d-9b55-93d41fa537ef-openshift-service-ca\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:29:29.546958 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:29.546945    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jph8x\" (UniqueName: \"kubernetes.io/projected/1c652359-c971-479d-9b55-93d41fa537ef-kube-api-access-jph8x\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:29:29.546958 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:29.546955    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c652359-c971-479d-9b55-93d41fa537ef-maas-api-tls\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:29:29.652679 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:29.652644    2559 generic.go:356] "Generic (PLEG): container finished" podID="1c652359-c971-479d-9b55-93d41fa537ef" containerID="c15c311fdebabed101a81be0810d11434e225a5c868b089ac5ced146fc6b2077" exitCode=0
Jul 29 14:29:29.652679 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:29.652681    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s"
Jul 29 14:29:29.652889 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:29.652683    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s" event={"ID":"1c652359-c971-479d-9b55-93d41fa537ef","Type":"ContainerDied","Data":"c15c311fdebabed101a81be0810d11434e225a5c868b089ac5ced146fc6b2077"}
Jul 29 14:29:29.652889 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:29.652818    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s" event={"ID":"1c652359-c971-479d-9b55-93d41fa537ef","Type":"ContainerDied","Data":"1bd6d4e46db897ea522f1257a33e5942b317b858df538ee3e34d33435ad7227f"}
Jul 29 14:29:29.652889 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:29.652849    2559 scope.go:117] "RemoveContainer" containerID="c15c311fdebabed101a81be0810d11434e225a5c868b089ac5ced146fc6b2077"
Jul 29 14:29:29.661731 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:29.661716    2559 scope.go:117] "RemoveContainer" containerID="c15c311fdebabed101a81be0810d11434e225a5c868b089ac5ced146fc6b2077"
Jul 29 14:29:29.661970 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:29:29.661951    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c15c311fdebabed101a81be0810d11434e225a5c868b089ac5ced146fc6b2077\": container with ID starting with c15c311fdebabed101a81be0810d11434e225a5c868b089ac5ced146fc6b2077 not found: ID does not exist" containerID="c15c311fdebabed101a81be0810d11434e225a5c868b089ac5ced146fc6b2077"
Jul 29 14:29:29.662004 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:29.661978    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c15c311fdebabed101a81be0810d11434e225a5c868b089ac5ced146fc6b2077"} err="failed to get container status \"c15c311fdebabed101a81be0810d11434e225a5c868b089ac5ced146fc6b2077\": rpc error: code = NotFound desc = could not find container \"c15c311fdebabed101a81be0810d11434e225a5c868b089ac5ced146fc6b2077\": container with ID starting with c15c311fdebabed101a81be0810d11434e225a5c868b089ac5ced146fc6b2077 not found: ID does not exist"
Jul 29 14:29:29.674146 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:29.674124    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s"]
Jul 29 14:29:29.678029 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:29.678010    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d8b7adbd-7b7cdc5c5b-rbv8s"]
Jul 29 14:29:30.483535 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.483500    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:29:30.537784 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.537748    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"]
Jul 29 14:29:30.538127 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.538113    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c652359-c971-479d-9b55-93d41fa537ef" containerName="maas-api"
Jul 29 14:29:30.538176 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.538128    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c652359-c971-479d-9b55-93d41fa537ef" containerName="maas-api"
Jul 29 14:29:30.538210 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.538178    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c652359-c971-479d-9b55-93d41fa537ef" containerName="maas-api"
Jul 29 14:29:30.540363 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.540346    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:30.543473 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.543453    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-59408d2e-gw-openshift-default-dockercfg-4nf5g\""
Jul 29 14:29:30.552933 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.552912    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"]
Jul 29 14:29:30.601300 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.601269    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:29:30.636509 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.636474    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c652359-c971-479d-9b55-93d41fa537ef" path="/var/lib/kubelet/pods/1c652359-c971-479d-9b55-93d41fa537ef/volumes"
Jul 29 14:29:30.657132 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.657091    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/852d18ff-69b0-4479-b76e-2290013cb49c-istiod-ca-cert\") pod \"e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") " pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:30.657272 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.657168    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/852d18ff-69b0-4479-b76e-2290013cb49c-istio-token\") pod \"e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") " pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:30.657272 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.657209    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/852d18ff-69b0-4479-b76e-2290013cb49c-istio-data\") pod \"e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") " pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:30.657272 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.657259    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/852d18ff-69b0-4479-b76e-2290013cb49c-credential-socket\") pod \"e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") " pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:30.657432 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.657301    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/852d18ff-69b0-4479-b76e-2290013cb49c-workload-socket\") pod \"e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") " pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:30.657432 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.657348    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/852d18ff-69b0-4479-b76e-2290013cb49c-istio-podinfo\") pod \"e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") " pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:30.657432 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.657378    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6s6h\" (UniqueName: \"kubernetes.io/projected/852d18ff-69b0-4479-b76e-2290013cb49c-kube-api-access-d6s6h\") pod \"e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") " pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:30.657569 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.657443    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/852d18ff-69b0-4479-b76e-2290013cb49c-workload-certs\") pod \"e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") " pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:30.657569 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.657476    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/852d18ff-69b0-4479-b76e-2290013cb49c-istio-envoy\") pod \"e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") " pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:30.737912 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.737831    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:29:30.758039 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.758017    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/852d18ff-69b0-4479-b76e-2290013cb49c-workload-certs\") pod \"e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") " pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:30.758166 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.758047    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/852d18ff-69b0-4479-b76e-2290013cb49c-istio-envoy\") pod \"e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") " pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:30.758166 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.758092    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/852d18ff-69b0-4479-b76e-2290013cb49c-istiod-ca-cert\") pod \"e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") " pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:30.758166 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.758130    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/852d18ff-69b0-4479-b76e-2290013cb49c-istio-token\") pod \"e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") " pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:30.758166 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.758150    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/852d18ff-69b0-4479-b76e-2290013cb49c-istio-data\") pod \"e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") " pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:30.758369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.758182    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/852d18ff-69b0-4479-b76e-2290013cb49c-credential-socket\") pod \"e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") " pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:30.758369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.758215    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/852d18ff-69b0-4479-b76e-2290013cb49c-workload-socket\") pod \"e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") " pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:30.758369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.758255    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/852d18ff-69b0-4479-b76e-2290013cb49c-istio-podinfo\") pod \"e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") " pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:30.758369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.758279    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6s6h\" (UniqueName: \"kubernetes.io/projected/852d18ff-69b0-4479-b76e-2290013cb49c-kube-api-access-d6s6h\") pod \"e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") " pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:30.758611 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.758575    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/852d18ff-69b0-4479-b76e-2290013cb49c-workload-certs\") pod \"e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") " pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:30.758746 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.758631    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/852d18ff-69b0-4479-b76e-2290013cb49c-credential-socket\") pod \"e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") " pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:30.758746 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.758662    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/852d18ff-69b0-4479-b76e-2290013cb49c-istio-data\") pod \"e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") " pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:30.758746 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.758735    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/852d18ff-69b0-4479-b76e-2290013cb49c-workload-socket\") pod \"e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") " pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:30.759100 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.759080    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/852d18ff-69b0-4479-b76e-2290013cb49c-istiod-ca-cert\") pod \"e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") " pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:30.760405 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.760381    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/852d18ff-69b0-4479-b76e-2290013cb49c-istio-envoy\") pod \"e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") " pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:30.760748 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.760726    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/852d18ff-69b0-4479-b76e-2290013cb49c-istio-podinfo\") pod \"e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") " pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:30.785475 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.785445    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/852d18ff-69b0-4479-b76e-2290013cb49c-istio-token\") pod \"e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") " pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:30.785623 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.785489    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6s6h\" (UniqueName: \"kubernetes.io/projected/852d18ff-69b0-4479-b76e-2290013cb49c-kube-api-access-d6s6h\") pod \"e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") " pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:30.850658 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.850621    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:30.984394 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.984371    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"]
Jul 29 14:29:30.986168 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:29:30.986142    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod852d18ff_69b0_4479_b76e_2290013cb49c.slice/crio-ec9d2b0a3531f0a3a3d11037a6d92984097a2be7c58d3848f30858eb567b8f08 WatchSource:0}: Error finding container ec9d2b0a3531f0a3a3d11037a6d92984097a2be7c58d3848f30858eb567b8f08: Status 404 returned error can't find the container with id ec9d2b0a3531f0a3a3d11037a6d92984097a2be7c58d3848f30858eb567b8f08
Jul 29 14:29:30.988080 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.988006    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:29:30.988159 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.988085    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:29:30.988159 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:30.988115    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:29:31.665708 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:31.665672    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7" event={"ID":"852d18ff-69b0-4479-b76e-2290013cb49c","Type":"ContainerStarted","Data":"b548944e5613808d52501ee6a8a483850fcca5d917df7895abdb5704db486358"}
Jul 29 14:29:31.666195 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:31.665722    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7" event={"ID":"852d18ff-69b0-4479-b76e-2290013cb49c","Type":"ContainerStarted","Data":"ec9d2b0a3531f0a3a3d11037a6d92984097a2be7c58d3848f30858eb567b8f08"}
Jul 29 14:29:31.690351 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:31.690304    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7" podStartSLOduration=1.690289894 podStartE2EDuration="1.690289894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:29:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:29:31.68889479 +0000 UTC m=+1701.579566124" watchObservedRunningTime="2026-07-29 14:29:31.690289894 +0000 UTC m=+1701.580961229"
Jul 29 14:29:31.851760 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:31.851728    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:32.855798 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:32.855718    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:33.230572 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:33.230499    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4"]
Jul 29 14:29:33.232685 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:33.232666    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4"
Jul 29 14:29:33.235762 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:33.235737    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-59408d2e-serving-cert\""
Jul 29 14:29:33.236092 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:33.236074    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7jqfw\""
Jul 29 14:29:33.245193 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:33.245164    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:29:33.246646 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:33.246625    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4"]
Jul 29 14:29:33.279830 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:33.279793    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef-openshift-service-ca\") pod \"maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4\" (UID: \"b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4"
Jul 29 14:29:33.279971 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:33.279848    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef-maas-api-tls\") pod \"maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4\" (UID: \"b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4"
Jul 29 14:29:33.279971 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:33.279934    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-525kz\" (UniqueName: \"kubernetes.io/projected/b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef-kube-api-access-525kz\") pod \"maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4\" (UID: \"b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4"
Jul 29 14:29:33.339750 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:33.339710    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:29:33.381000 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:33.380960    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef-maas-api-tls\") pod \"maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4\" (UID: \"b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4"
Jul 29 14:29:33.381172 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:33.381023    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-525kz\" (UniqueName: \"kubernetes.io/projected/b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef-kube-api-access-525kz\") pod \"maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4\" (UID: \"b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4"
Jul 29 14:29:33.381172 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:33.381109    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef-openshift-service-ca\") pod \"maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4\" (UID: \"b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4"
Jul 29 14:29:33.381172 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:29:33.381145    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-59408d2e-serving-cert: secret "maas-api-e2e-migrate-59408d2e-serving-cert" not found
Jul 29 14:29:33.381299 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:29:33.381209    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef-maas-api-tls podName:b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef nodeName:}" failed. No retries permitted until 2026-07-29 14:29:33.881189719 +0000 UTC m=+1703.771861035 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef-maas-api-tls") pod "maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4" (UID: "b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef") : secret "maas-api-e2e-migrate-59408d2e-serving-cert" not found
Jul 29 14:29:33.381761 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:33.381737    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef-openshift-service-ca\") pod \"maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4\" (UID: \"b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4"
Jul 29 14:29:33.395141 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:33.395116    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-525kz\" (UniqueName: \"kubernetes.io/projected/b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef-kube-api-access-525kz\") pod \"maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4\" (UID: \"b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4"
Jul 29 14:29:33.492692 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:33.492655    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:29:33.674737 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:33.674705    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:33.675856 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:33.675820    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:33.804718 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:33.804640    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:29:33.886520 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:33.886478    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef-maas-api-tls\") pod \"maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4\" (UID: \"b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4"
Jul 29 14:29:33.888988 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:33.888965    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef-maas-api-tls\") pod \"maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4\" (UID: \"b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4"
Jul 29 14:29:33.965841 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:33.965812    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:29:34.143437 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:34.143357    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4"
Jul 29 14:29:34.286963 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:34.286939    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4"]
Jul 29 14:29:34.288775 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:29:34.288743    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb664aaed_8cbd_4ccd_b6d7_108c91b8c3ef.slice/crio-ada41aa4c8b858f34b354d287bb654b896d1423dab9ffb77b746bdfc5b3f7816 WatchSource:0}: Error finding container ada41aa4c8b858f34b354d287bb654b896d1423dab9ffb77b746bdfc5b3f7816: Status 404 returned error can't find the container with id ada41aa4c8b858f34b354d287bb654b896d1423dab9ffb77b746bdfc5b3f7816
Jul 29 14:29:34.679360 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:34.679322    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4" event={"ID":"b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef","Type":"ContainerStarted","Data":"ada41aa4c8b858f34b354d287bb654b896d1423dab9ffb77b746bdfc5b3f7816"}
Jul 29 14:29:35.687442 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:35.687394    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4" event={"ID":"b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef","Type":"ContainerStarted","Data":"15c41948b6778d922bdc4dfcdde2f86f26f63610ed951a4f13f8019db976fd56"}
Jul 29 14:29:35.687848 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:35.687482    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4"
Jul 29 14:29:35.716527 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:35.716475    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4" podStartSLOduration=2.209328547 podStartE2EDuration="2.716460579s" totalImagesPullingTime="507.132032ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:29:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:29:35.716005333 +0000 UTC m=+1705.606676668" watchObservedRunningTime="2026-07-29 14:29:35.716460579 +0000 UTC m=+1705.607131914"
Jul 29 14:29:41.697416 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:41.697386    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4"
Jul 29 14:29:47.743998 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:47.743968    2559 generic.go:356] "Generic (PLEG): container finished" podID="852d18ff-69b0-4479-b76e-2290013cb49c" containerID="b548944e5613808d52501ee6a8a483850fcca5d917df7895abdb5704db486358" exitCode=0
Jul 29 14:29:47.744427 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:47.744047    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7" event={"ID":"852d18ff-69b0-4479-b76e-2290013cb49c","Type":"ContainerDied","Data":"b548944e5613808d52501ee6a8a483850fcca5d917df7895abdb5704db486358"}
Jul 29 14:29:47.744468 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:47.744457    2559 scope.go:117] "RemoveContainer" containerID="b548944e5613808d52501ee6a8a483850fcca5d917df7895abdb5704db486358"
Jul 29 14:29:47.745474 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:47.745448    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:29:47.745526 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:47.745513    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:29:47.745561 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:47.745544    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:29:48.675499 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:48.675462    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:48.750439 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:48.750406    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7" event={"ID":"852d18ff-69b0-4479-b76e-2290013cb49c","Type":"ContainerStarted","Data":"4a58af619fd7b44a9ff2b7f18dad415cb699998500e29819b892ad4f91e7207b"}
Jul 29 14:29:48.851156 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:48.851126    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:48.852168 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:48.852136    2559 patch_prober.go:28] interesting pod/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Jul 29 14:29:48.852357 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:48.852323    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7" podUID="852d18ff-69b0-4479-b76e-2290013cb49c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Jul 29 14:29:49.851691 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:49.851653    2559 patch_prober.go:28] interesting pod/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Jul 29 14:29:49.852322 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:49.852292    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7" podUID="852d18ff-69b0-4479-b76e-2290013cb49c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Jul 29 14:29:50.857536 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:50.857492    2559 patch_prober.go:28] interesting pod/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Jul 29 14:29:50.858029 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:50.857557    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7" podUID="852d18ff-69b0-4479-b76e-2290013cb49c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Jul 29 14:29:51.851519 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:51.851478    2559 patch_prober.go:28] interesting pod/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Jul 29 14:29:51.851826 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:51.851795    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7" podUID="852d18ff-69b0-4479-b76e-2290013cb49c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Jul 29 14:29:52.855941 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:52.855912    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:52.856651 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:52.856330    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:29:52.856940 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:29:52.856918    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:30:03.873692 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:03.873657    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4"]
Jul 29 14:30:03.874411 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:03.873915    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4" podUID="b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef" containerName="maas-api" containerID="cri-o://15c41948b6778d922bdc4dfcdde2f86f26f63610ed951a4f13f8019db976fd56" gracePeriod=30
Jul 29 14:30:03.941779 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:03.941743    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:30:04.042216 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:04.042185    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:30:04.111212 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:04.111175    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:30:04.120788 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:04.120755    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4"
Jul 29 14:30:04.247611 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:04.247574    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"]
Jul 29 14:30:04.247874 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:04.247847    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7" podUID="852d18ff-69b0-4479-b76e-2290013cb49c" containerName="istio-proxy" containerID="cri-o://4a58af619fd7b44a9ff2b7f18dad415cb699998500e29819b892ad4f91e7207b" gracePeriod=30
Jul 29 14:30:04.260030 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:04.260004    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef-maas-api-tls\") pod \"b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef\" (UID: \"b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef\") "
Jul 29 14:30:04.260208 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:04.260075    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef-openshift-service-ca\") pod \"b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef\" (UID: \"b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef\") "
Jul 29 14:30:04.260208 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:04.260129    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-525kz\" (UniqueName: \"kubernetes.io/projected/b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef-kube-api-access-525kz\") pod \"b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef\" (UID: \"b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef\") "
Jul 29 14:30:04.260459 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:04.260429    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef" (UID: "b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 14:30:04.262197 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:04.262170    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef" (UID: "b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 14:30:04.262302 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:04.262269    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef-kube-api-access-525kz" (OuterVolumeSpecName: "kube-api-access-525kz") pod "b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef" (UID: "b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef"). InnerVolumeSpecName "kube-api-access-525kz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:30:04.275783 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:04.275758    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:30:04.360824 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:04.360795    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef-maas-api-tls\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:30:04.360824 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:04.360823    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef-openshift-service-ca\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:30:04.360824 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:04.360833    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-525kz\" (UniqueName: \"kubernetes.io/projected/b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef-kube-api-access-525kz\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:30:04.490772 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:04.490733    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:30:04.725131 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:04.725095    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:30:04.829542 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:04.829506    2559 generic.go:356] "Generic (PLEG): container finished" podID="b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef" containerID="15c41948b6778d922bdc4dfcdde2f86f26f63610ed951a4f13f8019db976fd56" exitCode=0
Jul 29 14:30:04.829704 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:04.829563    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4"
Jul 29 14:30:04.829704 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:04.829589    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4" event={"ID":"b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef","Type":"ContainerDied","Data":"15c41948b6778d922bdc4dfcdde2f86f26f63610ed951a4f13f8019db976fd56"}
Jul 29 14:30:04.829704 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:04.829647    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4" event={"ID":"b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef","Type":"ContainerDied","Data":"ada41aa4c8b858f34b354d287bb654b896d1423dab9ffb77b746bdfc5b3f7816"}
Jul 29 14:30:04.829704 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:04.829673    2559 scope.go:117] "RemoveContainer" containerID="15c41948b6778d922bdc4dfcdde2f86f26f63610ed951a4f13f8019db976fd56"
Jul 29 14:30:04.837843 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:04.837824    2559 scope.go:117] "RemoveContainer" containerID="15c41948b6778d922bdc4dfcdde2f86f26f63610ed951a4f13f8019db976fd56"
Jul 29 14:30:04.838134 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:30:04.838111    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15c41948b6778d922bdc4dfcdde2f86f26f63610ed951a4f13f8019db976fd56\": container with ID starting with 15c41948b6778d922bdc4dfcdde2f86f26f63610ed951a4f13f8019db976fd56 not found: ID does not exist" containerID="15c41948b6778d922bdc4dfcdde2f86f26f63610ed951a4f13f8019db976fd56"
Jul 29 14:30:04.838239 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:04.838143    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15c41948b6778d922bdc4dfcdde2f86f26f63610ed951a4f13f8019db976fd56"} err="failed to get container status \"15c41948b6778d922bdc4dfcdde2f86f26f63610ed951a4f13f8019db976fd56\": rpc error: code = NotFound desc = could not find container \"15c41948b6778d922bdc4dfcdde2f86f26f63610ed951a4f13f8019db976fd56\": container with ID starting with 15c41948b6778d922bdc4dfcdde2f86f26f63610ed951a4f13f8019db976fd56 not found: ID does not exist"
Jul 29 14:30:04.847409 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:04.847385    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4"]
Jul 29 14:30:04.851766 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:04.851722    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-59408d2e-d7bb4cfbd-4rlq4"]
Jul 29 14:30:05.163163 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:05.163130    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:30:05.428254 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:05.428173    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:30:06.636271 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:06.636238    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef" path="/var/lib/kubelet/pods/b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef/volumes"
Jul 29 14:30:07.855726 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:07.855694    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:30:07.858443 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:07.858416    2559 patch_prober.go:28] interesting pod/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 29 14:30:07.858546 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:07.858467    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7" podUID="852d18ff-69b0-4479-b76e-2290013cb49c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 14:30:07.933937 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:07.933904    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl"]
Jul 29 14:30:07.934274 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:07.934258    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef" containerName="maas-api"
Jul 29 14:30:07.934274 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:07.934275    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef" containerName="maas-api"
Jul 29 14:30:07.934428 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:07.934351    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b664aaed-8cbd-4ccd-b6d7-108c91b8c3ef" containerName="maas-api"
Jul 29 14:30:07.936595 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:07.936576    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl"
Jul 29 14:30:07.939144 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:07.939126    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-ee08ac7d-serving-cert\""
Jul 29 14:30:07.939248 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:07.939185    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7jqfw\""
Jul 29 14:30:07.946504 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:07.946474    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl"]
Jul 29 14:30:07.951719 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:07.951699    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:30:07.991260 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:07.991227    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lg864\" (UniqueName: \"kubernetes.io/projected/de19703b-2f48-40ee-9b1d-ad720b928c80-kube-api-access-lg864\") pod \"maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl\" (UID: \"de19703b-2f48-40ee-9b1d-ad720b928c80\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl"
Jul 29 14:30:07.991406 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:07.991310    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de19703b-2f48-40ee-9b1d-ad720b928c80-openshift-service-ca\") pod \"maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl\" (UID: \"de19703b-2f48-40ee-9b1d-ad720b928c80\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl"
Jul 29 14:30:07.991406 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:07.991370    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de19703b-2f48-40ee-9b1d-ad720b928c80-maas-api-tls\") pod \"maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl\" (UID: \"de19703b-2f48-40ee-9b1d-ad720b928c80\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl"
Jul 29 14:30:08.081493 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:08.081461    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:30:08.091762 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:08.091728    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de19703b-2f48-40ee-9b1d-ad720b928c80-maas-api-tls\") pod \"maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl\" (UID: \"de19703b-2f48-40ee-9b1d-ad720b928c80\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl"
Jul 29 14:30:08.091897 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:08.091774    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lg864\" (UniqueName: \"kubernetes.io/projected/de19703b-2f48-40ee-9b1d-ad720b928c80-kube-api-access-lg864\") pod \"maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl\" (UID: \"de19703b-2f48-40ee-9b1d-ad720b928c80\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl"
Jul 29 14:30:08.091897 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:08.091850    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de19703b-2f48-40ee-9b1d-ad720b928c80-openshift-service-ca\") pod \"maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl\" (UID: \"de19703b-2f48-40ee-9b1d-ad720b928c80\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl"
Jul 29 14:30:08.091897 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:30:08.091867    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-ee08ac7d-serving-cert: secret "maas-api-e2e-ait-ee08ac7d-serving-cert" not found
Jul 29 14:30:08.092096 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:30:08.091918    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de19703b-2f48-40ee-9b1d-ad720b928c80-maas-api-tls podName:de19703b-2f48-40ee-9b1d-ad720b928c80 nodeName:}" failed. No retries permitted until 2026-07-29 14:30:08.59190378 +0000 UTC m=+1738.482575093 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/de19703b-2f48-40ee-9b1d-ad720b928c80-maas-api-tls") pod "maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl" (UID: "de19703b-2f48-40ee-9b1d-ad720b928c80") : secret "maas-api-e2e-ait-ee08ac7d-serving-cert" not found
Jul 29 14:30:08.092551 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:08.092530    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de19703b-2f48-40ee-9b1d-ad720b928c80-openshift-service-ca\") pod \"maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl\" (UID: \"de19703b-2f48-40ee-9b1d-ad720b928c80\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl"
Jul 29 14:30:08.109323 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:08.109265    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg864\" (UniqueName: \"kubernetes.io/projected/de19703b-2f48-40ee-9b1d-ad720b928c80-kube-api-access-lg864\") pod \"maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl\" (UID: \"de19703b-2f48-40ee-9b1d-ad720b928c80\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl"
Jul 29 14:30:08.289792 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:08.289732    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:30:08.599077 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:08.599011    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de19703b-2f48-40ee-9b1d-ad720b928c80-maas-api-tls\") pod \"maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl\" (UID: \"de19703b-2f48-40ee-9b1d-ad720b928c80\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl"
Jul 29 14:30:08.601462 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:08.601435    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de19703b-2f48-40ee-9b1d-ad720b928c80-maas-api-tls\") pod \"maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl\" (UID: \"de19703b-2f48-40ee-9b1d-ad720b928c80\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl"
Jul 29 14:30:08.684642 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:08.684601    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:30:08.783222 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:08.783175    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:30:08.847462 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:08.847429    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl"
Jul 29 14:30:08.989201 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:08.989175    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl"]
Jul 29 14:30:08.991687 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:30:08.991650    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde19703b_2f48_40ee_9b1d_ad720b928c80.slice/crio-0f191e8e1ad91e08cc29d8a5a4e423a6f8057ecfbad46285786de00ca15899c3 WatchSource:0}: Error finding container 0f191e8e1ad91e08cc29d8a5a4e423a6f8057ecfbad46285786de00ca15899c3: Status 404 returned error can't find the container with id 0f191e8e1ad91e08cc29d8a5a4e423a6f8057ecfbad46285786de00ca15899c3
Jul 29 14:30:09.496774 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.496750    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:30:09.506662 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.506643    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/852d18ff-69b0-4479-b76e-2290013cb49c-istio-token\") pod \"852d18ff-69b0-4479-b76e-2290013cb49c\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") "
Jul 29 14:30:09.506743 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.506685    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/852d18ff-69b0-4479-b76e-2290013cb49c-istio-podinfo\") pod \"852d18ff-69b0-4479-b76e-2290013cb49c\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") "
Jul 29 14:30:09.506743 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.506712    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/852d18ff-69b0-4479-b76e-2290013cb49c-istio-envoy\") pod \"852d18ff-69b0-4479-b76e-2290013cb49c\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") "
Jul 29 14:30:09.506906 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.506882    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/852d18ff-69b0-4479-b76e-2290013cb49c-workload-socket\") pod \"852d18ff-69b0-4479-b76e-2290013cb49c\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") "
Jul 29 14:30:09.506954 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.506942    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/852d18ff-69b0-4479-b76e-2290013cb49c-credential-socket\") pod \"852d18ff-69b0-4479-b76e-2290013cb49c\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") "
Jul 29 14:30:09.507009 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.506982    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6s6h\" (UniqueName: \"kubernetes.io/projected/852d18ff-69b0-4479-b76e-2290013cb49c-kube-api-access-d6s6h\") pod \"852d18ff-69b0-4479-b76e-2290013cb49c\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") "
Jul 29 14:30:09.507082 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.507019    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/852d18ff-69b0-4479-b76e-2290013cb49c-istiod-ca-cert\") pod \"852d18ff-69b0-4479-b76e-2290013cb49c\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") "
Jul 29 14:30:09.507140 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.507096    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/852d18ff-69b0-4479-b76e-2290013cb49c-istio-data\") pod \"852d18ff-69b0-4479-b76e-2290013cb49c\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") "
Jul 29 14:30:09.507194 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.507138    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/852d18ff-69b0-4479-b76e-2290013cb49c-workload-certs\") pod \"852d18ff-69b0-4479-b76e-2290013cb49c\" (UID: \"852d18ff-69b0-4479-b76e-2290013cb49c\") "
Jul 29 14:30:09.507194 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.507171    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/852d18ff-69b0-4479-b76e-2290013cb49c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "852d18ff-69b0-4479-b76e-2290013cb49c" (UID: "852d18ff-69b0-4479-b76e-2290013cb49c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:30:09.507594 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.507438    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/852d18ff-69b0-4479-b76e-2290013cb49c-workload-socket\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:30:09.507594 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.507505    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/852d18ff-69b0-4479-b76e-2290013cb49c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "852d18ff-69b0-4479-b76e-2290013cb49c" (UID: "852d18ff-69b0-4479-b76e-2290013cb49c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:30:09.507594 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.507515    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/852d18ff-69b0-4479-b76e-2290013cb49c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "852d18ff-69b0-4479-b76e-2290013cb49c" (UID: "852d18ff-69b0-4479-b76e-2290013cb49c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 14:30:09.507811 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.507603    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/852d18ff-69b0-4479-b76e-2290013cb49c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "852d18ff-69b0-4479-b76e-2290013cb49c" (UID: "852d18ff-69b0-4479-b76e-2290013cb49c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:30:09.507972 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.507940    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/852d18ff-69b0-4479-b76e-2290013cb49c-istio-data" (OuterVolumeSpecName: "istio-data") pod "852d18ff-69b0-4479-b76e-2290013cb49c" (UID: "852d18ff-69b0-4479-b76e-2290013cb49c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:30:09.509125 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.509100    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/852d18ff-69b0-4479-b76e-2290013cb49c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "852d18ff-69b0-4479-b76e-2290013cb49c" (UID: "852d18ff-69b0-4479-b76e-2290013cb49c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 14:30:09.509214 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.509138    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/852d18ff-69b0-4479-b76e-2290013cb49c-istio-token" (OuterVolumeSpecName: "istio-token") pod "852d18ff-69b0-4479-b76e-2290013cb49c" (UID: "852d18ff-69b0-4479-b76e-2290013cb49c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:30:09.509214 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.509196    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/852d18ff-69b0-4479-b76e-2290013cb49c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "852d18ff-69b0-4479-b76e-2290013cb49c" (UID: "852d18ff-69b0-4479-b76e-2290013cb49c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:30:09.509290 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.509262    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/852d18ff-69b0-4479-b76e-2290013cb49c-kube-api-access-d6s6h" (OuterVolumeSpecName: "kube-api-access-d6s6h") pod "852d18ff-69b0-4479-b76e-2290013cb49c" (UID: "852d18ff-69b0-4479-b76e-2290013cb49c"). InnerVolumeSpecName "kube-api-access-d6s6h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:30:09.608237 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.608201    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/852d18ff-69b0-4479-b76e-2290013cb49c-credential-socket\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:30:09.608237 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.608233    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6s6h\" (UniqueName: \"kubernetes.io/projected/852d18ff-69b0-4479-b76e-2290013cb49c-kube-api-access-d6s6h\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:30:09.608457 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.608251    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/852d18ff-69b0-4479-b76e-2290013cb49c-istiod-ca-cert\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:30:09.608457 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.608260    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/852d18ff-69b0-4479-b76e-2290013cb49c-istio-data\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:30:09.608457 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.608268    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/852d18ff-69b0-4479-b76e-2290013cb49c-workload-certs\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:30:09.608457 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.608276    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/852d18ff-69b0-4479-b76e-2290013cb49c-istio-token\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:30:09.608457 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.608287    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/852d18ff-69b0-4479-b76e-2290013cb49c-istio-podinfo\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:30:09.608457 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.608299    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/852d18ff-69b0-4479-b76e-2290013cb49c-istio-envoy\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:30:09.855713 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.855681    2559 generic.go:356] "Generic (PLEG): container finished" podID="852d18ff-69b0-4479-b76e-2290013cb49c" containerID="4a58af619fd7b44a9ff2b7f18dad415cb699998500e29819b892ad4f91e7207b" exitCode=0
Jul 29 14:30:09.855873 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.855733    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"
Jul 29 14:30:09.855873 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.855761    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7" event={"ID":"852d18ff-69b0-4479-b76e-2290013cb49c","Type":"ContainerDied","Data":"4a58af619fd7b44a9ff2b7f18dad415cb699998500e29819b892ad4f91e7207b"}
Jul 29 14:30:09.855873 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.855824    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7" event={"ID":"852d18ff-69b0-4479-b76e-2290013cb49c","Type":"ContainerDied","Data":"ec9d2b0a3531f0a3a3d11037a6d92984097a2be7c58d3848f30858eb567b8f08"}
Jul 29 14:30:09.855873 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.855853    2559 scope.go:117] "RemoveContainer" containerID="4a58af619fd7b44a9ff2b7f18dad415cb699998500e29819b892ad4f91e7207b"
Jul 29 14:30:09.857556 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.857523    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl" event={"ID":"de19703b-2f48-40ee-9b1d-ad720b928c80","Type":"ContainerStarted","Data":"44ee60f7f912c5d56c820909d7349b1635ebab690b3c7b0d049d78f6df267292"}
Jul 29 14:30:09.857653 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.857573    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl" event={"ID":"de19703b-2f48-40ee-9b1d-ad720b928c80","Type":"ContainerStarted","Data":"0f191e8e1ad91e08cc29d8a5a4e423a6f8057ecfbad46285786de00ca15899c3"}
Jul 29 14:30:09.857713 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.857655    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl"
Jul 29 14:30:09.866162 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.866140    2559 scope.go:117] "RemoveContainer" containerID="b548944e5613808d52501ee6a8a483850fcca5d917df7895abdb5704db486358"
Jul 29 14:30:09.873384 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.873365    2559 scope.go:117] "RemoveContainer" containerID="4a58af619fd7b44a9ff2b7f18dad415cb699998500e29819b892ad4f91e7207b"
Jul 29 14:30:09.873626 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:30:09.873607    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a58af619fd7b44a9ff2b7f18dad415cb699998500e29819b892ad4f91e7207b\": container with ID starting with 4a58af619fd7b44a9ff2b7f18dad415cb699998500e29819b892ad4f91e7207b not found: ID does not exist" containerID="4a58af619fd7b44a9ff2b7f18dad415cb699998500e29819b892ad4f91e7207b"
Jul 29 14:30:09.873699 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.873637    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a58af619fd7b44a9ff2b7f18dad415cb699998500e29819b892ad4f91e7207b"} err="failed to get container status \"4a58af619fd7b44a9ff2b7f18dad415cb699998500e29819b892ad4f91e7207b\": rpc error: code = NotFound desc = could not find container \"4a58af619fd7b44a9ff2b7f18dad415cb699998500e29819b892ad4f91e7207b\": container with ID starting with 4a58af619fd7b44a9ff2b7f18dad415cb699998500e29819b892ad4f91e7207b not found: ID does not exist"
Jul 29 14:30:09.873699 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.873660    2559 scope.go:117] "RemoveContainer" containerID="b548944e5613808d52501ee6a8a483850fcca5d917df7895abdb5704db486358"
Jul 29 14:30:09.873892 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:30:09.873877    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b548944e5613808d52501ee6a8a483850fcca5d917df7895abdb5704db486358\": container with ID starting with b548944e5613808d52501ee6a8a483850fcca5d917df7895abdb5704db486358 not found: ID does not exist" containerID="b548944e5613808d52501ee6a8a483850fcca5d917df7895abdb5704db486358"
Jul 29 14:30:09.873932 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.873898    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b548944e5613808d52501ee6a8a483850fcca5d917df7895abdb5704db486358"} err="failed to get container status \"b548944e5613808d52501ee6a8a483850fcca5d917df7895abdb5704db486358\": rpc error: code = NotFound desc = could not find container \"b548944e5613808d52501ee6a8a483850fcca5d917df7895abdb5704db486358\": container with ID starting with b548944e5613808d52501ee6a8a483850fcca5d917df7895abdb5704db486358 not found: ID does not exist"
Jul 29 14:30:09.877451 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.877410    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl" podStartSLOduration=2.091765477 podStartE2EDuration="2.877397606s" totalImagesPullingTime="785.632129ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:30:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:30:09.876281628 +0000 UTC m=+1739.766952962" watchObservedRunningTime="2026-07-29 14:30:09.877397606 +0000 UTC m=+1739.768068940"
Jul 29 14:30:09.891403 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.891375    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"]
Jul 29 14:30:09.894995 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:09.894974    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-59408d2e-gw-openshift-default-85bffb7868-sffb7"]
Jul 29 14:30:10.637184 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:10.637151    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="852d18ff-69b0-4479-b76e-2290013cb49c" path="/var/lib/kubelet/pods/852d18ff-69b0-4479-b76e-2290013cb49c/volumes"
Jul 29 14:30:15.867617 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:15.867584    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl"
Jul 29 14:30:34.678371 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:34.678335    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:30:36.953532 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:36.953500    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:30:38.564446 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:38.564409    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl"]
Jul 29 14:30:38.564865 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:38.564629    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl" podUID="de19703b-2f48-40ee-9b1d-ad720b928c80" containerName="maas-api" containerID="cri-o://44ee60f7f912c5d56c820909d7349b1635ebab690b3c7b0d049d78f6df267292" gracePeriod=30
Jul 29 14:30:38.595137 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:38.595110    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:30:38.673765 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:38.673708    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:30:38.738857 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:38.738812    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:30:38.797409 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:38.797382    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:30:38.815599 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:38.815537    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl"
Jul 29 14:30:38.963503 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:38.963476    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lg864\" (UniqueName: \"kubernetes.io/projected/de19703b-2f48-40ee-9b1d-ad720b928c80-kube-api-access-lg864\") pod \"de19703b-2f48-40ee-9b1d-ad720b928c80\" (UID: \"de19703b-2f48-40ee-9b1d-ad720b928c80\") "
Jul 29 14:30:38.963656 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:38.963524    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de19703b-2f48-40ee-9b1d-ad720b928c80-openshift-service-ca\") pod \"de19703b-2f48-40ee-9b1d-ad720b928c80\" (UID: \"de19703b-2f48-40ee-9b1d-ad720b928c80\") "
Jul 29 14:30:38.963656 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:38.963553    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de19703b-2f48-40ee-9b1d-ad720b928c80-maas-api-tls\") pod \"de19703b-2f48-40ee-9b1d-ad720b928c80\" (UID: \"de19703b-2f48-40ee-9b1d-ad720b928c80\") "
Jul 29 14:30:38.963913 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:38.963887    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de19703b-2f48-40ee-9b1d-ad720b928c80-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "de19703b-2f48-40ee-9b1d-ad720b928c80" (UID: "de19703b-2f48-40ee-9b1d-ad720b928c80"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 14:30:38.965621 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:38.965587    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de19703b-2f48-40ee-9b1d-ad720b928c80-kube-api-access-lg864" (OuterVolumeSpecName: "kube-api-access-lg864") pod "de19703b-2f48-40ee-9b1d-ad720b928c80" (UID: "de19703b-2f48-40ee-9b1d-ad720b928c80"). InnerVolumeSpecName "kube-api-access-lg864". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:30:38.965909 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:38.965881    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de19703b-2f48-40ee-9b1d-ad720b928c80-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "de19703b-2f48-40ee-9b1d-ad720b928c80" (UID: "de19703b-2f48-40ee-9b1d-ad720b928c80"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 14:30:38.980739 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:38.980710    2559 generic.go:356] "Generic (PLEG): container finished" podID="de19703b-2f48-40ee-9b1d-ad720b928c80" containerID="44ee60f7f912c5d56c820909d7349b1635ebab690b3c7b0d049d78f6df267292" exitCode=0
Jul 29 14:30:38.980857 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:38.980759    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl"
Jul 29 14:30:38.980857 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:38.980776    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl" event={"ID":"de19703b-2f48-40ee-9b1d-ad720b928c80","Type":"ContainerDied","Data":"44ee60f7f912c5d56c820909d7349b1635ebab690b3c7b0d049d78f6df267292"}
Jul 29 14:30:38.980857 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:38.980829    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl" event={"ID":"de19703b-2f48-40ee-9b1d-ad720b928c80","Type":"ContainerDied","Data":"0f191e8e1ad91e08cc29d8a5a4e423a6f8057ecfbad46285786de00ca15899c3"}
Jul 29 14:30:38.980967 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:38.980859    2559 scope.go:117] "RemoveContainer" containerID="44ee60f7f912c5d56c820909d7349b1635ebab690b3c7b0d049d78f6df267292"
Jul 29 14:30:38.989216 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:38.989193    2559 scope.go:117] "RemoveContainer" containerID="44ee60f7f912c5d56c820909d7349b1635ebab690b3c7b0d049d78f6df267292"
Jul 29 14:30:38.989518 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:30:38.989493    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44ee60f7f912c5d56c820909d7349b1635ebab690b3c7b0d049d78f6df267292\": container with ID starting with 44ee60f7f912c5d56c820909d7349b1635ebab690b3c7b0d049d78f6df267292 not found: ID does not exist" containerID="44ee60f7f912c5d56c820909d7349b1635ebab690b3c7b0d049d78f6df267292"
Jul 29 14:30:38.989594 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:38.989528    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44ee60f7f912c5d56c820909d7349b1635ebab690b3c7b0d049d78f6df267292"} err="failed to get container status \"44ee60f7f912c5d56c820909d7349b1635ebab690b3c7b0d049d78f6df267292\": rpc error: code = NotFound desc = could not find container \"44ee60f7f912c5d56c820909d7349b1635ebab690b3c7b0d049d78f6df267292\": container with ID starting with 44ee60f7f912c5d56c820909d7349b1635ebab690b3c7b0d049d78f6df267292 not found: ID does not exist"
Jul 29 14:30:39.002995 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:39.002975    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl"]
Jul 29 14:30:39.006989 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:39.006967    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ee08ac7d-fc6754d45-wbrxl"]
Jul 29 14:30:39.065170 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:39.065145    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de19703b-2f48-40ee-9b1d-ad720b928c80-openshift-service-ca\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:30:39.065170 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:39.065168    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de19703b-2f48-40ee-9b1d-ad720b928c80-maas-api-tls\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:30:39.065328 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:39.065178    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lg864\" (UniqueName: \"kubernetes.io/projected/de19703b-2f48-40ee-9b1d-ad720b928c80-kube-api-access-lg864\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:30:39.956117 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:39.956075    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:30:40.193189 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.193152    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:30:40.407603 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.407569    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:30:40.636023 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.635989    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de19703b-2f48-40ee-9b1d-ad720b928c80" path="/var/lib/kubelet/pods/de19703b-2f48-40ee-9b1d-ad720b928c80/volumes"
Jul 29 14:30:40.699812 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.699731    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"]
Jul 29 14:30:40.700121 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.700105    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de19703b-2f48-40ee-9b1d-ad720b928c80" containerName="maas-api"
Jul 29 14:30:40.700121 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.700120    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="de19703b-2f48-40ee-9b1d-ad720b928c80" containerName="maas-api"
Jul 29 14:30:40.700261 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.700134    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="852d18ff-69b0-4479-b76e-2290013cb49c" containerName="istio-proxy"
Jul 29 14:30:40.700261 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.700141    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="852d18ff-69b0-4479-b76e-2290013cb49c" containerName="istio-proxy"
Jul 29 14:30:40.700261 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.700148    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="852d18ff-69b0-4479-b76e-2290013cb49c" containerName="istio-proxy"
Jul 29 14:30:40.700261 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.700156    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="852d18ff-69b0-4479-b76e-2290013cb49c" containerName="istio-proxy"
Jul 29 14:30:40.700261 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.700213    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="852d18ff-69b0-4479-b76e-2290013cb49c" containerName="istio-proxy"
Jul 29 14:30:40.700261 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.700229    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="de19703b-2f48-40ee-9b1d-ad720b928c80" containerName="maas-api"
Jul 29 14:30:40.702948 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.702927    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:40.705732 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.705711    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-330348dc-openshift-default-dockercfg-z48wc\""
Jul 29 14:30:40.714211 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.714190    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"]
Jul 29 14:30:40.724789 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.724760    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:30:40.779179 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.779155    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3358bfe7-94ae-4fd1-8089-325e6e40a669-istiod-ca-cert\") pod \"e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") " pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:40.779295 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.779190    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3358bfe7-94ae-4fd1-8089-325e6e40a669-istio-token\") pod \"e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") " pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:40.779295 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.779210    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdcjr\" (UniqueName: \"kubernetes.io/projected/3358bfe7-94ae-4fd1-8089-325e6e40a669-kube-api-access-tdcjr\") pod \"e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") " pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:40.779368 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.779303    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3358bfe7-94ae-4fd1-8089-325e6e40a669-istio-data\") pod \"e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") " pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:40.779368 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.779333    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3358bfe7-94ae-4fd1-8089-325e6e40a669-workload-certs\") pod \"e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") " pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:40.779368 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.779362    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3358bfe7-94ae-4fd1-8089-325e6e40a669-istio-podinfo\") pod \"e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") " pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:40.779472 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.779400    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3358bfe7-94ae-4fd1-8089-325e6e40a669-istio-envoy\") pod \"e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") " pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:40.779506 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.779469    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3358bfe7-94ae-4fd1-8089-325e6e40a669-workload-socket\") pod \"e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") " pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:40.779506 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.779493    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3358bfe7-94ae-4fd1-8089-325e6e40a669-credential-socket\") pod \"e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") " pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:40.880330 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.880296    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3358bfe7-94ae-4fd1-8089-325e6e40a669-istio-envoy\") pod \"e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") " pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:40.880503 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.880342    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3358bfe7-94ae-4fd1-8089-325e6e40a669-workload-socket\") pod \"e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") " pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:40.880503 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.880362    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3358bfe7-94ae-4fd1-8089-325e6e40a669-credential-socket\") pod \"e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") " pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:40.880503 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.880392    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3358bfe7-94ae-4fd1-8089-325e6e40a669-istiod-ca-cert\") pod \"e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") " pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:40.880503 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.880419    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3358bfe7-94ae-4fd1-8089-325e6e40a669-istio-token\") pod \"e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") " pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:40.880503 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.880445    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdcjr\" (UniqueName: \"kubernetes.io/projected/3358bfe7-94ae-4fd1-8089-325e6e40a669-kube-api-access-tdcjr\") pod \"e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") " pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:40.880503 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.880501    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3358bfe7-94ae-4fd1-8089-325e6e40a669-istio-data\") pod \"e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") " pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:40.880832 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.880541    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3358bfe7-94ae-4fd1-8089-325e6e40a669-workload-certs\") pod \"e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") " pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:40.880832 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.880582    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3358bfe7-94ae-4fd1-8089-325e6e40a669-istio-podinfo\") pod \"e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") " pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:40.880832 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.880729    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3358bfe7-94ae-4fd1-8089-325e6e40a669-workload-socket\") pod \"e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") " pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:40.880966 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.880916    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3358bfe7-94ae-4fd1-8089-325e6e40a669-credential-socket\") pod \"e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") " pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:40.881032 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.880964    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3358bfe7-94ae-4fd1-8089-325e6e40a669-istio-data\") pod \"e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") " pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:40.881145 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.881121    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3358bfe7-94ae-4fd1-8089-325e6e40a669-istiod-ca-cert\") pod \"e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") " pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:40.881217 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.881135    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3358bfe7-94ae-4fd1-8089-325e6e40a669-workload-certs\") pod \"e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") " pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:40.882474 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.882451    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3358bfe7-94ae-4fd1-8089-325e6e40a669-istio-envoy\") pod \"e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") " pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:40.882789 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.882772    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3358bfe7-94ae-4fd1-8089-325e6e40a669-istio-podinfo\") pod \"e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") " pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:40.889375 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.889355    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdcjr\" (UniqueName: \"kubernetes.io/projected/3358bfe7-94ae-4fd1-8089-325e6e40a669-kube-api-access-tdcjr\") pod \"e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") " pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:40.889528 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.889508    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3358bfe7-94ae-4fd1-8089-325e6e40a669-istio-token\") pod \"e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") " pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:40.962370 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:40.962286    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:30:41.013212 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:41.013188    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:41.149040 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:41.149006    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"]
Jul 29 14:30:41.151395 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:30:41.151365    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3358bfe7_94ae_4fd1_8089_325e6e40a669.slice/crio-092995ca5614796ac680c3a055725bab2696e9f52cfb471c4fcbf7b1584be99a WatchSource:0}: Error finding container 092995ca5614796ac680c3a055725bab2696e9f52cfb471c4fcbf7b1584be99a: Status 404 returned error can't find the container with id 092995ca5614796ac680c3a055725bab2696e9f52cfb471c4fcbf7b1584be99a
Jul 29 14:30:41.153580 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:41.153544    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:30:41.153642 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:41.153625    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:30:41.153694 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:41.153668    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:30:41.994933 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:41.994896    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s" event={"ID":"3358bfe7-94ae-4fd1-8089-325e6e40a669","Type":"ContainerStarted","Data":"32e76109623ab7cef18ea42a7fd94894ffea5ec0cb6a236c49d5ee911f92322c"}
Jul 29 14:30:41.995375 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:41.994950    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s" event={"ID":"3358bfe7-94ae-4fd1-8089-325e6e40a669","Type":"ContainerStarted","Data":"092995ca5614796ac680c3a055725bab2696e9f52cfb471c4fcbf7b1584be99a"}
Jul 29 14:30:42.013726 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:42.013699    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:42.018243 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:42.018193    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s" podStartSLOduration=2.018176522 podStartE2EDuration="2.018176522s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:30:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:30:42.016910077 +0000 UTC m=+1771.907581412" watchObservedRunningTime="2026-07-29 14:30:42.018176522 +0000 UTC m=+1771.908847858"
Jul 29 14:30:42.019178 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:42.019155    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:42.999032 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:42.999000    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:42.999853 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:42.999833    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:43.365567 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:43.365411    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:30:43.463317 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:43.463282    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:30:43.496375 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:43.496343    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-330348dc-544d79dbc9-ffrln"]
Jul 29 14:30:43.496717 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:43.496704    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="852d18ff-69b0-4479-b76e-2290013cb49c" containerName="istio-proxy"
Jul 29 14:30:43.498357 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:43.498343    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-330348dc-544d79dbc9-ffrln"
Jul 29 14:30:43.500930 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:43.500901    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-330348dc-serving-cert\""
Jul 29 14:30:43.501050 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:43.500950    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7jqfw\""
Jul 29 14:30:43.509573 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:43.509548    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-330348dc-544d79dbc9-ffrln"]
Jul 29 14:30:43.602130 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:43.602097    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54a93fbc-be71-4060-aa28-78a7e6d8411b-openshift-service-ca\") pod \"maas-api-e2e-derive-330348dc-544d79dbc9-ffrln\" (UID: \"54a93fbc-be71-4060-aa28-78a7e6d8411b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-330348dc-544d79dbc9-ffrln"
Jul 29 14:30:43.602295 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:43.602201    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54a93fbc-be71-4060-aa28-78a7e6d8411b-maas-api-tls\") pod \"maas-api-e2e-derive-330348dc-544d79dbc9-ffrln\" (UID: \"54a93fbc-be71-4060-aa28-78a7e6d8411b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-330348dc-544d79dbc9-ffrln"
Jul 29 14:30:43.602295 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:43.602269    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlt24\" (UniqueName: \"kubernetes.io/projected/54a93fbc-be71-4060-aa28-78a7e6d8411b-kube-api-access-tlt24\") pod \"maas-api-e2e-derive-330348dc-544d79dbc9-ffrln\" (UID: \"54a93fbc-be71-4060-aa28-78a7e6d8411b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-330348dc-544d79dbc9-ffrln"
Jul 29 14:30:43.603661 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:43.603631    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:30:43.703088 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:43.702966    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlt24\" (UniqueName: \"kubernetes.io/projected/54a93fbc-be71-4060-aa28-78a7e6d8411b-kube-api-access-tlt24\") pod \"maas-api-e2e-derive-330348dc-544d79dbc9-ffrln\" (UID: \"54a93fbc-be71-4060-aa28-78a7e6d8411b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-330348dc-544d79dbc9-ffrln"
Jul 29 14:30:43.703088 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:43.703011    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54a93fbc-be71-4060-aa28-78a7e6d8411b-openshift-service-ca\") pod \"maas-api-e2e-derive-330348dc-544d79dbc9-ffrln\" (UID: \"54a93fbc-be71-4060-aa28-78a7e6d8411b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-330348dc-544d79dbc9-ffrln"
Jul 29 14:30:43.703287 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:43.703144    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54a93fbc-be71-4060-aa28-78a7e6d8411b-maas-api-tls\") pod \"maas-api-e2e-derive-330348dc-544d79dbc9-ffrln\" (UID: \"54a93fbc-be71-4060-aa28-78a7e6d8411b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-330348dc-544d79dbc9-ffrln"
Jul 29 14:30:43.703287 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:30:43.703259    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-330348dc-serving-cert: secret "maas-api-e2e-derive-330348dc-serving-cert" not found
Jul 29 14:30:43.703354 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:30:43.703323    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54a93fbc-be71-4060-aa28-78a7e6d8411b-maas-api-tls podName:54a93fbc-be71-4060-aa28-78a7e6d8411b nodeName:}" failed. No retries permitted until 2026-07-29 14:30:44.203295213 +0000 UTC m=+1774.093966527 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/54a93fbc-be71-4060-aa28-78a7e6d8411b-maas-api-tls") pod "maas-api-e2e-derive-330348dc-544d79dbc9-ffrln" (UID: "54a93fbc-be71-4060-aa28-78a7e6d8411b") : secret "maas-api-e2e-derive-330348dc-serving-cert" not found
Jul 29 14:30:43.703607 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:43.703590    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54a93fbc-be71-4060-aa28-78a7e6d8411b-openshift-service-ca\") pod \"maas-api-e2e-derive-330348dc-544d79dbc9-ffrln\" (UID: \"54a93fbc-be71-4060-aa28-78a7e6d8411b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-330348dc-544d79dbc9-ffrln"
Jul 29 14:30:43.720585 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:43.720561    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlt24\" (UniqueName: \"kubernetes.io/projected/54a93fbc-be71-4060-aa28-78a7e6d8411b-kube-api-access-tlt24\") pod \"maas-api-e2e-derive-330348dc-544d79dbc9-ffrln\" (UID: \"54a93fbc-be71-4060-aa28-78a7e6d8411b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-330348dc-544d79dbc9-ffrln"
Jul 29 14:30:44.115541 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:44.115510    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:30:44.207046 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:44.207018    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54a93fbc-be71-4060-aa28-78a7e6d8411b-maas-api-tls\") pod \"maas-api-e2e-derive-330348dc-544d79dbc9-ffrln\" (UID: \"54a93fbc-be71-4060-aa28-78a7e6d8411b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-330348dc-544d79dbc9-ffrln"
Jul 29 14:30:44.209347 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:44.209320    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54a93fbc-be71-4060-aa28-78a7e6d8411b-maas-api-tls\") pod \"maas-api-e2e-derive-330348dc-544d79dbc9-ffrln\" (UID: \"54a93fbc-be71-4060-aa28-78a7e6d8411b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-330348dc-544d79dbc9-ffrln"
Jul 29 14:30:44.242393 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:44.242362    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:30:44.342473 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:44.342439    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:30:44.408997 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:44.408916    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-330348dc-544d79dbc9-ffrln"
Jul 29 14:30:44.546245 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:44.546210    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-330348dc-544d79dbc9-ffrln"]
Jul 29 14:30:44.550270 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:30:44.550242    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54a93fbc_be71_4060_aa28_78a7e6d8411b.slice/crio-d2acf6bd51902e1d6a4b85d59e61054fcfd98ac1717c92f51411e57e01ed4868 WatchSource:0}: Error finding container d2acf6bd51902e1d6a4b85d59e61054fcfd98ac1717c92f51411e57e01ed4868: Status 404 returned error can't find the container with id d2acf6bd51902e1d6a4b85d59e61054fcfd98ac1717c92f51411e57e01ed4868
Jul 29 14:30:45.008341 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:45.008295    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-330348dc-544d79dbc9-ffrln" event={"ID":"54a93fbc-be71-4060-aa28-78a7e6d8411b","Type":"ContainerStarted","Data":"d2acf6bd51902e1d6a4b85d59e61054fcfd98ac1717c92f51411e57e01ed4868"}
Jul 29 14:30:48.023090 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:48.023044    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-330348dc-544d79dbc9-ffrln" event={"ID":"54a93fbc-be71-4060-aa28-78a7e6d8411b","Type":"ContainerStarted","Data":"626f14127ecad3a3dbc8f050053df31bf874066fdc13d010f3b62305c931f177"}
Jul 29 14:30:48.023522 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:48.023408    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-330348dc-544d79dbc9-ffrln"
Jul 29 14:30:48.044853 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:48.044803    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-330348dc-544d79dbc9-ffrln" podStartSLOduration=2.578846319 podStartE2EDuration="5.044789506s" totalImagesPullingTime="2.465943187s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:30:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:30:48.043237853 +0000 UTC m=+1777.933909206" watchObservedRunningTime="2026-07-29 14:30:48.044789506 +0000 UTC m=+1777.935460841"
Jul 29 14:30:54.033136 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:54.033104    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-330348dc-544d79dbc9-ffrln"
Jul 29 14:30:57.063252 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:57.063160    2559 generic.go:356] "Generic (PLEG): container finished" podID="3358bfe7-94ae-4fd1-8089-325e6e40a669" containerID="32e76109623ab7cef18ea42a7fd94894ffea5ec0cb6a236c49d5ee911f92322c" exitCode=0
Jul 29 14:30:57.063762 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:57.063236    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s" event={"ID":"3358bfe7-94ae-4fd1-8089-325e6e40a669","Type":"ContainerDied","Data":"32e76109623ab7cef18ea42a7fd94894ffea5ec0cb6a236c49d5ee911f92322c"}
Jul 29 14:30:57.063762 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:57.063716    2559 scope.go:117] "RemoveContainer" containerID="32e76109623ab7cef18ea42a7fd94894ffea5ec0cb6a236c49d5ee911f92322c"
Jul 29 14:30:57.064736 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:57.064708    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:30:57.064795 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:57.064777    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:30:57.064830 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:57.064810    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:30:57.998861 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:57.998827    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:58.071665 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:58.071618    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s" event={"ID":"3358bfe7-94ae-4fd1-8089-325e6e40a669","Type":"ContainerStarted","Data":"581fecf680c6e04714927d681e47a063930cdd6406589e801ef44299b1af57ac"}
Jul 29 14:30:59.014037 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:59.014005    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:30:59.015158 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:59.015128    2559 patch_prober.go:28] interesting pod/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Jul 29 14:30:59.015294 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:30:59.015200    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s" podUID="3358bfe7-94ae-4fd1-8089-325e6e40a669" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Jul 29 14:31:00.014345 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:00.014304    2559 patch_prober.go:28] interesting pod/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Jul 29 14:31:00.014827 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:00.014364    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s" podUID="3358bfe7-94ae-4fd1-8089-325e6e40a669" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Jul 29 14:31:01.013779 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:01.013749    2559 patch_prober.go:28] interesting pod/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Jul 29 14:31:01.013974 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:01.013799    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s" podUID="3358bfe7-94ae-4fd1-8089-325e6e40a669" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Jul 29 14:31:02.018301 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:02.018270    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:31:02.018772 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:02.018653    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:31:02.019192 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:02.019173    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:31:10.674318 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:10.674281    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdw8l_b93f6faa-c8bf-43e9-94e0-b3dcd97b7296/ovn-acl-logging/0.log"
Jul 29 14:31:10.678781 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:10.678758    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdw8l_b93f6faa-c8bf-43e9-94e0-b3dcd97b7296/ovn-acl-logging/0.log"
Jul 29 14:31:14.186314 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:14.186277    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-330348dc-544d79dbc9-ffrln"]
Jul 29 14:31:14.186682 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:14.186560    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-330348dc-544d79dbc9-ffrln" podUID="54a93fbc-be71-4060-aa28-78a7e6d8411b" containerName="maas-api" containerID="cri-o://626f14127ecad3a3dbc8f050053df31bf874066fdc13d010f3b62305c931f177" gracePeriod=30
Jul 29 14:31:14.213951 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:14.213914    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:31:14.293817 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:14.293782    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:31:14.389556 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:14.385901    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:31:14.442998 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:14.442887    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:31:14.447343 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:14.447318    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-330348dc-544d79dbc9-ffrln"
Jul 29 14:31:14.541257 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:14.541210    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"]
Jul 29 14:31:14.541494 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:14.541465    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s" podUID="3358bfe7-94ae-4fd1-8089-325e6e40a669" containerName="istio-proxy" containerID="cri-o://581fecf680c6e04714927d681e47a063930cdd6406589e801ef44299b1af57ac" gracePeriod=30
Jul 29 14:31:14.586299 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:14.586269    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54a93fbc-be71-4060-aa28-78a7e6d8411b-openshift-service-ca\") pod \"54a93fbc-be71-4060-aa28-78a7e6d8411b\" (UID: \"54a93fbc-be71-4060-aa28-78a7e6d8411b\") "
Jul 29 14:31:14.586299 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:14.586312    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlt24\" (UniqueName: \"kubernetes.io/projected/54a93fbc-be71-4060-aa28-78a7e6d8411b-kube-api-access-tlt24\") pod \"54a93fbc-be71-4060-aa28-78a7e6d8411b\" (UID: \"54a93fbc-be71-4060-aa28-78a7e6d8411b\") "
Jul 29 14:31:14.586531 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:14.586395    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54a93fbc-be71-4060-aa28-78a7e6d8411b-maas-api-tls\") pod \"54a93fbc-be71-4060-aa28-78a7e6d8411b\" (UID: \"54a93fbc-be71-4060-aa28-78a7e6d8411b\") "
Jul 29 14:31:14.586674 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:14.586646    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54a93fbc-be71-4060-aa28-78a7e6d8411b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "54a93fbc-be71-4060-aa28-78a7e6d8411b" (UID: "54a93fbc-be71-4060-aa28-78a7e6d8411b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 14:31:14.586773 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:14.586710    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/54a93fbc-be71-4060-aa28-78a7e6d8411b-openshift-service-ca\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:31:14.588704 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:14.588674    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54a93fbc-be71-4060-aa28-78a7e6d8411b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "54a93fbc-be71-4060-aa28-78a7e6d8411b" (UID: "54a93fbc-be71-4060-aa28-78a7e6d8411b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 14:31:14.588842 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:14.588795    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:31:14.588842 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:14.588813    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54a93fbc-be71-4060-aa28-78a7e6d8411b-kube-api-access-tlt24" (OuterVolumeSpecName: "kube-api-access-tlt24") pod "54a93fbc-be71-4060-aa28-78a7e6d8411b" (UID: "54a93fbc-be71-4060-aa28-78a7e6d8411b"). InnerVolumeSpecName "kube-api-access-tlt24". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:31:14.687971 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:14.687937    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/54a93fbc-be71-4060-aa28-78a7e6d8411b-maas-api-tls\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:31:14.687971 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:14.687965    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlt24\" (UniqueName: \"kubernetes.io/projected/54a93fbc-be71-4060-aa28-78a7e6d8411b-kube-api-access-tlt24\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:31:14.799785 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:14.799739    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:31:15.146862 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:15.146783    2559 generic.go:356] "Generic (PLEG): container finished" podID="54a93fbc-be71-4060-aa28-78a7e6d8411b" containerID="626f14127ecad3a3dbc8f050053df31bf874066fdc13d010f3b62305c931f177" exitCode=0
Jul 29 14:31:15.146862 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:15.146827    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-330348dc-544d79dbc9-ffrln"
Jul 29 14:31:15.146862 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:15.146851    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-330348dc-544d79dbc9-ffrln" event={"ID":"54a93fbc-be71-4060-aa28-78a7e6d8411b","Type":"ContainerDied","Data":"626f14127ecad3a3dbc8f050053df31bf874066fdc13d010f3b62305c931f177"}
Jul 29 14:31:15.147089 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:15.146889    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-330348dc-544d79dbc9-ffrln" event={"ID":"54a93fbc-be71-4060-aa28-78a7e6d8411b","Type":"ContainerDied","Data":"d2acf6bd51902e1d6a4b85d59e61054fcfd98ac1717c92f51411e57e01ed4868"}
Jul 29 14:31:15.147089 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:15.146913    2559 scope.go:117] "RemoveContainer" containerID="626f14127ecad3a3dbc8f050053df31bf874066fdc13d010f3b62305c931f177"
Jul 29 14:31:15.154876 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:15.154858    2559 scope.go:117] "RemoveContainer" containerID="626f14127ecad3a3dbc8f050053df31bf874066fdc13d010f3b62305c931f177"
Jul 29 14:31:15.155148 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:31:15.155129    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"626f14127ecad3a3dbc8f050053df31bf874066fdc13d010f3b62305c931f177\": container with ID starting with 626f14127ecad3a3dbc8f050053df31bf874066fdc13d010f3b62305c931f177 not found: ID does not exist" containerID="626f14127ecad3a3dbc8f050053df31bf874066fdc13d010f3b62305c931f177"
Jul 29 14:31:15.155206 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:15.155160    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"626f14127ecad3a3dbc8f050053df31bf874066fdc13d010f3b62305c931f177"} err="failed to get container status \"626f14127ecad3a3dbc8f050053df31bf874066fdc13d010f3b62305c931f177\": rpc error: code = NotFound desc = could not find container \"626f14127ecad3a3dbc8f050053df31bf874066fdc13d010f3b62305c931f177\": container with ID starting with 626f14127ecad3a3dbc8f050053df31bf874066fdc13d010f3b62305c931f177 not found: ID does not exist"
Jul 29 14:31:15.164659 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:15.164636    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-330348dc-544d79dbc9-ffrln"]
Jul 29 14:31:15.171551 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:15.171526    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-330348dc-544d79dbc9-ffrln"]
Jul 29 14:31:15.278307 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:15.278275    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:31:15.375755 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:15.375705    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:31:15.476798 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:15.476729    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:31:16.635633 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:16.635598    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54a93fbc-be71-4060-aa28-78a7e6d8411b" path="/var/lib/kubelet/pods/54a93fbc-be71-4060-aa28-78a7e6d8411b/volumes"
Jul 29 14:31:16.858025 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:16.857993    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:31:17.020836 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:17.020801    2559 patch_prober.go:28] interesting pod/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 29 14:31:17.020995 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:17.020856    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s" podUID="3358bfe7-94ae-4fd1-8089-325e6e40a669" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 14:31:19.793355 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:19.793329    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:31:19.936307 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:19.936235    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3358bfe7-94ae-4fd1-8089-325e6e40a669-workload-certs\") pod \"3358bfe7-94ae-4fd1-8089-325e6e40a669\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") "
Jul 29 14:31:19.936307 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:19.936279    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3358bfe7-94ae-4fd1-8089-325e6e40a669-workload-socket\") pod \"3358bfe7-94ae-4fd1-8089-325e6e40a669\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") "
Jul 29 14:31:19.936485 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:19.936321    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdcjr\" (UniqueName: \"kubernetes.io/projected/3358bfe7-94ae-4fd1-8089-325e6e40a669-kube-api-access-tdcjr\") pod \"3358bfe7-94ae-4fd1-8089-325e6e40a669\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") "
Jul 29 14:31:19.936485 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:19.936355    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3358bfe7-94ae-4fd1-8089-325e6e40a669-istio-podinfo\") pod \"3358bfe7-94ae-4fd1-8089-325e6e40a669\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") "
Jul 29 14:31:19.936485 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:19.936382    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3358bfe7-94ae-4fd1-8089-325e6e40a669-istio-envoy\") pod \"3358bfe7-94ae-4fd1-8089-325e6e40a669\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") "
Jul 29 14:31:19.936485 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:19.936406    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3358bfe7-94ae-4fd1-8089-325e6e40a669-credential-socket\") pod \"3358bfe7-94ae-4fd1-8089-325e6e40a669\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") "
Jul 29 14:31:19.936700 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:19.936549    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3358bfe7-94ae-4fd1-8089-325e6e40a669-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3358bfe7-94ae-4fd1-8089-325e6e40a669" (UID: "3358bfe7-94ae-4fd1-8089-325e6e40a669"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:31:19.936700 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:19.936551    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3358bfe7-94ae-4fd1-8089-325e6e40a669-istio-data\") pod \"3358bfe7-94ae-4fd1-8089-325e6e40a669\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") "
Jul 29 14:31:19.936700 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:19.936644    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3358bfe7-94ae-4fd1-8089-325e6e40a669-istiod-ca-cert\") pod \"3358bfe7-94ae-4fd1-8089-325e6e40a669\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") "
Jul 29 14:31:19.936700 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:19.936661    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3358bfe7-94ae-4fd1-8089-325e6e40a669-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3358bfe7-94ae-4fd1-8089-325e6e40a669" (UID: "3358bfe7-94ae-4fd1-8089-325e6e40a669"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:31:19.936700 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:19.936680    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3358bfe7-94ae-4fd1-8089-325e6e40a669-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3358bfe7-94ae-4fd1-8089-325e6e40a669" (UID: "3358bfe7-94ae-4fd1-8089-325e6e40a669"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:31:19.936959 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:19.936737    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3358bfe7-94ae-4fd1-8089-325e6e40a669-istio-token\") pod \"3358bfe7-94ae-4fd1-8089-325e6e40a669\" (UID: \"3358bfe7-94ae-4fd1-8089-325e6e40a669\") "
Jul 29 14:31:19.936959 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:19.936919    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3358bfe7-94ae-4fd1-8089-325e6e40a669-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3358bfe7-94ae-4fd1-8089-325e6e40a669" (UID: "3358bfe7-94ae-4fd1-8089-325e6e40a669"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 14:31:19.937087 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:19.936996    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3358bfe7-94ae-4fd1-8089-325e6e40a669-workload-socket\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:31:19.937087 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:19.937012    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3358bfe7-94ae-4fd1-8089-325e6e40a669-credential-socket\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:31:19.937087 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:19.937021    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3358bfe7-94ae-4fd1-8089-325e6e40a669-istiod-ca-cert\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:31:19.937087 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:19.937031    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3358bfe7-94ae-4fd1-8089-325e6e40a669-workload-certs\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:31:19.937459 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:19.937437    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3358bfe7-94ae-4fd1-8089-325e6e40a669-istio-data" (OuterVolumeSpecName: "istio-data") pod "3358bfe7-94ae-4fd1-8089-325e6e40a669" (UID: "3358bfe7-94ae-4fd1-8089-325e6e40a669"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:31:19.938694 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:19.938665    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3358bfe7-94ae-4fd1-8089-325e6e40a669-kube-api-access-tdcjr" (OuterVolumeSpecName: "kube-api-access-tdcjr") pod "3358bfe7-94ae-4fd1-8089-325e6e40a669" (UID: "3358bfe7-94ae-4fd1-8089-325e6e40a669"). InnerVolumeSpecName "kube-api-access-tdcjr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:31:19.938694 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:19.938676    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3358bfe7-94ae-4fd1-8089-325e6e40a669-istio-token" (OuterVolumeSpecName: "istio-token") pod "3358bfe7-94ae-4fd1-8089-325e6e40a669" (UID: "3358bfe7-94ae-4fd1-8089-325e6e40a669"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:31:19.938957 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:19.938927    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3358bfe7-94ae-4fd1-8089-325e6e40a669-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3358bfe7-94ae-4fd1-8089-325e6e40a669" (UID: "3358bfe7-94ae-4fd1-8089-325e6e40a669"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 14:31:19.938957 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:19.938928    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3358bfe7-94ae-4fd1-8089-325e6e40a669-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3358bfe7-94ae-4fd1-8089-325e6e40a669" (UID: "3358bfe7-94ae-4fd1-8089-325e6e40a669"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:31:20.037863 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:20.037824    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3358bfe7-94ae-4fd1-8089-325e6e40a669-istio-token\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:31:20.037863 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:20.037855    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdcjr\" (UniqueName: \"kubernetes.io/projected/3358bfe7-94ae-4fd1-8089-325e6e40a669-kube-api-access-tdcjr\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:31:20.037863 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:20.037865    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3358bfe7-94ae-4fd1-8089-325e6e40a669-istio-podinfo\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:31:20.037863 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:20.037874    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3358bfe7-94ae-4fd1-8089-325e6e40a669-istio-envoy\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:31:20.038144 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:20.037882    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3358bfe7-94ae-4fd1-8089-325e6e40a669-istio-data\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:31:20.171261 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:20.171226    2559 generic.go:356] "Generic (PLEG): container finished" podID="3358bfe7-94ae-4fd1-8089-325e6e40a669" containerID="581fecf680c6e04714927d681e47a063930cdd6406589e801ef44299b1af57ac" exitCode=0
Jul 29 14:31:20.171437 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:20.171271    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"
Jul 29 14:31:20.171437 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:20.171290    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s" event={"ID":"3358bfe7-94ae-4fd1-8089-325e6e40a669","Type":"ContainerDied","Data":"581fecf680c6e04714927d681e47a063930cdd6406589e801ef44299b1af57ac"}
Jul 29 14:31:20.171437 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:20.171340    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s" event={"ID":"3358bfe7-94ae-4fd1-8089-325e6e40a669","Type":"ContainerDied","Data":"092995ca5614796ac680c3a055725bab2696e9f52cfb471c4fcbf7b1584be99a"}
Jul 29 14:31:20.171437 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:20.171364    2559 scope.go:117] "RemoveContainer" containerID="581fecf680c6e04714927d681e47a063930cdd6406589e801ef44299b1af57ac"
Jul 29 14:31:20.179667 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:20.179607    2559 scope.go:117] "RemoveContainer" containerID="32e76109623ab7cef18ea42a7fd94894ffea5ec0cb6a236c49d5ee911f92322c"
Jul 29 14:31:20.187335 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:20.187317    2559 scope.go:117] "RemoveContainer" containerID="581fecf680c6e04714927d681e47a063930cdd6406589e801ef44299b1af57ac"
Jul 29 14:31:20.187583 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:31:20.187565    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"581fecf680c6e04714927d681e47a063930cdd6406589e801ef44299b1af57ac\": container with ID starting with 581fecf680c6e04714927d681e47a063930cdd6406589e801ef44299b1af57ac not found: ID does not exist" containerID="581fecf680c6e04714927d681e47a063930cdd6406589e801ef44299b1af57ac"
Jul 29 14:31:20.187622 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:20.187591    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"581fecf680c6e04714927d681e47a063930cdd6406589e801ef44299b1af57ac"} err="failed to get container status \"581fecf680c6e04714927d681e47a063930cdd6406589e801ef44299b1af57ac\": rpc error: code = NotFound desc = could not find container \"581fecf680c6e04714927d681e47a063930cdd6406589e801ef44299b1af57ac\": container with ID starting with 581fecf680c6e04714927d681e47a063930cdd6406589e801ef44299b1af57ac not found: ID does not exist"
Jul 29 14:31:20.187622 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:20.187607    2559 scope.go:117] "RemoveContainer" containerID="32e76109623ab7cef18ea42a7fd94894ffea5ec0cb6a236c49d5ee911f92322c"
Jul 29 14:31:20.187859 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:31:20.187841    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32e76109623ab7cef18ea42a7fd94894ffea5ec0cb6a236c49d5ee911f92322c\": container with ID starting with 32e76109623ab7cef18ea42a7fd94894ffea5ec0cb6a236c49d5ee911f92322c not found: ID does not exist" containerID="32e76109623ab7cef18ea42a7fd94894ffea5ec0cb6a236c49d5ee911f92322c"
Jul 29 14:31:20.187913 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:20.187864    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32e76109623ab7cef18ea42a7fd94894ffea5ec0cb6a236c49d5ee911f92322c"} err="failed to get container status \"32e76109623ab7cef18ea42a7fd94894ffea5ec0cb6a236c49d5ee911f92322c\": rpc error: code = NotFound desc = could not find container \"32e76109623ab7cef18ea42a7fd94894ffea5ec0cb6a236c49d5ee911f92322c\": container with ID starting with 32e76109623ab7cef18ea42a7fd94894ffea5ec0cb6a236c49d5ee911f92322c not found: ID does not exist"
Jul 29 14:31:20.193720 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:20.193695    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"]
Jul 29 14:31:20.200648 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:20.200627    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-330348dc-openshift-default-866f8c99d4-4ld9s"]
Jul 29 14:31:20.636289 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:20.636254    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3358bfe7-94ae-4fd1-8089-325e6e40a669" path="/var/lib/kubelet/pods/3358bfe7-94ae-4fd1-8089-325e6e40a669/volumes"
Jul 29 14:31:23.265854 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:23.265820    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:31:23.439292 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:23.439255    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:31:23.543329 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:23.543245    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:31:23.998092 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:23.998039    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:31:49.945329 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:49.945276    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:31:56.655379 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:56.655346    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:31:56.790053 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:56.790007    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:31:56.938307 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:31:56.938206    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:32:12.749467 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:32:12.749431    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:32:18.848232 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:32:18.848190    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:32:18.950442 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:32:18.950402    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:32:19.054193 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:32:19.054163    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:32:19.157755 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:32:19.157674    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:32:29.591350 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:32:29.591299    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:32:30.394998 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:32:30.394953    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:32:36.947894 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:32:36.947806    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:32:37.056898 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:32:37.056868    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:32:37.227748 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:32:37.227661    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:32:37.331627 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:32:37.331593    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:32:38.110134 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:32:38.110101    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:32:38.205028 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:32:38.204996    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:32:42.416007 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:32:42.415970    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:32:44.739229 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:32:44.739188    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:32:44.798657 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:32:44.798610    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:32:50.446367 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:32:50.446319    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:32:50.636586 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:32:50.636549    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:32:51.136675 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:32:51.136642    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:32:56.838455 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:32:56.838419    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:32:56.902234 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:32:56.902183    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:32:59.650215 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:32:59.650176    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:32:59.774538 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:32:59.774505    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:33:00.594747 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:00.594706    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:33:00.742288 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:00.742250    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh"]
Jul 29 14:33:00.742837 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:00.742689    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3358bfe7-94ae-4fd1-8089-325e6e40a669" containerName="istio-proxy"
Jul 29 14:33:00.742837 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:00.742709    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3358bfe7-94ae-4fd1-8089-325e6e40a669" containerName="istio-proxy"
Jul 29 14:33:00.742837 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:00.742724    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3358bfe7-94ae-4fd1-8089-325e6e40a669" containerName="istio-proxy"
Jul 29 14:33:00.742837 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:00.742733    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3358bfe7-94ae-4fd1-8089-325e6e40a669" containerName="istio-proxy"
Jul 29 14:33:00.742837 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:00.742760    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54a93fbc-be71-4060-aa28-78a7e6d8411b" containerName="maas-api"
Jul 29 14:33:00.742837 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:00.742769    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="54a93fbc-be71-4060-aa28-78a7e6d8411b" containerName="maas-api"
Jul 29 14:33:00.742837 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:00.742840    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3358bfe7-94ae-4fd1-8089-325e6e40a669" containerName="istio-proxy"
Jul 29 14:33:00.743216 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:00.742851    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3358bfe7-94ae-4fd1-8089-325e6e40a669" containerName="istio-proxy"
Jul 29 14:33:00.743216 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:00.742861    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="54a93fbc-be71-4060-aa28-78a7e6d8411b" containerName="maas-api"
Jul 29 14:33:00.745661 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:00.745639    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh"
Jul 29 14:33:00.750765 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:00.750742    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-10f19eac-serving-cert\""
Jul 29 14:33:00.752374 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:00.752352    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7jqfw\""
Jul 29 14:33:00.789309 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:00.789280    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh"]
Jul 29 14:33:00.790669 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:00.790646    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:33:00.876920 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:00.876828    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/245bca78-ac66-418a-a807-805e595a0e34-maas-api-tls\") pod \"maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh\" (UID: \"245bca78-ac66-418a-a807-805e595a0e34\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh"
Jul 29 14:33:00.876920 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:00.876862    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvbn9\" (UniqueName: \"kubernetes.io/projected/245bca78-ac66-418a-a807-805e595a0e34-kube-api-access-gvbn9\") pod \"maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh\" (UID: \"245bca78-ac66-418a-a807-805e595a0e34\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh"
Jul 29 14:33:00.877179 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:00.876935    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/245bca78-ac66-418a-a807-805e595a0e34-openshift-service-ca\") pod \"maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh\" (UID: \"245bca78-ac66-418a-a807-805e595a0e34\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh"
Jul 29 14:33:00.915067 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:00.915038    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:33:00.977762 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:00.977726    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/245bca78-ac66-418a-a807-805e595a0e34-maas-api-tls\") pod \"maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh\" (UID: \"245bca78-ac66-418a-a807-805e595a0e34\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh"
Jul 29 14:33:00.977923 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:00.977764    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvbn9\" (UniqueName: \"kubernetes.io/projected/245bca78-ac66-418a-a807-805e595a0e34-kube-api-access-gvbn9\") pod \"maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh\" (UID: \"245bca78-ac66-418a-a807-805e595a0e34\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh"
Jul 29 14:33:00.977923 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:00.977812    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/245bca78-ac66-418a-a807-805e595a0e34-openshift-service-ca\") pod \"maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh\" (UID: \"245bca78-ac66-418a-a807-805e595a0e34\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh"
Jul 29 14:33:00.978500 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:00.978475    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/245bca78-ac66-418a-a807-805e595a0e34-openshift-service-ca\") pod \"maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh\" (UID: \"245bca78-ac66-418a-a807-805e595a0e34\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh"
Jul 29 14:33:00.980120 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:00.980101    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/245bca78-ac66-418a-a807-805e595a0e34-maas-api-tls\") pod \"maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh\" (UID: \"245bca78-ac66-418a-a807-805e595a0e34\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh"
Jul 29 14:33:00.992928 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:00.992898    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvbn9\" (UniqueName: \"kubernetes.io/projected/245bca78-ac66-418a-a807-805e595a0e34-kube-api-access-gvbn9\") pod \"maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh\" (UID: \"245bca78-ac66-418a-a807-805e595a0e34\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh"
Jul 29 14:33:01.056132 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:01.056104    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh"
Jul 29 14:33:01.082757 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:01.082715    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:33:01.200893 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:01.200859    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh"]
Jul 29 14:33:01.204095 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:33:01.204053    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod245bca78_ac66_418a_a807_805e595a0e34.slice/crio-0ce7b00d68128874c97a8f90fd333c25812e14155a2cc7a36dcf2e758dcb0bc7 WatchSource:0}: Error finding container 0ce7b00d68128874c97a8f90fd333c25812e14155a2cc7a36dcf2e758dcb0bc7: Status 404 returned error can't find the container with id 0ce7b00d68128874c97a8f90fd333c25812e14155a2cc7a36dcf2e758dcb0bc7
Jul 29 14:33:01.600855 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:01.600817    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh" event={"ID":"245bca78-ac66-418a-a807-805e595a0e34","Type":"ContainerStarted","Data":"0ce7b00d68128874c97a8f90fd333c25812e14155a2cc7a36dcf2e758dcb0bc7"}
Jul 29 14:33:01.615461 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:01.615408    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:33:01.717053 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:01.717022    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:33:02.606252 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:02.606209    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh" event={"ID":"245bca78-ac66-418a-a807-805e595a0e34","Type":"ContainerStarted","Data":"453c7b8301a4a46298582adb613940e5bcb852f83402ade7c1669a394b413a0f"}
Jul 29 14:33:02.606660 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:02.606323    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh"
Jul 29 14:33:02.627673 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:02.627620    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh" podStartSLOduration=2.191620071 podStartE2EDuration="2.627606692s" totalImagesPullingTime="435.986621ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:33:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:33:02.626243599 +0000 UTC m=+1912.516914934" watchObservedRunningTime="2026-07-29 14:33:02.627606692 +0000 UTC m=+1912.518278026"
Jul 29 14:33:08.616648 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:08.616616    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh"
Jul 29 14:33:25.599702 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:25.599662    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:33:26.297854 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:26.297820    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:33:30.524282 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:30.524251    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:33:32.452035 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:32.451994    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:33:32.511197 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:32.511168    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:33:43.381943 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:43.381906    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh"]
Jul 29 14:33:43.382388 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:43.382165    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh" podUID="245bca78-ac66-418a-a807-805e595a0e34" containerName="maas-api" containerID="cri-o://453c7b8301a4a46298582adb613940e5bcb852f83402ade7c1669a394b413a0f" gracePeriod=30
Jul 29 14:33:43.410300 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:43.410266    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:33:43.478292 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:43.478260    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:33:43.577780 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:43.577748    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:33:43.627412 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:43.627387    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh"
Jul 29 14:33:43.630870 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:43.630847    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:33:43.739528 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:43.739500    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvbn9\" (UniqueName: \"kubernetes.io/projected/245bca78-ac66-418a-a807-805e595a0e34-kube-api-access-gvbn9\") pod \"245bca78-ac66-418a-a807-805e595a0e34\" (UID: \"245bca78-ac66-418a-a807-805e595a0e34\") "
Jul 29 14:33:43.739674 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:43.739570    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/245bca78-ac66-418a-a807-805e595a0e34-maas-api-tls\") pod \"245bca78-ac66-418a-a807-805e595a0e34\" (UID: \"245bca78-ac66-418a-a807-805e595a0e34\") "
Jul 29 14:33:43.739674 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:43.739622    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/245bca78-ac66-418a-a807-805e595a0e34-openshift-service-ca\") pod \"245bca78-ac66-418a-a807-805e595a0e34\" (UID: \"245bca78-ac66-418a-a807-805e595a0e34\") "
Jul 29 14:33:43.740153 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:43.740122    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/245bca78-ac66-418a-a807-805e595a0e34-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "245bca78-ac66-418a-a807-805e595a0e34" (UID: "245bca78-ac66-418a-a807-805e595a0e34"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 14:33:43.741658 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:43.741633    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/245bca78-ac66-418a-a807-805e595a0e34-kube-api-access-gvbn9" (OuterVolumeSpecName: "kube-api-access-gvbn9") pod "245bca78-ac66-418a-a807-805e595a0e34" (UID: "245bca78-ac66-418a-a807-805e595a0e34"). InnerVolumeSpecName "kube-api-access-gvbn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:33:43.741764 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:43.741658    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/245bca78-ac66-418a-a807-805e595a0e34-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "245bca78-ac66-418a-a807-805e595a0e34" (UID: "245bca78-ac66-418a-a807-805e595a0e34"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 14:33:43.772813 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:43.772785    2559 generic.go:356] "Generic (PLEG): container finished" podID="245bca78-ac66-418a-a807-805e595a0e34" containerID="453c7b8301a4a46298582adb613940e5bcb852f83402ade7c1669a394b413a0f" exitCode=0
Jul 29 14:33:43.772940 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:43.772830    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh"
Jul 29 14:33:43.772940 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:43.772849    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh" event={"ID":"245bca78-ac66-418a-a807-805e595a0e34","Type":"ContainerDied","Data":"453c7b8301a4a46298582adb613940e5bcb852f83402ade7c1669a394b413a0f"}
Jul 29 14:33:43.772940 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:43.772887    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh" event={"ID":"245bca78-ac66-418a-a807-805e595a0e34","Type":"ContainerDied","Data":"0ce7b00d68128874c97a8f90fd333c25812e14155a2cc7a36dcf2e758dcb0bc7"}
Jul 29 14:33:43.772940 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:43.772909    2559 scope.go:117] "RemoveContainer" containerID="453c7b8301a4a46298582adb613940e5bcb852f83402ade7c1669a394b413a0f"
Jul 29 14:33:43.781874 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:43.781855    2559 scope.go:117] "RemoveContainer" containerID="453c7b8301a4a46298582adb613940e5bcb852f83402ade7c1669a394b413a0f"
Jul 29 14:33:43.782234 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:33:43.782215    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"453c7b8301a4a46298582adb613940e5bcb852f83402ade7c1669a394b413a0f\": container with ID starting with 453c7b8301a4a46298582adb613940e5bcb852f83402ade7c1669a394b413a0f not found: ID does not exist" containerID="453c7b8301a4a46298582adb613940e5bcb852f83402ade7c1669a394b413a0f"
Jul 29 14:33:43.782308 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:43.782240    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"453c7b8301a4a46298582adb613940e5bcb852f83402ade7c1669a394b413a0f"} err="failed to get container status \"453c7b8301a4a46298582adb613940e5bcb852f83402ade7c1669a394b413a0f\": rpc error: code = NotFound desc = could not find container \"453c7b8301a4a46298582adb613940e5bcb852f83402ade7c1669a394b413a0f\": container with ID starting with 453c7b8301a4a46298582adb613940e5bcb852f83402ade7c1669a394b413a0f not found: ID does not exist"
Jul 29 14:33:43.795555 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:43.795527    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh"]
Jul 29 14:33:43.797587 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:43.797564    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh"]
Jul 29 14:33:43.841176 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:43.841157    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/245bca78-ac66-418a-a807-805e595a0e34-openshift-service-ca\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:33:43.841176 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:43.841175    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvbn9\" (UniqueName: \"kubernetes.io/projected/245bca78-ac66-418a-a807-805e595a0e34-kube-api-access-gvbn9\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:33:43.841287 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:43.841186    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/245bca78-ac66-418a-a807-805e595a0e34-maas-api-tls\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:33:43.969745 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:43.969710    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:33:44.636051 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:44.636017    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="245bca78-ac66-418a-a807-805e595a0e34" path="/var/lib/kubelet/pods/245bca78-ac66-418a-a807-805e595a0e34/volumes"
Jul 29 14:33:46.612386 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:46.612324    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-10f19eac-6b9f9998fc-n6bbh" podUID="245bca78-ac66-418a-a807-805e595a0e34" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.41:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 29 14:33:49.806634 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:49.806600    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:33:49.911617 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:49.911580    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:33:50.095250 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:50.095168    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:33:50.310186 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:50.310154    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:33:50.465219 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:50.465138    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:33:50.565305 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:50.565274    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:33:50.665118 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:50.665086    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:33:56.302123 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:56.302091    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:33:56.382931 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:56.382897    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:33:56.455810 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:56.455776    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:33:56.557132 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:56.557032    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:33:56.655868 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:33:56.655819    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:34:02.467683 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:02.467647    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:34:02.567602 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:02.567568    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:34:09.443976 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.443925    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:34:09.726960 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.726886    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"]
Jul 29 14:34:09.727212 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.727199    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="245bca78-ac66-418a-a807-805e595a0e34" containerName="maas-api"
Jul 29 14:34:09.727258 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.727214    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="245bca78-ac66-418a-a807-805e595a0e34" containerName="maas-api"
Jul 29 14:34:09.727292 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.727271    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="245bca78-ac66-418a-a807-805e595a0e34" containerName="maas-api"
Jul 29 14:34:09.731543 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.731525    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:09.734426 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.734401    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-c3fb6290-openshift-default-dockercfg-bkb98\""
Jul 29 14:34:09.742777 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.742753    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"]
Jul 29 14:34:09.761660 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.761635    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:34:09.866938 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.866903    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b0dc249-5fd7-4c59-a566-e6fe99e42073-workload-socket\") pod \"e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") " pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:09.867124 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.866947    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b0dc249-5fd7-4c59-a566-e6fe99e42073-istio-token\") pod \"e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") " pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:09.867124 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.866965    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kpxn\" (UniqueName: \"kubernetes.io/projected/8b0dc249-5fd7-4c59-a566-e6fe99e42073-kube-api-access-9kpxn\") pod \"e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") " pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:09.867124 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.867033    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b0dc249-5fd7-4c59-a566-e6fe99e42073-credential-socket\") pod \"e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") " pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:09.867124 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.867089    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b0dc249-5fd7-4c59-a566-e6fe99e42073-workload-certs\") pod \"e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") " pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:09.867124 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.867119    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b0dc249-5fd7-4c59-a566-e6fe99e42073-istio-data\") pod \"e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") " pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:09.867345 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.867135    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b0dc249-5fd7-4c59-a566-e6fe99e42073-istiod-ca-cert\") pod \"e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") " pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:09.867345 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.867207    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b0dc249-5fd7-4c59-a566-e6fe99e42073-istio-envoy\") pod \"e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") " pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:09.867345 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.867262    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b0dc249-5fd7-4c59-a566-e6fe99e42073-istio-podinfo\") pod \"e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") " pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:09.968188 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.968155    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b0dc249-5fd7-4c59-a566-e6fe99e42073-credential-socket\") pod \"e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") " pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:09.968188 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.968186    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b0dc249-5fd7-4c59-a566-e6fe99e42073-workload-certs\") pod \"e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") " pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:09.968440 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.968207    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b0dc249-5fd7-4c59-a566-e6fe99e42073-istio-data\") pod \"e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") " pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:09.968440 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.968224    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b0dc249-5fd7-4c59-a566-e6fe99e42073-istiod-ca-cert\") pod \"e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") " pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:09.968440 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.968251    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b0dc249-5fd7-4c59-a566-e6fe99e42073-istio-envoy\") pod \"e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") " pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:09.968440 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.968287    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b0dc249-5fd7-4c59-a566-e6fe99e42073-istio-podinfo\") pod \"e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") " pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:09.968672 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.968498    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b0dc249-5fd7-4c59-a566-e6fe99e42073-workload-socket\") pod \"e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") " pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:09.968672 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.968561    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b0dc249-5fd7-4c59-a566-e6fe99e42073-credential-socket\") pod \"e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") " pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:09.968672 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.968565    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b0dc249-5fd7-4c59-a566-e6fe99e42073-istio-token\") pod \"e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") " pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:09.968672 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.968630    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kpxn\" (UniqueName: \"kubernetes.io/projected/8b0dc249-5fd7-4c59-a566-e6fe99e42073-kube-api-access-9kpxn\") pod \"e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") " pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:09.968912 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.968670    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b0dc249-5fd7-4c59-a566-e6fe99e42073-istio-data\") pod \"e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") " pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:09.968912 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.968630    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b0dc249-5fd7-4c59-a566-e6fe99e42073-workload-certs\") pod \"e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") " pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:09.968912 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.968864    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b0dc249-5fd7-4c59-a566-e6fe99e42073-workload-socket\") pod \"e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") " pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:09.969026 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.968997    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b0dc249-5fd7-4c59-a566-e6fe99e42073-istiod-ca-cert\") pod \"e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") " pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:09.970571 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.970550    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b0dc249-5fd7-4c59-a566-e6fe99e42073-istio-envoy\") pod \"e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") " pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:09.970819 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.970803    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b0dc249-5fd7-4c59-a566-e6fe99e42073-istio-podinfo\") pod \"e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") " pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:09.977092 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.977008    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b0dc249-5fd7-4c59-a566-e6fe99e42073-istio-token\") pod \"e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") " pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:09.977363 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:09.977335    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kpxn\" (UniqueName: \"kubernetes.io/projected/8b0dc249-5fd7-4c59-a566-e6fe99e42073-kube-api-access-9kpxn\") pod \"e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") " pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:10.042281 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:10.042251    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:10.107730 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:10.107699    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:34:10.175079 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:10.175042    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"]
Jul 29 14:34:10.177526 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:34:10.177495    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b0dc249_5fd7_4c59_a566_e6fe99e42073.slice/crio-9319806431f270a80d247e87cac95057da42e1efb5e603468364fed2746867cc WatchSource:0}: Error finding container 9319806431f270a80d247e87cac95057da42e1efb5e603468364fed2746867cc: Status 404 returned error can't find the container with id 9319806431f270a80d247e87cac95057da42e1efb5e603468364fed2746867cc
Jul 29 14:34:10.179560 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:10.179524    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:34:10.179643 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:10.179589    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:34:10.179643 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:10.179625    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:34:10.888950 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:10.888910    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t" event={"ID":"8b0dc249-5fd7-4c59-a566-e6fe99e42073","Type":"ContainerStarted","Data":"6f910636758f1755435286aedbb195f08389d2be68400e42e4d05561794e4279"}
Jul 29 14:34:10.889407 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:10.888984    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t" event={"ID":"8b0dc249-5fd7-4c59-a566-e6fe99e42073","Type":"ContainerStarted","Data":"9319806431f270a80d247e87cac95057da42e1efb5e603468364fed2746867cc"}
Jul 29 14:34:10.911161 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:10.911114    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t" podStartSLOduration=1.911100019 podStartE2EDuration="1.911100019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:34:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:34:10.910103089 +0000 UTC m=+1980.800774424" watchObservedRunningTime="2026-07-29 14:34:10.911100019 +0000 UTC m=+1980.801771354"
Jul 29 14:34:11.043072 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:11.042982    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:11.047511 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:11.047488    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:11.893069 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:11.893022    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:11.893972 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:11.893952    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:12.879425 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:12.879382    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:34:12.987619 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:12.987578    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:34:13.078259 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:13.078225    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:34:13.487076 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:13.487027    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:34:13.645170 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:13.645134    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:34:13.748728 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:13.748694    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:34:26.894654 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:26.894619    2559 patch_prober.go:28] interesting pod/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 29 14:34:26.895095 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:26.894685    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t" podUID="8b0dc249-5fd7-4c59-a566-e6fe99e42073" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 29 14:34:26.958484 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:26.958448    2559 generic.go:356] "Generic (PLEG): container finished" podID="8b0dc249-5fd7-4c59-a566-e6fe99e42073" containerID="6f910636758f1755435286aedbb195f08389d2be68400e42e4d05561794e4279" exitCode=0
Jul 29 14:34:26.958645 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:26.958519    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t" event={"ID":"8b0dc249-5fd7-4c59-a566-e6fe99e42073","Type":"ContainerDied","Data":"6f910636758f1755435286aedbb195f08389d2be68400e42e4d05561794e4279"}
Jul 29 14:34:26.958921 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:26.958900    2559 scope.go:117] "RemoveContainer" containerID="6f910636758f1755435286aedbb195f08389d2be68400e42e4d05561794e4279"
Jul 29 14:34:26.959961 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:26.959930    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:34:26.960023 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:26.960002    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:34:26.960081 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:26.960041    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:34:27.964972 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:27.964938    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t" event={"ID":"8b0dc249-5fd7-4c59-a566-e6fe99e42073","Type":"ContainerStarted","Data":"b93e9b60380f3b778ea03114643a0ad3456d140d54b20aaae5c2d1dea4d97926"}
Jul 29 14:34:28.042447 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:28.042411    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:28.042634 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:28.042594    2559 patch_prober.go:28] interesting pod/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 29 14:34:28.042717 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:28.042643    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t" podUID="8b0dc249-5fd7-4c59-a566-e6fe99e42073" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 29 14:34:29.042671 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:29.042641    2559 patch_prober.go:28] interesting pod/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 29 14:34:29.043102 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:29.042692    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t" podUID="8b0dc249-5fd7-4c59-a566-e6fe99e42073" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 29 14:34:30.043069 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:30.043032    2559 patch_prober.go:28] interesting pod/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 29 14:34:30.043466 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:30.043104    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t" podUID="8b0dc249-5fd7-4c59-a566-e6fe99e42073" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 29 14:34:31.042656 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:31.042610    2559 patch_prober.go:28] interesting pod/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 29 14:34:31.042837 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:31.042678    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t" podUID="8b0dc249-5fd7-4c59-a566-e6fe99e42073" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 29 14:34:32.043266 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:32.043223    2559 patch_prober.go:28] interesting pod/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 29 14:34:32.043662 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:32.043292    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t" podUID="8b0dc249-5fd7-4c59-a566-e6fe99e42073" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 29 14:34:33.047254 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:33.047224    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:33.047663 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:33.047480    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:33.048211 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:33.048193    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:34:37.866716 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:37.866678    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:34:46.405823 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:46.405786    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:34:46.503980 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:46.503940    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:34:55.735735 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:55.735699    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k"]
Jul 29 14:34:55.738919 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:55.738900    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k"
Jul 29 14:34:55.741778 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:55.741751    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-c3fb6290\"/\"kube-root-ca.crt\""
Jul 29 14:34:55.743122 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:55.743100    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-c3fb6290\"/\"openshift-service-ca.crt\""
Jul 29 14:34:55.743262 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:55.743240    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-c3fb6290\"/\"default-dockercfg-vtbwx\""
Jul 29 14:34:55.743337 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:55.743263    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-c3fb6290\"/\"e2e-lifecycle-model-c3fb6290-kserve-self-signed-certs\""
Jul 29 14:34:55.749007 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:55.748984    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k"]
Jul 29 14:34:55.775089 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:55.775042    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:34:55.872120 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:55.872088    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9721ceec-05f4-498a-9129-07b8581f2440-dshm\") pod \"e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k\" (UID: \"9721ceec-05f4-498a-9129-07b8581f2440\") " pod="ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k"
Jul 29 14:34:55.872260 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:55.872126    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9721ceec-05f4-498a-9129-07b8581f2440-model-cache\") pod \"e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k\" (UID: \"9721ceec-05f4-498a-9129-07b8581f2440\") " pod="ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k"
Jul 29 14:34:55.872260 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:55.872155    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9721ceec-05f4-498a-9129-07b8581f2440-home\") pod \"e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k\" (UID: \"9721ceec-05f4-498a-9129-07b8581f2440\") " pod="ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k"
Jul 29 14:34:55.872260 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:55.872195    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5tt9\" (UniqueName: \"kubernetes.io/projected/9721ceec-05f4-498a-9129-07b8581f2440-kube-api-access-v5tt9\") pod \"e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k\" (UID: \"9721ceec-05f4-498a-9129-07b8581f2440\") " pod="ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k"
Jul 29 14:34:55.872260 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:55.872228    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9721ceec-05f4-498a-9129-07b8581f2440-tmp-dir\") pod \"e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k\" (UID: \"9721ceec-05f4-498a-9129-07b8581f2440\") " pod="ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k"
Jul 29 14:34:55.872397 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:55.872280    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9721ceec-05f4-498a-9129-07b8581f2440-tls-certs\") pod \"e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k\" (UID: \"9721ceec-05f4-498a-9129-07b8581f2440\") " pod="ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k"
Jul 29 14:34:55.973578 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:55.973548    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9721ceec-05f4-498a-9129-07b8581f2440-dshm\") pod \"e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k\" (UID: \"9721ceec-05f4-498a-9129-07b8581f2440\") " pod="ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k"
Jul 29 14:34:55.973739 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:55.973585    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9721ceec-05f4-498a-9129-07b8581f2440-model-cache\") pod \"e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k\" (UID: \"9721ceec-05f4-498a-9129-07b8581f2440\") " pod="ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k"
Jul 29 14:34:55.973739 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:55.973604    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9721ceec-05f4-498a-9129-07b8581f2440-home\") pod \"e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k\" (UID: \"9721ceec-05f4-498a-9129-07b8581f2440\") " pod="ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k"
Jul 29 14:34:55.973739 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:55.973625    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5tt9\" (UniqueName: \"kubernetes.io/projected/9721ceec-05f4-498a-9129-07b8581f2440-kube-api-access-v5tt9\") pod \"e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k\" (UID: \"9721ceec-05f4-498a-9129-07b8581f2440\") " pod="ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k"
Jul 29 14:34:55.973902 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:55.973753    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9721ceec-05f4-498a-9129-07b8581f2440-tmp-dir\") pod \"e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k\" (UID: \"9721ceec-05f4-498a-9129-07b8581f2440\") " pod="ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k"
Jul 29 14:34:55.973902 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:55.973811    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9721ceec-05f4-498a-9129-07b8581f2440-tls-certs\") pod \"e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k\" (UID: \"9721ceec-05f4-498a-9129-07b8581f2440\") " pod="ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k"
Jul 29 14:34:55.974036 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:55.974018    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9721ceec-05f4-498a-9129-07b8581f2440-model-cache\") pod \"e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k\" (UID: \"9721ceec-05f4-498a-9129-07b8581f2440\") " pod="ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k"
Jul 29 14:34:55.974127 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:55.974109    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9721ceec-05f4-498a-9129-07b8581f2440-home\") pod \"e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k\" (UID: \"9721ceec-05f4-498a-9129-07b8581f2440\") " pod="ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k"
Jul 29 14:34:55.974170 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:55.974126    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9721ceec-05f4-498a-9129-07b8581f2440-tmp-dir\") pod \"e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k\" (UID: \"9721ceec-05f4-498a-9129-07b8581f2440\") " pod="ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k"
Jul 29 14:34:55.975858 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:55.975826    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9721ceec-05f4-498a-9129-07b8581f2440-dshm\") pod \"e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k\" (UID: \"9721ceec-05f4-498a-9129-07b8581f2440\") " pod="ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k"
Jul 29 14:34:55.976148 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:55.976129    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9721ceec-05f4-498a-9129-07b8581f2440-tls-certs\") pod \"e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k\" (UID: \"9721ceec-05f4-498a-9129-07b8581f2440\") " pod="ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k"
Jul 29 14:34:55.982363 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:55.982339    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5tt9\" (UniqueName: \"kubernetes.io/projected/9721ceec-05f4-498a-9129-07b8581f2440-kube-api-access-v5tt9\") pod \"e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k\" (UID: \"9721ceec-05f4-498a-9129-07b8581f2440\") " pod="ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k"
Jul 29 14:34:56.049246 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:56.049204    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k"
Jul 29 14:34:56.184166 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:56.179995    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k"]
Jul 29 14:34:56.187386 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:34:56.187357    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9721ceec_05f4_498a_9129_07b8581f2440.slice/crio-3f08f53b1cb48071d8244f6762c3cbf2a4a5df5208e5acb6afc390e43ede821f WatchSource:0}: Error finding container 3f08f53b1cb48071d8244f6762c3cbf2a4a5df5208e5acb6afc390e43ede821f: Status 404 returned error can't find the container with id 3f08f53b1cb48071d8244f6762c3cbf2a4a5df5208e5acb6afc390e43ede821f
Jul 29 14:34:57.091744 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:57.091711    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k" event={"ID":"9721ceec-05f4-498a-9129-07b8581f2440","Type":"ContainerStarted","Data":"3c8824a8870b5a7d95c52c59450a42c859a4d658ada0a84b1d53a4b3998d6493"}
Jul 29 14:34:57.091744 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:57.091751    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k" event={"ID":"9721ceec-05f4-498a-9129-07b8581f2440","Type":"ContainerStarted","Data":"3f08f53b1cb48071d8244f6762c3cbf2a4a5df5208e5acb6afc390e43ede821f"}
Jul 29 14:34:57.110982 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:34:57.110926    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k" podStartSLOduration=2.11091225 podStartE2EDuration="2.11091225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:34:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:34:57.109755211 +0000 UTC m=+2027.000426558" watchObservedRunningTime="2026-07-29 14:34:57.11091225 +0000 UTC m=+2027.001583584"
Jul 29 14:35:06.049959 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:35:06.049925    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k"
Jul 29 14:35:06.050377 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:35:06.049970    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k"
Jul 29 14:35:06.062390 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:35:06.062352    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k"
Jul 29 14:35:06.140960 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:35:06.140930    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k"
Jul 29 14:35:14.330563 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:35:14.330522    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:35:48.412102 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:35:48.412001    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:35:48.511752 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:35:48.511717    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:35:48.599571 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:35:48.599533    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:35:51.880349 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:35:51.880315    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:35:56.838401 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:35:56.838363    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k"]
Jul 29 14:35:56.934427 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:35:56.934394    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:36:10.699511 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:10.699480    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdw8l_b93f6faa-c8bf-43e9-94e0-b3dcd97b7296/ovn-acl-logging/0.log"
Jul 29 14:36:10.706678 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:10.706657    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdw8l_b93f6faa-c8bf-43e9-94e0-b3dcd97b7296/ovn-acl-logging/0.log"
Jul 29 14:36:11.856965 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:11.856922    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k" podUID="9721ceec-05f4-498a-9129-07b8581f2440" containerName="main" containerID="cri-o://3c8824a8870b5a7d95c52c59450a42c859a4d658ada0a84b1d53a4b3998d6493" gracePeriod=45
Jul 29 14:36:12.098320 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:12.098294    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k"
Jul 29 14:36:12.223190 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:12.223097    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9721ceec-05f4-498a-9129-07b8581f2440-tmp-dir\") pod \"9721ceec-05f4-498a-9129-07b8581f2440\" (UID: \"9721ceec-05f4-498a-9129-07b8581f2440\") "
Jul 29 14:36:12.223190 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:12.223137    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9721ceec-05f4-498a-9129-07b8581f2440-home\") pod \"9721ceec-05f4-498a-9129-07b8581f2440\" (UID: \"9721ceec-05f4-498a-9129-07b8581f2440\") "
Jul 29 14:36:12.223425 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:12.223194    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9721ceec-05f4-498a-9129-07b8581f2440-model-cache\") pod \"9721ceec-05f4-498a-9129-07b8581f2440\" (UID: \"9721ceec-05f4-498a-9129-07b8581f2440\") "
Jul 29 14:36:12.223425 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:12.223213    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9721ceec-05f4-498a-9129-07b8581f2440-tls-certs\") pod \"9721ceec-05f4-498a-9129-07b8581f2440\" (UID: \"9721ceec-05f4-498a-9129-07b8581f2440\") "
Jul 29 14:36:12.223425 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:12.223246    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v5tt9\" (UniqueName: \"kubernetes.io/projected/9721ceec-05f4-498a-9129-07b8581f2440-kube-api-access-v5tt9\") pod \"9721ceec-05f4-498a-9129-07b8581f2440\" (UID: \"9721ceec-05f4-498a-9129-07b8581f2440\") "
Jul 29 14:36:12.223425 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:12.223271    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9721ceec-05f4-498a-9129-07b8581f2440-dshm\") pod \"9721ceec-05f4-498a-9129-07b8581f2440\" (UID: \"9721ceec-05f4-498a-9129-07b8581f2440\") "
Jul 29 14:36:12.223610 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:12.223432    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9721ceec-05f4-498a-9129-07b8581f2440-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9721ceec-05f4-498a-9129-07b8581f2440" (UID: "9721ceec-05f4-498a-9129-07b8581f2440"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:36:12.223610 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:12.223468    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9721ceec-05f4-498a-9129-07b8581f2440-model-cache" (OuterVolumeSpecName: "model-cache") pod "9721ceec-05f4-498a-9129-07b8581f2440" (UID: "9721ceec-05f4-498a-9129-07b8581f2440"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:36:12.223610 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:12.223515    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9721ceec-05f4-498a-9129-07b8581f2440-home" (OuterVolumeSpecName: "home") pod "9721ceec-05f4-498a-9129-07b8581f2440" (UID: "9721ceec-05f4-498a-9129-07b8581f2440"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:36:12.225613 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:12.225586    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9721ceec-05f4-498a-9129-07b8581f2440-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9721ceec-05f4-498a-9129-07b8581f2440" (UID: "9721ceec-05f4-498a-9129-07b8581f2440"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 14:36:12.225743 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:12.225607    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9721ceec-05f4-498a-9129-07b8581f2440-kube-api-access-v5tt9" (OuterVolumeSpecName: "kube-api-access-v5tt9") pod "9721ceec-05f4-498a-9129-07b8581f2440" (UID: "9721ceec-05f4-498a-9129-07b8581f2440"). InnerVolumeSpecName "kube-api-access-v5tt9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:36:12.225743 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:12.225599    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9721ceec-05f4-498a-9129-07b8581f2440-dshm" (OuterVolumeSpecName: "dshm") pod "9721ceec-05f4-498a-9129-07b8581f2440" (UID: "9721ceec-05f4-498a-9129-07b8581f2440"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:36:12.324465 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:12.324437    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9721ceec-05f4-498a-9129-07b8581f2440-dshm\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:36:12.324465 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:12.324462    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9721ceec-05f4-498a-9129-07b8581f2440-tmp-dir\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:36:12.324465 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:12.324472    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9721ceec-05f4-498a-9129-07b8581f2440-home\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:36:12.324671 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:12.324480    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9721ceec-05f4-498a-9129-07b8581f2440-model-cache\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:36:12.324671 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:12.324488    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9721ceec-05f4-498a-9129-07b8581f2440-tls-certs\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:36:12.324671 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:12.324500    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5tt9\" (UniqueName: \"kubernetes.io/projected/9721ceec-05f4-498a-9129-07b8581f2440-kube-api-access-v5tt9\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:36:12.416508 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:12.416469    2559 generic.go:356] "Generic (PLEG): container finished" podID="9721ceec-05f4-498a-9129-07b8581f2440" containerID="3c8824a8870b5a7d95c52c59450a42c859a4d658ada0a84b1d53a4b3998d6493" exitCode=0
Jul 29 14:36:12.416669 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:12.416527    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k"
Jul 29 14:36:12.416669 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:12.416557    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k" event={"ID":"9721ceec-05f4-498a-9129-07b8581f2440","Type":"ContainerDied","Data":"3c8824a8870b5a7d95c52c59450a42c859a4d658ada0a84b1d53a4b3998d6493"}
Jul 29 14:36:12.416669 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:12.416600    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k" event={"ID":"9721ceec-05f4-498a-9129-07b8581f2440","Type":"ContainerDied","Data":"3f08f53b1cb48071d8244f6762c3cbf2a4a5df5208e5acb6afc390e43ede821f"}
Jul 29 14:36:12.416669 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:12.416617    2559 scope.go:117] "RemoveContainer" containerID="3c8824a8870b5a7d95c52c59450a42c859a4d658ada0a84b1d53a4b3998d6493"
Jul 29 14:36:12.425123 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:12.425107    2559 scope.go:117] "RemoveContainer" containerID="3c8824a8870b5a7d95c52c59450a42c859a4d658ada0a84b1d53a4b3998d6493"
Jul 29 14:36:12.425363 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:36:12.425343    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c8824a8870b5a7d95c52c59450a42c859a4d658ada0a84b1d53a4b3998d6493\": container with ID starting with 3c8824a8870b5a7d95c52c59450a42c859a4d658ada0a84b1d53a4b3998d6493 not found: ID does not exist" containerID="3c8824a8870b5a7d95c52c59450a42c859a4d658ada0a84b1d53a4b3998d6493"
Jul 29 14:36:12.425417 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:12.425371    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c8824a8870b5a7d95c52c59450a42c859a4d658ada0a84b1d53a4b3998d6493"} err="failed to get container status \"3c8824a8870b5a7d95c52c59450a42c859a4d658ada0a84b1d53a4b3998d6493\": rpc error: code = NotFound desc = could not find container \"3c8824a8870b5a7d95c52c59450a42c859a4d658ada0a84b1d53a4b3998d6493\": container with ID starting with 3c8824a8870b5a7d95c52c59450a42c859a4d658ada0a84b1d53a4b3998d6493 not found: ID does not exist"
Jul 29 14:36:12.438184 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:12.438159    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k"]
Jul 29 14:36:12.441383 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:12.441361    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-c3fb6290/e2e-lifecycle-model-c3fb6290-kserve-fc6b68cff-mxg8k"]
Jul 29 14:36:12.636416 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:12.636388    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9721ceec-05f4-498a-9129-07b8581f2440" path="/var/lib/kubelet/pods/9721ceec-05f4-498a-9129-07b8581f2440/volumes"
Jul 29 14:36:33.106483 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:33.106448    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:36:34.107702 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:34.107657    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:36:39.898922 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:39.898886    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:36:39.998714 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:39.998680    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:36:40.174381 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:40.174302    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"]
Jul 29 14:36:40.174557 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:40.174534    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t" podUID="8b0dc249-5fd7-4c59-a566-e6fe99e42073" containerName="istio-proxy" containerID="cri-o://b93e9b60380f3b778ea03114643a0ad3456d140d54b20aaae5c2d1dea4d97926" gracePeriod=30
Jul 29 14:36:40.191863 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:40.191828    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:36:40.418949 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:40.418911    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:36:41.502830 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:41.502796    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:36:41.650800 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:41.650770    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:36:41.776720 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:41.776631    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:36:42.461154 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:42.461118    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:36:42.552906 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:42.552877    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:36:42.610592 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:42.610565    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:36:44.113163 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:44.113129    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:36:45.421991 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.421968    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:36:45.556112 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.556081    2559 generic.go:356] "Generic (PLEG): container finished" podID="8b0dc249-5fd7-4c59-a566-e6fe99e42073" containerID="b93e9b60380f3b778ea03114643a0ad3456d140d54b20aaae5c2d1dea4d97926" exitCode=0
Jul 29 14:36:45.556278 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.556141    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"
Jul 29 14:36:45.556278 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.556146    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t" event={"ID":"8b0dc249-5fd7-4c59-a566-e6fe99e42073","Type":"ContainerDied","Data":"b93e9b60380f3b778ea03114643a0ad3456d140d54b20aaae5c2d1dea4d97926"}
Jul 29 14:36:45.556278 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.556211    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t" event={"ID":"8b0dc249-5fd7-4c59-a566-e6fe99e42073","Type":"ContainerDied","Data":"9319806431f270a80d247e87cac95057da42e1efb5e603468364fed2746867cc"}
Jul 29 14:36:45.556278 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.556233    2559 scope.go:117] "RemoveContainer" containerID="b93e9b60380f3b778ea03114643a0ad3456d140d54b20aaae5c2d1dea4d97926"
Jul 29 14:36:45.566134 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.566113    2559 scope.go:117] "RemoveContainer" containerID="6f910636758f1755435286aedbb195f08389d2be68400e42e4d05561794e4279"
Jul 29 14:36:45.573578 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.573560    2559 scope.go:117] "RemoveContainer" containerID="b93e9b60380f3b778ea03114643a0ad3456d140d54b20aaae5c2d1dea4d97926"
Jul 29 14:36:45.573826 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:36:45.573807    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b93e9b60380f3b778ea03114643a0ad3456d140d54b20aaae5c2d1dea4d97926\": container with ID starting with b93e9b60380f3b778ea03114643a0ad3456d140d54b20aaae5c2d1dea4d97926 not found: ID does not exist" containerID="b93e9b60380f3b778ea03114643a0ad3456d140d54b20aaae5c2d1dea4d97926"
Jul 29 14:36:45.573882 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.573834    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b93e9b60380f3b778ea03114643a0ad3456d140d54b20aaae5c2d1dea4d97926"} err="failed to get container status \"b93e9b60380f3b778ea03114643a0ad3456d140d54b20aaae5c2d1dea4d97926\": rpc error: code = NotFound desc = could not find container \"b93e9b60380f3b778ea03114643a0ad3456d140d54b20aaae5c2d1dea4d97926\": container with ID starting with b93e9b60380f3b778ea03114643a0ad3456d140d54b20aaae5c2d1dea4d97926 not found: ID does not exist"
Jul 29 14:36:45.573882 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.573851    2559 scope.go:117] "RemoveContainer" containerID="6f910636758f1755435286aedbb195f08389d2be68400e42e4d05561794e4279"
Jul 29 14:36:45.574068 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:36:45.574042    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f910636758f1755435286aedbb195f08389d2be68400e42e4d05561794e4279\": container with ID starting with 6f910636758f1755435286aedbb195f08389d2be68400e42e4d05561794e4279 not found: ID does not exist" containerID="6f910636758f1755435286aedbb195f08389d2be68400e42e4d05561794e4279"
Jul 29 14:36:45.574117 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.574080    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f910636758f1755435286aedbb195f08389d2be68400e42e4d05561794e4279"} err="failed to get container status \"6f910636758f1755435286aedbb195f08389d2be68400e42e4d05561794e4279\": rpc error: code = NotFound desc = could not find container \"6f910636758f1755435286aedbb195f08389d2be68400e42e4d05561794e4279\": container with ID starting with 6f910636758f1755435286aedbb195f08389d2be68400e42e4d05561794e4279 not found: ID does not exist"
Jul 29 14:36:45.609869 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.609845    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b0dc249-5fd7-4c59-a566-e6fe99e42073-istio-token\") pod \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") "
Jul 29 14:36:45.609970 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.609882    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b0dc249-5fd7-4c59-a566-e6fe99e42073-istio-envoy\") pod \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") "
Jul 29 14:36:45.609970 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.609913    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b0dc249-5fd7-4c59-a566-e6fe99e42073-istio-podinfo\") pod \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") "
Jul 29 14:36:45.609970 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.609931    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b0dc249-5fd7-4c59-a566-e6fe99e42073-istiod-ca-cert\") pod \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") "
Jul 29 14:36:45.609970 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.609954    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9kpxn\" (UniqueName: \"kubernetes.io/projected/8b0dc249-5fd7-4c59-a566-e6fe99e42073-kube-api-access-9kpxn\") pod \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") "
Jul 29 14:36:45.609970 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.609970    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b0dc249-5fd7-4c59-a566-e6fe99e42073-workload-certs\") pod \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") "
Jul 29 14:36:45.610264 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.610001    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b0dc249-5fd7-4c59-a566-e6fe99e42073-credential-socket\") pod \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") "
Jul 29 14:36:45.610264 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.610032    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b0dc249-5fd7-4c59-a566-e6fe99e42073-istio-data\") pod \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") "
Jul 29 14:36:45.610264 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.610110    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b0dc249-5fd7-4c59-a566-e6fe99e42073-workload-socket\") pod \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\" (UID: \"8b0dc249-5fd7-4c59-a566-e6fe99e42073\") "
Jul 29 14:36:45.610439 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.610331    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b0dc249-5fd7-4c59-a566-e6fe99e42073-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8b0dc249-5fd7-4c59-a566-e6fe99e42073" (UID: "8b0dc249-5fd7-4c59-a566-e6fe99e42073"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:36:45.610439 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.610393    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b0dc249-5fd7-4c59-a566-e6fe99e42073-credential-socket\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:36:45.610545 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.610418    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b0dc249-5fd7-4c59-a566-e6fe99e42073-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8b0dc249-5fd7-4c59-a566-e6fe99e42073" (UID: "8b0dc249-5fd7-4c59-a566-e6fe99e42073"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 14:36:45.610603 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.610543    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b0dc249-5fd7-4c59-a566-e6fe99e42073-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8b0dc249-5fd7-4c59-a566-e6fe99e42073" (UID: "8b0dc249-5fd7-4c59-a566-e6fe99e42073"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:36:45.610876 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.610849    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b0dc249-5fd7-4c59-a566-e6fe99e42073-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8b0dc249-5fd7-4c59-a566-e6fe99e42073" (UID: "8b0dc249-5fd7-4c59-a566-e6fe99e42073"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:36:45.611157 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.611128    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b0dc249-5fd7-4c59-a566-e6fe99e42073-istio-data" (OuterVolumeSpecName: "istio-data") pod "8b0dc249-5fd7-4c59-a566-e6fe99e42073" (UID: "8b0dc249-5fd7-4c59-a566-e6fe99e42073"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:36:45.612147 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.612125    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b0dc249-5fd7-4c59-a566-e6fe99e42073-istio-token" (OuterVolumeSpecName: "istio-token") pod "8b0dc249-5fd7-4c59-a566-e6fe99e42073" (UID: "8b0dc249-5fd7-4c59-a566-e6fe99e42073"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:36:45.612147 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.612118    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8b0dc249-5fd7-4c59-a566-e6fe99e42073-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8b0dc249-5fd7-4c59-a566-e6fe99e42073" (UID: "8b0dc249-5fd7-4c59-a566-e6fe99e42073"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 14:36:45.612272 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.612155    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b0dc249-5fd7-4c59-a566-e6fe99e42073-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8b0dc249-5fd7-4c59-a566-e6fe99e42073" (UID: "8b0dc249-5fd7-4c59-a566-e6fe99e42073"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:36:45.612313 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.612275    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b0dc249-5fd7-4c59-a566-e6fe99e42073-kube-api-access-9kpxn" (OuterVolumeSpecName: "kube-api-access-9kpxn") pod "8b0dc249-5fd7-4c59-a566-e6fe99e42073" (UID: "8b0dc249-5fd7-4c59-a566-e6fe99e42073"). InnerVolumeSpecName "kube-api-access-9kpxn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:36:45.711617 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.711585    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b0dc249-5fd7-4c59-a566-e6fe99e42073-istio-podinfo\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:36:45.711617 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.711612    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b0dc249-5fd7-4c59-a566-e6fe99e42073-istiod-ca-cert\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:36:45.711617 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.711621    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9kpxn\" (UniqueName: \"kubernetes.io/projected/8b0dc249-5fd7-4c59-a566-e6fe99e42073-kube-api-access-9kpxn\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:36:45.711835 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.711630    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b0dc249-5fd7-4c59-a566-e6fe99e42073-workload-certs\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:36:45.711835 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.711639    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b0dc249-5fd7-4c59-a566-e6fe99e42073-istio-data\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:36:45.711835 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.711648    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b0dc249-5fd7-4c59-a566-e6fe99e42073-workload-socket\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:36:45.711835 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.711656    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b0dc249-5fd7-4c59-a566-e6fe99e42073-istio-token\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:36:45.711835 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.711663    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b0dc249-5fd7-4c59-a566-e6fe99e42073-istio-envoy\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:36:45.882467 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.882438    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"]
Jul 29 14:36:45.885318 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:45.885289    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-c3fb6290-openshift-default-698cd767c6-vgx6t"]
Jul 29 14:36:46.635377 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:46.635345    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b0dc249-5fd7-4c59-a566-e6fe99e42073" path="/var/lib/kubelet/pods/8b0dc249-5fd7-4c59-a566-e6fe99e42073/volumes"
Jul 29 14:36:49.796515 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:49.796484    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:36:49.901997 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:49.901965    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:36:50.505149 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:50.505110    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:36:57.241711 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:57.241672    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:36:57.355792 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:57.355759    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:36:57.443708 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:36:57.443678    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:37:07.967585 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:07.967508    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:37:08.310504 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:08.310474    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:37:29.282160 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:29.282110    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:37:31.336033 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:31.335993    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:37:35.757955 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:35.757923    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:37:35.896997 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:35.896963    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:37:36.100989 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:36.100906    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:37:36.413724 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:36.413647    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:37:36.520812 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:36.520779    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:37:39.408854 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:39.408813    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:37:39.507571 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:39.507539    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-055496-c7f84b588-5tdq4"]
Jul 29 14:37:39.507868 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:39.507856    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9721ceec-05f4-498a-9129-07b8581f2440" containerName="main"
Jul 29 14:37:39.507912 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:39.507869    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9721ceec-05f4-498a-9129-07b8581f2440" containerName="main"
Jul 29 14:37:39.507912 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:39.507880    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b0dc249-5fd7-4c59-a566-e6fe99e42073" containerName="istio-proxy"
Jul 29 14:37:39.507912 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:39.507886    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b0dc249-5fd7-4c59-a566-e6fe99e42073" containerName="istio-proxy"
Jul 29 14:37:39.507912 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:39.507897    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b0dc249-5fd7-4c59-a566-e6fe99e42073" containerName="istio-proxy"
Jul 29 14:37:39.507912 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:39.507903    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b0dc249-5fd7-4c59-a566-e6fe99e42073" containerName="istio-proxy"
Jul 29 14:37:39.508075 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:39.507946    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b0dc249-5fd7-4c59-a566-e6fe99e42073" containerName="istio-proxy"
Jul 29 14:37:39.508075 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:39.507954    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b0dc249-5fd7-4c59-a566-e6fe99e42073" containerName="istio-proxy"
Jul 29 14:37:39.508075 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:39.507962    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9721ceec-05f4-498a-9129-07b8581f2440" containerName="main"
Jul 29 14:37:39.510751 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:39.510733    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-055496-c7f84b588-5tdq4"
Jul 29 14:37:39.511135 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:39.511107    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:37:39.513747 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:39.513724    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-055496-serving-cert\""
Jul 29 14:37:39.513873 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:39.513851    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7jqfw\""
Jul 29 14:37:39.521470 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:39.521449    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-055496-c7f84b588-5tdq4"]
Jul 29 14:37:39.611281 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:39.611253    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:37:39.661362 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:39.661284    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl7dz\" (UniqueName: \"kubernetes.io/projected/a0d73721-ea56-4540-9cff-827db21cc230-kube-api-access-cl7dz\") pod \"maas-api-e2e-inf-a-055496-c7f84b588-5tdq4\" (UID: \"a0d73721-ea56-4540-9cff-827db21cc230\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-055496-c7f84b588-5tdq4"
Jul 29 14:37:39.661362 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:39.661348    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0d73721-ea56-4540-9cff-827db21cc230-openshift-service-ca\") pod \"maas-api-e2e-inf-a-055496-c7f84b588-5tdq4\" (UID: \"a0d73721-ea56-4540-9cff-827db21cc230\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-055496-c7f84b588-5tdq4"
Jul 29 14:37:39.661531 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:39.661390    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0d73721-ea56-4540-9cff-827db21cc230-maas-api-tls\") pod \"maas-api-e2e-inf-a-055496-c7f84b588-5tdq4\" (UID: \"a0d73721-ea56-4540-9cff-827db21cc230\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-055496-c7f84b588-5tdq4"
Jul 29 14:37:39.759029 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:39.758995    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:37:39.762140 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:39.762117    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0d73721-ea56-4540-9cff-827db21cc230-maas-api-tls\") pod \"maas-api-e2e-inf-a-055496-c7f84b588-5tdq4\" (UID: \"a0d73721-ea56-4540-9cff-827db21cc230\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-055496-c7f84b588-5tdq4"
Jul 29 14:37:39.762202 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:39.762172    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cl7dz\" (UniqueName: \"kubernetes.io/projected/a0d73721-ea56-4540-9cff-827db21cc230-kube-api-access-cl7dz\") pod \"maas-api-e2e-inf-a-055496-c7f84b588-5tdq4\" (UID: \"a0d73721-ea56-4540-9cff-827db21cc230\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-055496-c7f84b588-5tdq4"
Jul 29 14:37:39.762239 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:39.762233    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0d73721-ea56-4540-9cff-827db21cc230-openshift-service-ca\") pod \"maas-api-e2e-inf-a-055496-c7f84b588-5tdq4\" (UID: \"a0d73721-ea56-4540-9cff-827db21cc230\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-055496-c7f84b588-5tdq4"
Jul 29 14:37:39.762839 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:39.762820    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0d73721-ea56-4540-9cff-827db21cc230-openshift-service-ca\") pod \"maas-api-e2e-inf-a-055496-c7f84b588-5tdq4\" (UID: \"a0d73721-ea56-4540-9cff-827db21cc230\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-055496-c7f84b588-5tdq4"
Jul 29 14:37:39.764553 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:39.764534    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0d73721-ea56-4540-9cff-827db21cc230-maas-api-tls\") pod \"maas-api-e2e-inf-a-055496-c7f84b588-5tdq4\" (UID: \"a0d73721-ea56-4540-9cff-827db21cc230\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-055496-c7f84b588-5tdq4"
Jul 29 14:37:39.772615 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:39.772595    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl7dz\" (UniqueName: \"kubernetes.io/projected/a0d73721-ea56-4540-9cff-827db21cc230-kube-api-access-cl7dz\") pod \"maas-api-e2e-inf-a-055496-c7f84b588-5tdq4\" (UID: \"a0d73721-ea56-4540-9cff-827db21cc230\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-055496-c7f84b588-5tdq4"
Jul 29 14:37:39.821338 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:39.821312    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-055496-c7f84b588-5tdq4"
Jul 29 14:37:39.951953 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:39.951848    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-055496-c7f84b588-5tdq4"]
Jul 29 14:37:39.954401 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:37:39.954375    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0d73721_ea56_4540_9cff_827db21cc230.slice/crio-348793fb351871029c8edfb5f5ac8cb7877872a11cb30c76a1dbe8fcb9ebe818 WatchSource:0}: Error finding container 348793fb351871029c8edfb5f5ac8cb7877872a11cb30c76a1dbe8fcb9ebe818: Status 404 returned error can't find the container with id 348793fb351871029c8edfb5f5ac8cb7877872a11cb30c76a1dbe8fcb9ebe818
Jul 29 14:37:39.955858 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:39.955837    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 14:37:40.118238 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:40.118207    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:37:40.218681 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:40.218594    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:37:40.775741 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:40.775702    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-055496-c7f84b588-5tdq4" event={"ID":"a0d73721-ea56-4540-9cff-827db21cc230","Type":"ContainerStarted","Data":"e60a78621d4ad0d725556974311df21e3f2856e556dae54507b268c12e47288c"}
Jul 29 14:37:40.776134 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:40.775762    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-055496-c7f84b588-5tdq4" event={"ID":"a0d73721-ea56-4540-9cff-827db21cc230","Type":"ContainerStarted","Data":"348793fb351871029c8edfb5f5ac8cb7877872a11cb30c76a1dbe8fcb9ebe818"}
Jul 29 14:37:40.776134 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:40.775799    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-055496-c7f84b588-5tdq4"
Jul 29 14:37:40.794408 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:40.794364    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-055496-c7f84b588-5tdq4" podStartSLOduration=1.276299958 podStartE2EDuration="1.794347811s" totalImagesPullingTime="518.047853ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:37:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:37:40.793884085 +0000 UTC m=+2190.684555421" watchObservedRunningTime="2026-07-29 14:37:40.794347811 +0000 UTC m=+2190.685019169"
Jul 29 14:37:46.786509 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:37:46.786477    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-055496-c7f84b588-5tdq4"
Jul 29 14:38:04.446883 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:04.446840    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:38:04.775848 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:04.775820    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:38:05.053167 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.052533    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"]
Jul 29 14:38:05.057282 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.057075    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:05.059925 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.059898    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-02f676-openshift-default-dockercfg-4t2m2\""
Jul 29 14:38:05.065957 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.065923    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"]
Jul 29 14:38:05.084168 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.084141    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:38:05.177435 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.177404    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aafde015-1d18-4771-b22a-723d039fc0e2-istio-data\") pod \"e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") " pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:05.177591 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.177452    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aafde015-1d18-4771-b22a-723d039fc0e2-istiod-ca-cert\") pod \"e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") " pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:05.177591 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.177511    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j474d\" (UniqueName: \"kubernetes.io/projected/aafde015-1d18-4771-b22a-723d039fc0e2-kube-api-access-j474d\") pod \"e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") " pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:05.177591 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.177548    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aafde015-1d18-4771-b22a-723d039fc0e2-istio-podinfo\") pod \"e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") " pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:05.177696 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.177592    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aafde015-1d18-4771-b22a-723d039fc0e2-workload-certs\") pod \"e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") " pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:05.177696 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.177612    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aafde015-1d18-4771-b22a-723d039fc0e2-istio-token\") pod \"e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") " pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:05.177696 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.177656    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aafde015-1d18-4771-b22a-723d039fc0e2-workload-socket\") pod \"e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") " pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:05.177696 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.177672    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aafde015-1d18-4771-b22a-723d039fc0e2-credential-socket\") pod \"e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") " pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:05.177828 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.177702    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aafde015-1d18-4771-b22a-723d039fc0e2-istio-envoy\") pod \"e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") " pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:05.278961 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.278917    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aafde015-1d18-4771-b22a-723d039fc0e2-istiod-ca-cert\") pod \"e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") " pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:05.278961 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.278964    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j474d\" (UniqueName: \"kubernetes.io/projected/aafde015-1d18-4771-b22a-723d039fc0e2-kube-api-access-j474d\") pod \"e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") " pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:05.279240 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.279112    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aafde015-1d18-4771-b22a-723d039fc0e2-istio-podinfo\") pod \"e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") " pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:05.279240 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.279159    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aafde015-1d18-4771-b22a-723d039fc0e2-workload-certs\") pod \"e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") " pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:05.279358 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.279287    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aafde015-1d18-4771-b22a-723d039fc0e2-istio-token\") pod \"e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") " pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:05.279404 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.279363    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aafde015-1d18-4771-b22a-723d039fc0e2-workload-socket\") pod \"e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") " pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:05.279404 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.279391    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aafde015-1d18-4771-b22a-723d039fc0e2-credential-socket\") pod \"e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") " pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:05.279501 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.279446    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aafde015-1d18-4771-b22a-723d039fc0e2-istio-envoy\") pod \"e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") " pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:05.279501 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.279476    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aafde015-1d18-4771-b22a-723d039fc0e2-workload-certs\") pod \"e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") " pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:05.279501 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.279490    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aafde015-1d18-4771-b22a-723d039fc0e2-istio-data\") pod \"e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") " pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:05.279727 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.279696    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aafde015-1d18-4771-b22a-723d039fc0e2-istiod-ca-cert\") pod \"e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") " pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:05.279852 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.279726    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aafde015-1d18-4771-b22a-723d039fc0e2-workload-socket\") pod \"e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") " pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:05.279852 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.279822    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aafde015-1d18-4771-b22a-723d039fc0e2-credential-socket\") pod \"e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") " pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:05.279942 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.279893    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aafde015-1d18-4771-b22a-723d039fc0e2-istio-data\") pod \"e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") " pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:05.281569 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.281548    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aafde015-1d18-4771-b22a-723d039fc0e2-istio-envoy\") pod \"e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") " pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:05.281661 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.281617    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aafde015-1d18-4771-b22a-723d039fc0e2-istio-podinfo\") pod \"e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") " pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:05.288181 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.288155    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aafde015-1d18-4771-b22a-723d039fc0e2-istio-token\") pod \"e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") " pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:05.288363 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.288345    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j474d\" (UniqueName: \"kubernetes.io/projected/aafde015-1d18-4771-b22a-723d039fc0e2-kube-api-access-j474d\") pod \"e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") " pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:05.368981 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.368886    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:05.502444 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.502418    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"]
Jul 29 14:38:05.504447 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:38:05.504415    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaafde015_1d18_4771_b22a_723d039fc0e2.slice/crio-6133058918dd80be9d77424216a31944688f8fb203752477e8de6fac7032fc36 WatchSource:0}: Error finding container 6133058918dd80be9d77424216a31944688f8fb203752477e8de6fac7032fc36: Status 404 returned error can't find the container with id 6133058918dd80be9d77424216a31944688f8fb203752477e8de6fac7032fc36
Jul 29 14:38:05.506558 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.506530    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:38:05.506627 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.506592    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:38:05.506627 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.506621    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:38:05.699917 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.699833    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:38:05.882338 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.882301    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5" event={"ID":"aafde015-1d18-4771-b22a-723d039fc0e2","Type":"ContainerStarted","Data":"4f4075e2628b762d5fa9aa6756b9420c0ea0c9f4d7478eef3c7a112ff4a30c05"}
Jul 29 14:38:05.882508 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.882357    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5" event={"ID":"aafde015-1d18-4771-b22a-723d039fc0e2","Type":"ContainerStarted","Data":"6133058918dd80be9d77424216a31944688f8fb203752477e8de6fac7032fc36"}
Jul 29 14:38:05.906011 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:05.905961    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5" podStartSLOduration=0.905946465 podStartE2EDuration="905.946465ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:38:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:38:05.904309652 +0000 UTC m=+2215.794980986" watchObservedRunningTime="2026-07-29 14:38:05.905946465 +0000 UTC m=+2215.796617799"
Jul 29 14:38:06.369541 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:06.369505    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:06.374377 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:06.374346    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:06.886739 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:06.886707    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:06.887796 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:06.887777    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:07.820735 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:07.820683    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:38:07.912754 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:07.912723    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:38:08.149593 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:08.149495    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:38:08.249965 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:08.249933    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:38:08.650137 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:08.650104    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:38:08.746811 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:08.746773    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:38:22.928232 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:22.928195    2559 patch_prober.go:28] interesting pod/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 29 14:38:22.929543 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:22.928266    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5" podUID="aafde015-1d18-4771-b22a-723d039fc0e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 29 14:38:23.961580 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:23.961548    2559 generic.go:356] "Generic (PLEG): container finished" podID="aafde015-1d18-4771-b22a-723d039fc0e2" containerID="4f4075e2628b762d5fa9aa6756b9420c0ea0c9f4d7478eef3c7a112ff4a30c05" exitCode=0
Jul 29 14:38:23.962078 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:23.961628    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5" event={"ID":"aafde015-1d18-4771-b22a-723d039fc0e2","Type":"ContainerDied","Data":"4f4075e2628b762d5fa9aa6756b9420c0ea0c9f4d7478eef3c7a112ff4a30c05"}
Jul 29 14:38:23.962185 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:23.962172    2559 scope.go:117] "RemoveContainer" containerID="4f4075e2628b762d5fa9aa6756b9420c0ea0c9f4d7478eef3c7a112ff4a30c05"
Jul 29 14:38:23.963167 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:23.963140    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:38:23.963231 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:23.963206    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:38:23.963277 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:23.963237    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:38:24.970850 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:24.970805    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5" event={"ID":"aafde015-1d18-4771-b22a-723d039fc0e2","Type":"ContainerStarted","Data":"575b18b076e3e9909c37603773213b81f21286032fd4fed9dd5f455b4aca0d98"}
Jul 29 14:38:25.370071 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:25.370013    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:25.370290 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:25.370161    2559 patch_prober.go:28] interesting pod/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Jul 29 14:38:25.370290 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:25.370219    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5" podUID="aafde015-1d18-4771-b22a-723d039fc0e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Jul 29 14:38:26.369705 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:26.369660    2559 patch_prober.go:28] interesting pod/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Jul 29 14:38:26.370240 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:26.369726    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5" podUID="aafde015-1d18-4771-b22a-723d039fc0e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Jul 29 14:38:27.369928 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:27.369886    2559 patch_prober.go:28] interesting pod/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Jul 29 14:38:27.370584 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:27.370544    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5" podUID="aafde015-1d18-4771-b22a-723d039fc0e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Jul 29 14:38:28.369833 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:28.369380    2559 patch_prober.go:28] interesting pod/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Jul 29 14:38:28.369833 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:28.369459    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5" podUID="aafde015-1d18-4771-b22a-723d039fc0e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Jul 29 14:38:29.372986 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:29.372955    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:29.373397 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:29.373173    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:29.373922 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:29.373905    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:38:32.871560 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:32.871480    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:38:51.346516 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:51.346478    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7"]
Jul 29 14:38:51.348436 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:51.348418    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7"
Jul 29 14:38:51.351155 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:51.351133    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-055496\"/\"openshift-service-ca.crt\""
Jul 29 14:38:51.352448 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:51.352425    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-055496\"/\"kube-root-ca.crt\""
Jul 29 14:38:51.352572 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:51.352426    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-055496\"/\"default-dockercfg-6rxw5\""
Jul 29 14:38:51.352572 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:51.352465    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-055496\"/\"test-model-055496-kserve-self-signed-certs\""
Jul 29 14:38:51.359248 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:51.359225    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7"]
Jul 29 14:38:51.439235 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:51.439203    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:38:51.479256 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:51.479220    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a900204-9a2f-4327-9fc2-a20b23629928-dshm\") pod \"test-model-055496-kserve-7d6b6c4f5f-bcww7\" (UID: \"8a900204-9a2f-4327-9fc2-a20b23629928\") " pod="ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7"
Jul 29 14:38:51.479414 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:51.479271    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a900204-9a2f-4327-9fc2-a20b23629928-home\") pod \"test-model-055496-kserve-7d6b6c4f5f-bcww7\" (UID: \"8a900204-9a2f-4327-9fc2-a20b23629928\") " pod="ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7"
Jul 29 14:38:51.479414 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:51.479295    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a900204-9a2f-4327-9fc2-a20b23629928-model-cache\") pod \"test-model-055496-kserve-7d6b6c4f5f-bcww7\" (UID: \"8a900204-9a2f-4327-9fc2-a20b23629928\") " pod="ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7"
Jul 29 14:38:51.479486 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:51.479405    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9tsd\" (UniqueName: \"kubernetes.io/projected/8a900204-9a2f-4327-9fc2-a20b23629928-kube-api-access-m9tsd\") pod \"test-model-055496-kserve-7d6b6c4f5f-bcww7\" (UID: \"8a900204-9a2f-4327-9fc2-a20b23629928\") " pod="ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7"
Jul 29 14:38:51.479486 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:51.479451    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a900204-9a2f-4327-9fc2-a20b23629928-tmp-dir\") pod \"test-model-055496-kserve-7d6b6c4f5f-bcww7\" (UID: \"8a900204-9a2f-4327-9fc2-a20b23629928\") " pod="ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7"
Jul 29 14:38:51.479486 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:51.479470    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a900204-9a2f-4327-9fc2-a20b23629928-tls-certs\") pod \"test-model-055496-kserve-7d6b6c4f5f-bcww7\" (UID: \"8a900204-9a2f-4327-9fc2-a20b23629928\") " pod="ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7"
Jul 29 14:38:51.580868 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:51.580831    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9tsd\" (UniqueName: \"kubernetes.io/projected/8a900204-9a2f-4327-9fc2-a20b23629928-kube-api-access-m9tsd\") pod \"test-model-055496-kserve-7d6b6c4f5f-bcww7\" (UID: \"8a900204-9a2f-4327-9fc2-a20b23629928\") " pod="ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7"
Jul 29 14:38:51.580868 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:51.580869    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a900204-9a2f-4327-9fc2-a20b23629928-tmp-dir\") pod \"test-model-055496-kserve-7d6b6c4f5f-bcww7\" (UID: \"8a900204-9a2f-4327-9fc2-a20b23629928\") " pod="ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7"
Jul 29 14:38:51.581104 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:51.580888    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a900204-9a2f-4327-9fc2-a20b23629928-tls-certs\") pod \"test-model-055496-kserve-7d6b6c4f5f-bcww7\" (UID: \"8a900204-9a2f-4327-9fc2-a20b23629928\") " pod="ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7"
Jul 29 14:38:51.581104 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:51.580937    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a900204-9a2f-4327-9fc2-a20b23629928-dshm\") pod \"test-model-055496-kserve-7d6b6c4f5f-bcww7\" (UID: \"8a900204-9a2f-4327-9fc2-a20b23629928\") " pod="ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7"
Jul 29 14:38:51.581104 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:51.580983    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a900204-9a2f-4327-9fc2-a20b23629928-home\") pod \"test-model-055496-kserve-7d6b6c4f5f-bcww7\" (UID: \"8a900204-9a2f-4327-9fc2-a20b23629928\") " pod="ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7"
Jul 29 14:38:51.581104 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:51.581003    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a900204-9a2f-4327-9fc2-a20b23629928-model-cache\") pod \"test-model-055496-kserve-7d6b6c4f5f-bcww7\" (UID: \"8a900204-9a2f-4327-9fc2-a20b23629928\") " pod="ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7"
Jul 29 14:38:51.581352 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:51.581330    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a900204-9a2f-4327-9fc2-a20b23629928-tmp-dir\") pod \"test-model-055496-kserve-7d6b6c4f5f-bcww7\" (UID: \"8a900204-9a2f-4327-9fc2-a20b23629928\") " pod="ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7"
Jul 29 14:38:51.581405 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:51.581347    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a900204-9a2f-4327-9fc2-a20b23629928-home\") pod \"test-model-055496-kserve-7d6b6c4f5f-bcww7\" (UID: \"8a900204-9a2f-4327-9fc2-a20b23629928\") " pod="ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7"
Jul 29 14:38:51.581446 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:51.581434    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a900204-9a2f-4327-9fc2-a20b23629928-model-cache\") pod \"test-model-055496-kserve-7d6b6c4f5f-bcww7\" (UID: \"8a900204-9a2f-4327-9fc2-a20b23629928\") " pod="ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7"
Jul 29 14:38:51.583306 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:51.583284    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a900204-9a2f-4327-9fc2-a20b23629928-dshm\") pod \"test-model-055496-kserve-7d6b6c4f5f-bcww7\" (UID: \"8a900204-9a2f-4327-9fc2-a20b23629928\") " pod="ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7"
Jul 29 14:38:51.583482 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:51.583465    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a900204-9a2f-4327-9fc2-a20b23629928-tls-certs\") pod \"test-model-055496-kserve-7d6b6c4f5f-bcww7\" (UID: \"8a900204-9a2f-4327-9fc2-a20b23629928\") " pod="ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7"
Jul 29 14:38:51.589575 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:51.589552    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9tsd\" (UniqueName: \"kubernetes.io/projected/8a900204-9a2f-4327-9fc2-a20b23629928-kube-api-access-m9tsd\") pod \"test-model-055496-kserve-7d6b6c4f5f-bcww7\" (UID: \"8a900204-9a2f-4327-9fc2-a20b23629928\") " pod="ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7"
Jul 29 14:38:51.657924 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:51.657840    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7"
Jul 29 14:38:51.785256 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:51.785225    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7"]
Jul 29 14:38:51.787114 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:38:51.787082    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a900204_9a2f_4327_9fc2_a20b23629928.slice/crio-7fb0f0da679a3f9fd0be4b3c0670a326b6e6c38cf3651c444d5f0966c89a9cdf WatchSource:0}: Error finding container 7fb0f0da679a3f9fd0be4b3c0670a326b6e6c38cf3651c444d5f0966c89a9cdf: Status 404 returned error can't find the container with id 7fb0f0da679a3f9fd0be4b3c0670a326b6e6c38cf3651c444d5f0966c89a9cdf
Jul 29 14:38:52.091829 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:52.091790    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7" event={"ID":"8a900204-9a2f-4327-9fc2-a20b23629928","Type":"ContainerStarted","Data":"0d72bf8e13d9151c0481eec9ac2b72e70c37ec7148d1f8dada4ba8d71095f0e5"}
Jul 29 14:38:52.091829 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:52.091843    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7" event={"ID":"8a900204-9a2f-4327-9fc2-a20b23629928","Type":"ContainerStarted","Data":"7fb0f0da679a3f9fd0be4b3c0670a326b6e6c38cf3651c444d5f0966c89a9cdf"}
Jul 29 14:38:52.110633 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:38:52.110585    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7" podStartSLOduration=1.110571363 podStartE2EDuration="1.110571363s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:38:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:38:52.109543899 +0000 UTC m=+2262.000215231" watchObservedRunningTime="2026-07-29 14:38:52.110571363 +0000 UTC m=+2262.001242698"
Jul 29 14:39:01.658171 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:01.658133    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7"
Jul 29 14:39:01.658645 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:01.658385    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7"
Jul 29 14:39:01.670871 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:01.670846    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7"
Jul 29 14:39:02.146439 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:02.146400    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7"
Jul 29 14:39:19.850226 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:19.850191    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff"]
Jul 29 14:39:19.852218 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:19.852201    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff"
Jul 29 14:39:19.856909 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:19.856884    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-02f676\"/\"openshift-service-ca.crt\""
Jul 29 14:39:19.857025 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:19.856885    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-02f676\"/\"default-dockercfg-rzd4n\""
Jul 29 14:39:19.857025 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:19.856884    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-02f676\"/\"kube-root-ca.crt\""
Jul 29 14:39:19.857025 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:19.856942    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-02f676\"/\"test-model-02f676-kserve-self-signed-certs\""
Jul 29 14:39:19.865519 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:19.865494    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff"]
Jul 29 14:39:19.887410 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:19.887381    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:39:19.927340 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:19.927315    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bea54895-501a-4e04-88ca-b309bf6e8cac-model-cache\") pod \"test-model-02f676-kserve-b7857b6cf-jtsff\" (UID: \"bea54895-501a-4e04-88ca-b309bf6e8cac\") " pod="ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff"
Jul 29 14:39:19.927482 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:19.927342    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bea54895-501a-4e04-88ca-b309bf6e8cac-tmp-dir\") pod \"test-model-02f676-kserve-b7857b6cf-jtsff\" (UID: \"bea54895-501a-4e04-88ca-b309bf6e8cac\") " pod="ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff"
Jul 29 14:39:19.927482 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:19.927362    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sxpl\" (UniqueName: \"kubernetes.io/projected/bea54895-501a-4e04-88ca-b309bf6e8cac-kube-api-access-5sxpl\") pod \"test-model-02f676-kserve-b7857b6cf-jtsff\" (UID: \"bea54895-501a-4e04-88ca-b309bf6e8cac\") " pod="ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff"
Jul 29 14:39:19.927604 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:19.927472    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bea54895-501a-4e04-88ca-b309bf6e8cac-home\") pod \"test-model-02f676-kserve-b7857b6cf-jtsff\" (UID: \"bea54895-501a-4e04-88ca-b309bf6e8cac\") " pod="ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff"
Jul 29 14:39:19.927604 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:19.927566    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bea54895-501a-4e04-88ca-b309bf6e8cac-tls-certs\") pod \"test-model-02f676-kserve-b7857b6cf-jtsff\" (UID: \"bea54895-501a-4e04-88ca-b309bf6e8cac\") " pod="ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff"
Jul 29 14:39:19.927702 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:19.927613    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bea54895-501a-4e04-88ca-b309bf6e8cac-dshm\") pod \"test-model-02f676-kserve-b7857b6cf-jtsff\" (UID: \"bea54895-501a-4e04-88ca-b309bf6e8cac\") " pod="ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff"
Jul 29 14:39:20.028076 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:20.028020    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bea54895-501a-4e04-88ca-b309bf6e8cac-dshm\") pod \"test-model-02f676-kserve-b7857b6cf-jtsff\" (UID: \"bea54895-501a-4e04-88ca-b309bf6e8cac\") " pod="ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff"
Jul 29 14:39:20.028076 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:20.028078    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bea54895-501a-4e04-88ca-b309bf6e8cac-model-cache\") pod \"test-model-02f676-kserve-b7857b6cf-jtsff\" (UID: \"bea54895-501a-4e04-88ca-b309bf6e8cac\") " pod="ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff"
Jul 29 14:39:20.028324 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:20.028184    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bea54895-501a-4e04-88ca-b309bf6e8cac-tmp-dir\") pod \"test-model-02f676-kserve-b7857b6cf-jtsff\" (UID: \"bea54895-501a-4e04-88ca-b309bf6e8cac\") " pod="ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff"
Jul 29 14:39:20.028324 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:20.028221    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sxpl\" (UniqueName: \"kubernetes.io/projected/bea54895-501a-4e04-88ca-b309bf6e8cac-kube-api-access-5sxpl\") pod \"test-model-02f676-kserve-b7857b6cf-jtsff\" (UID: \"bea54895-501a-4e04-88ca-b309bf6e8cac\") " pod="ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff"
Jul 29 14:39:20.028432 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:20.028341    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bea54895-501a-4e04-88ca-b309bf6e8cac-home\") pod \"test-model-02f676-kserve-b7857b6cf-jtsff\" (UID: \"bea54895-501a-4e04-88ca-b309bf6e8cac\") " pod="ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff"
Jul 29 14:39:20.028432 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:20.028420    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bea54895-501a-4e04-88ca-b309bf6e8cac-model-cache\") pod \"test-model-02f676-kserve-b7857b6cf-jtsff\" (UID: \"bea54895-501a-4e04-88ca-b309bf6e8cac\") " pod="ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff"
Jul 29 14:39:20.028560 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:20.028457    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bea54895-501a-4e04-88ca-b309bf6e8cac-tls-certs\") pod \"test-model-02f676-kserve-b7857b6cf-jtsff\" (UID: \"bea54895-501a-4e04-88ca-b309bf6e8cac\") " pod="ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff"
Jul 29 14:39:20.028560 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:20.028548    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bea54895-501a-4e04-88ca-b309bf6e8cac-tmp-dir\") pod \"test-model-02f676-kserve-b7857b6cf-jtsff\" (UID: \"bea54895-501a-4e04-88ca-b309bf6e8cac\") " pod="ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff"
Jul 29 14:39:20.028763 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:20.028739    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bea54895-501a-4e04-88ca-b309bf6e8cac-home\") pod \"test-model-02f676-kserve-b7857b6cf-jtsff\" (UID: \"bea54895-501a-4e04-88ca-b309bf6e8cac\") " pod="ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff"
Jul 29 14:39:20.030402 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:20.030373    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bea54895-501a-4e04-88ca-b309bf6e8cac-dshm\") pod \"test-model-02f676-kserve-b7857b6cf-jtsff\" (UID: \"bea54895-501a-4e04-88ca-b309bf6e8cac\") " pod="ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff"
Jul 29 14:39:20.030658 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:20.030641    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bea54895-501a-4e04-88ca-b309bf6e8cac-tls-certs\") pod \"test-model-02f676-kserve-b7857b6cf-jtsff\" (UID: \"bea54895-501a-4e04-88ca-b309bf6e8cac\") " pod="ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff"
Jul 29 14:39:20.037004 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:20.036986    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sxpl\" (UniqueName: \"kubernetes.io/projected/bea54895-501a-4e04-88ca-b309bf6e8cac-kube-api-access-5sxpl\") pod \"test-model-02f676-kserve-b7857b6cf-jtsff\" (UID: \"bea54895-501a-4e04-88ca-b309bf6e8cac\") " pod="ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff"
Jul 29 14:39:20.161382 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:20.161288    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff"
Jul 29 14:39:20.290132 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:20.290095    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff"]
Jul 29 14:39:20.293303 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:39:20.293275    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbea54895_501a_4e04_88ca_b309bf6e8cac.slice/crio-6e71fb8ba0d80ce1a65a0721f4de4fb695b5284eb2dbada1d71df0f9a3f771aa WatchSource:0}: Error finding container 6e71fb8ba0d80ce1a65a0721f4de4fb695b5284eb2dbada1d71df0f9a3f771aa: Status 404 returned error can't find the container with id 6e71fb8ba0d80ce1a65a0721f4de4fb695b5284eb2dbada1d71df0f9a3f771aa
Jul 29 14:39:21.225863 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:21.225827    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff" event={"ID":"bea54895-501a-4e04-88ca-b309bf6e8cac","Type":"ContainerStarted","Data":"3436be19f43cd08d6708411dfb4bcf0f3448351b0de9afa6868ccaab95e6ba3b"}
Jul 29 14:39:21.225863 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:21.225866    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff" event={"ID":"bea54895-501a-4e04-88ca-b309bf6e8cac","Type":"ContainerStarted","Data":"6e71fb8ba0d80ce1a65a0721f4de4fb695b5284eb2dbada1d71df0f9a3f771aa"}
Jul 29 14:39:21.247047 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:21.246996    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff" podStartSLOduration=2.246980841 podStartE2EDuration="2.246980841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:39:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:39:21.245849455 +0000 UTC m=+2291.136520789" watchObservedRunningTime="2026-07-29 14:39:21.246980841 +0000 UTC m=+2291.137652222"
Jul 29 14:39:30.161480 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:30.161442    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff"
Jul 29 14:39:30.161931 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:30.161496    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff"
Jul 29 14:39:30.174097 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:30.174070    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff"
Jul 29 14:39:30.274922 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:30.274893    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff"
Jul 29 14:39:57.828271 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:57.828238    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7"]
Jul 29 14:39:57.843149 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:57.843120    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:39:58.789658 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:58.789626    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff"]
Jul 29 14:39:58.818049 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:39:58.818015    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:40:09.566766 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:09.566671    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-055496-c7f84b588-5tdq4"]
Jul 29 14:40:09.567316 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:09.566936    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-055496-c7f84b588-5tdq4" podUID="a0d73721-ea56-4540-9cff-827db21cc230" containerName="maas-api" containerID="cri-o://e60a78621d4ad0d725556974311df21e3f2856e556dae54507b268c12e47288c" gracePeriod=30
Jul 29 14:40:09.594938 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:09.594909    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:40:09.694302 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:09.694271    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:40:09.791108 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:09.791050    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:40:09.826287 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:09.826230    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-055496-c7f84b588-5tdq4"
Jul 29 14:40:09.850386 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:09.850357    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:40:09.867269 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:09.867245    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0d73721-ea56-4540-9cff-827db21cc230-openshift-service-ca\") pod \"a0d73721-ea56-4540-9cff-827db21cc230\" (UID: \"a0d73721-ea56-4540-9cff-827db21cc230\") "
Jul 29 14:40:09.867414 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:09.867281    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cl7dz\" (UniqueName: \"kubernetes.io/projected/a0d73721-ea56-4540-9cff-827db21cc230-kube-api-access-cl7dz\") pod \"a0d73721-ea56-4540-9cff-827db21cc230\" (UID: \"a0d73721-ea56-4540-9cff-827db21cc230\") "
Jul 29 14:40:09.867414 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:09.867318    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0d73721-ea56-4540-9cff-827db21cc230-maas-api-tls\") pod \"a0d73721-ea56-4540-9cff-827db21cc230\" (UID: \"a0d73721-ea56-4540-9cff-827db21cc230\") "
Jul 29 14:40:09.867653 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:09.867626    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0d73721-ea56-4540-9cff-827db21cc230-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a0d73721-ea56-4540-9cff-827db21cc230" (UID: "a0d73721-ea56-4540-9cff-827db21cc230"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 14:40:09.869455 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:09.869427    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0d73721-ea56-4540-9cff-827db21cc230-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a0d73721-ea56-4540-9cff-827db21cc230" (UID: "a0d73721-ea56-4540-9cff-827db21cc230"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 14:40:09.869574 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:09.869476    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0d73721-ea56-4540-9cff-827db21cc230-kube-api-access-cl7dz" (OuterVolumeSpecName: "kube-api-access-cl7dz") pod "a0d73721-ea56-4540-9cff-827db21cc230" (UID: "a0d73721-ea56-4540-9cff-827db21cc230"). InnerVolumeSpecName "kube-api-access-cl7dz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:40:09.968591 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:09.968558    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0d73721-ea56-4540-9cff-827db21cc230-openshift-service-ca\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:40:09.968591 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:09.968586    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cl7dz\" (UniqueName: \"kubernetes.io/projected/a0d73721-ea56-4540-9cff-827db21cc230-kube-api-access-cl7dz\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:40:09.968764 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:09.968600    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0d73721-ea56-4540-9cff-827db21cc230-maas-api-tls\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:40:10.314011 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:10.313975    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:40:10.439924 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:10.439894    2559 generic.go:356] "Generic (PLEG): container finished" podID="a0d73721-ea56-4540-9cff-827db21cc230" containerID="e60a78621d4ad0d725556974311df21e3f2856e556dae54507b268c12e47288c" exitCode=0
Jul 29 14:40:10.440106 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:10.439955    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-055496-c7f84b588-5tdq4"
Jul 29 14:40:10.440106 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:10.439980    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-055496-c7f84b588-5tdq4" event={"ID":"a0d73721-ea56-4540-9cff-827db21cc230","Type":"ContainerDied","Data":"e60a78621d4ad0d725556974311df21e3f2856e556dae54507b268c12e47288c"}
Jul 29 14:40:10.440106 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:10.440070    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-055496-c7f84b588-5tdq4" event={"ID":"a0d73721-ea56-4540-9cff-827db21cc230","Type":"ContainerDied","Data":"348793fb351871029c8edfb5f5ac8cb7877872a11cb30c76a1dbe8fcb9ebe818"}
Jul 29 14:40:10.440106 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:10.440083    2559 scope.go:117] "RemoveContainer" containerID="e60a78621d4ad0d725556974311df21e3f2856e556dae54507b268c12e47288c"
Jul 29 14:40:10.449080 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:10.449047    2559 scope.go:117] "RemoveContainer" containerID="e60a78621d4ad0d725556974311df21e3f2856e556dae54507b268c12e47288c"
Jul 29 14:40:10.449318 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:40:10.449298    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e60a78621d4ad0d725556974311df21e3f2856e556dae54507b268c12e47288c\": container with ID starting with e60a78621d4ad0d725556974311df21e3f2856e556dae54507b268c12e47288c not found: ID does not exist" containerID="e60a78621d4ad0d725556974311df21e3f2856e556dae54507b268c12e47288c"
Jul 29 14:40:10.449369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:10.449324    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e60a78621d4ad0d725556974311df21e3f2856e556dae54507b268c12e47288c"} err="failed to get container status \"e60a78621d4ad0d725556974311df21e3f2856e556dae54507b268c12e47288c\": rpc error: code = NotFound desc = could not find container \"e60a78621d4ad0d725556974311df21e3f2856e556dae54507b268c12e47288c\": container with ID starting with e60a78621d4ad0d725556974311df21e3f2856e556dae54507b268c12e47288c not found: ID does not exist"
Jul 29 14:40:10.462690 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:10.462669    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-055496-c7f84b588-5tdq4"]
Jul 29 14:40:10.466830 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:10.466806    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-055496-c7f84b588-5tdq4"]
Jul 29 14:40:10.638000 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:10.637915    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0d73721-ea56-4540-9cff-827db21cc230" path="/var/lib/kubelet/pods/a0d73721-ea56-4540-9cff-827db21cc230/volumes"
Jul 29 14:40:12.640950 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:12.640915    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:40:12.846799 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:12.846756    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7" podUID="8a900204-9a2f-4327-9fc2-a20b23629928" containerName="main" containerID="cri-o://0d72bf8e13d9151c0481eec9ac2b72e70c37ec7148d1f8dada4ba8d71095f0e5" gracePeriod=45
Jul 29 14:40:13.101199 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:13.101175    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7"
Jul 29 14:40:13.196725 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:13.196694    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a900204-9a2f-4327-9fc2-a20b23629928-home\") pod \"8a900204-9a2f-4327-9fc2-a20b23629928\" (UID: \"8a900204-9a2f-4327-9fc2-a20b23629928\") "
Jul 29 14:40:13.196917 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:13.196741    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a900204-9a2f-4327-9fc2-a20b23629928-dshm\") pod \"8a900204-9a2f-4327-9fc2-a20b23629928\" (UID: \"8a900204-9a2f-4327-9fc2-a20b23629928\") "
Jul 29 14:40:13.196917 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:13.196768    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a900204-9a2f-4327-9fc2-a20b23629928-tmp-dir\") pod \"8a900204-9a2f-4327-9fc2-a20b23629928\" (UID: \"8a900204-9a2f-4327-9fc2-a20b23629928\") "
Jul 29 14:40:13.196917 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:13.196792    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a900204-9a2f-4327-9fc2-a20b23629928-model-cache\") pod \"8a900204-9a2f-4327-9fc2-a20b23629928\" (UID: \"8a900204-9a2f-4327-9fc2-a20b23629928\") "
Jul 29 14:40:13.196917 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:13.196810    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a900204-9a2f-4327-9fc2-a20b23629928-tls-certs\") pod \"8a900204-9a2f-4327-9fc2-a20b23629928\" (UID: \"8a900204-9a2f-4327-9fc2-a20b23629928\") "
Jul 29 14:40:13.196917 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:13.196828    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m9tsd\" (UniqueName: \"kubernetes.io/projected/8a900204-9a2f-4327-9fc2-a20b23629928-kube-api-access-m9tsd\") pod \"8a900204-9a2f-4327-9fc2-a20b23629928\" (UID: \"8a900204-9a2f-4327-9fc2-a20b23629928\") "
Jul 29 14:40:13.197243 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:13.196995    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a900204-9a2f-4327-9fc2-a20b23629928-home" (OuterVolumeSpecName: "home") pod "8a900204-9a2f-4327-9fc2-a20b23629928" (UID: "8a900204-9a2f-4327-9fc2-a20b23629928"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:40:13.197243 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:13.197102    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a900204-9a2f-4327-9fc2-a20b23629928-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8a900204-9a2f-4327-9fc2-a20b23629928" (UID: "8a900204-9a2f-4327-9fc2-a20b23629928"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:40:13.197243 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:13.197113    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a900204-9a2f-4327-9fc2-a20b23629928-model-cache" (OuterVolumeSpecName: "model-cache") pod "8a900204-9a2f-4327-9fc2-a20b23629928" (UID: "8a900204-9a2f-4327-9fc2-a20b23629928"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:40:13.197243 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:13.197182    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a900204-9a2f-4327-9fc2-a20b23629928-home\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:40:13.198988 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:13.198962    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a900204-9a2f-4327-9fc2-a20b23629928-kube-api-access-m9tsd" (OuterVolumeSpecName: "kube-api-access-m9tsd") pod "8a900204-9a2f-4327-9fc2-a20b23629928" (UID: "8a900204-9a2f-4327-9fc2-a20b23629928"). InnerVolumeSpecName "kube-api-access-m9tsd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:40:13.199100 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:13.198988    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a900204-9a2f-4327-9fc2-a20b23629928-dshm" (OuterVolumeSpecName: "dshm") pod "8a900204-9a2f-4327-9fc2-a20b23629928" (UID: "8a900204-9a2f-4327-9fc2-a20b23629928"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:40:13.199100 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:13.198988    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a900204-9a2f-4327-9fc2-a20b23629928-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8a900204-9a2f-4327-9fc2-a20b23629928" (UID: "8a900204-9a2f-4327-9fc2-a20b23629928"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 14:40:13.298522 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:13.298495    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a900204-9a2f-4327-9fc2-a20b23629928-dshm\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:40:13.298522 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:13.298519    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a900204-9a2f-4327-9fc2-a20b23629928-tmp-dir\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:40:13.298522 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:13.298528    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a900204-9a2f-4327-9fc2-a20b23629928-model-cache\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:40:13.298741 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:13.298537    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a900204-9a2f-4327-9fc2-a20b23629928-tls-certs\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:40:13.298741 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:13.298546    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9tsd\" (UniqueName: \"kubernetes.io/projected/8a900204-9a2f-4327-9fc2-a20b23629928-kube-api-access-m9tsd\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:40:13.454113 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:13.454082    2559 generic.go:356] "Generic (PLEG): container finished" podID="8a900204-9a2f-4327-9fc2-a20b23629928" containerID="0d72bf8e13d9151c0481eec9ac2b72e70c37ec7148d1f8dada4ba8d71095f0e5" exitCode=0
Jul 29 14:40:13.454303 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:13.454125    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7"
Jul 29 14:40:13.454303 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:13.454134    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7" event={"ID":"8a900204-9a2f-4327-9fc2-a20b23629928","Type":"ContainerDied","Data":"0d72bf8e13d9151c0481eec9ac2b72e70c37ec7148d1f8dada4ba8d71095f0e5"}
Jul 29 14:40:13.454303 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:13.454175    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7" event={"ID":"8a900204-9a2f-4327-9fc2-a20b23629928","Type":"ContainerDied","Data":"7fb0f0da679a3f9fd0be4b3c0670a326b6e6c38cf3651c444d5f0966c89a9cdf"}
Jul 29 14:40:13.454303 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:13.454197    2559 scope.go:117] "RemoveContainer" containerID="0d72bf8e13d9151c0481eec9ac2b72e70c37ec7148d1f8dada4ba8d71095f0e5"
Jul 29 14:40:13.463281 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:13.463261    2559 scope.go:117] "RemoveContainer" containerID="0d72bf8e13d9151c0481eec9ac2b72e70c37ec7148d1f8dada4ba8d71095f0e5"
Jul 29 14:40:13.463516 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:40:13.463494    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d72bf8e13d9151c0481eec9ac2b72e70c37ec7148d1f8dada4ba8d71095f0e5\": container with ID starting with 0d72bf8e13d9151c0481eec9ac2b72e70c37ec7148d1f8dada4ba8d71095f0e5 not found: ID does not exist" containerID="0d72bf8e13d9151c0481eec9ac2b72e70c37ec7148d1f8dada4ba8d71095f0e5"
Jul 29 14:40:13.463561 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:13.463524    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d72bf8e13d9151c0481eec9ac2b72e70c37ec7148d1f8dada4ba8d71095f0e5"} err="failed to get container status \"0d72bf8e13d9151c0481eec9ac2b72e70c37ec7148d1f8dada4ba8d71095f0e5\": rpc error: code = NotFound desc = could not find container \"0d72bf8e13d9151c0481eec9ac2b72e70c37ec7148d1f8dada4ba8d71095f0e5\": container with ID starting with 0d72bf8e13d9151c0481eec9ac2b72e70c37ec7148d1f8dada4ba8d71095f0e5 not found: ID does not exist"
Jul 29 14:40:13.476846 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:13.476819    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7"]
Jul 29 14:40:13.481343 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:13.481311    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-055496/test-model-055496-kserve-7d6b6c4f5f-bcww7"]
Jul 29 14:40:13.808221 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:13.808184    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff" podUID="bea54895-501a-4e04-88ca-b309bf6e8cac" containerName="main" containerID="cri-o://3436be19f43cd08d6708411dfb4bcf0f3448351b0de9afa6868ccaab95e6ba3b" gracePeriod=45
Jul 29 14:40:14.049909 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:14.049884    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff"
Jul 29 14:40:14.104886 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:14.104808    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5sxpl\" (UniqueName: \"kubernetes.io/projected/bea54895-501a-4e04-88ca-b309bf6e8cac-kube-api-access-5sxpl\") pod \"bea54895-501a-4e04-88ca-b309bf6e8cac\" (UID: \"bea54895-501a-4e04-88ca-b309bf6e8cac\") "
Jul 29 14:40:14.104886 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:14.104854    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bea54895-501a-4e04-88ca-b309bf6e8cac-model-cache\") pod \"bea54895-501a-4e04-88ca-b309bf6e8cac\" (UID: \"bea54895-501a-4e04-88ca-b309bf6e8cac\") "
Jul 29 14:40:14.104886 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:14.104886    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bea54895-501a-4e04-88ca-b309bf6e8cac-tls-certs\") pod \"bea54895-501a-4e04-88ca-b309bf6e8cac\" (UID: \"bea54895-501a-4e04-88ca-b309bf6e8cac\") "
Jul 29 14:40:14.105175 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:14.104903    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bea54895-501a-4e04-88ca-b309bf6e8cac-dshm\") pod \"bea54895-501a-4e04-88ca-b309bf6e8cac\" (UID: \"bea54895-501a-4e04-88ca-b309bf6e8cac\") "
Jul 29 14:40:14.105175 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:14.104946    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bea54895-501a-4e04-88ca-b309bf6e8cac-home\") pod \"bea54895-501a-4e04-88ca-b309bf6e8cac\" (UID: \"bea54895-501a-4e04-88ca-b309bf6e8cac\") "
Jul 29 14:40:14.105175 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:14.104969    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bea54895-501a-4e04-88ca-b309bf6e8cac-tmp-dir\") pod \"bea54895-501a-4e04-88ca-b309bf6e8cac\" (UID: \"bea54895-501a-4e04-88ca-b309bf6e8cac\") "
Jul 29 14:40:14.105344 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:14.105245    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bea54895-501a-4e04-88ca-b309bf6e8cac-model-cache" (OuterVolumeSpecName: "model-cache") pod "bea54895-501a-4e04-88ca-b309bf6e8cac" (UID: "bea54895-501a-4e04-88ca-b309bf6e8cac"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:40:14.105344 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:14.105262    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bea54895-501a-4e04-88ca-b309bf6e8cac-home" (OuterVolumeSpecName: "home") pod "bea54895-501a-4e04-88ca-b309bf6e8cac" (UID: "bea54895-501a-4e04-88ca-b309bf6e8cac"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:40:14.105344 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:14.105298    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bea54895-501a-4e04-88ca-b309bf6e8cac-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bea54895-501a-4e04-88ca-b309bf6e8cac" (UID: "bea54895-501a-4e04-88ca-b309bf6e8cac"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:40:14.107067 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:14.107033    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bea54895-501a-4e04-88ca-b309bf6e8cac-dshm" (OuterVolumeSpecName: "dshm") pod "bea54895-501a-4e04-88ca-b309bf6e8cac" (UID: "bea54895-501a-4e04-88ca-b309bf6e8cac"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:40:14.107178 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:14.107156    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bea54895-501a-4e04-88ca-b309bf6e8cac-kube-api-access-5sxpl" (OuterVolumeSpecName: "kube-api-access-5sxpl") pod "bea54895-501a-4e04-88ca-b309bf6e8cac" (UID: "bea54895-501a-4e04-88ca-b309bf6e8cac"). InnerVolumeSpecName "kube-api-access-5sxpl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:40:14.107258 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:14.107239    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bea54895-501a-4e04-88ca-b309bf6e8cac-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bea54895-501a-4e04-88ca-b309bf6e8cac" (UID: "bea54895-501a-4e04-88ca-b309bf6e8cac"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 14:40:14.205876 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:14.205839    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bea54895-501a-4e04-88ca-b309bf6e8cac-tmp-dir\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:40:14.205876 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:14.205869    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5sxpl\" (UniqueName: \"kubernetes.io/projected/bea54895-501a-4e04-88ca-b309bf6e8cac-kube-api-access-5sxpl\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:40:14.205876 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:14.205879    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bea54895-501a-4e04-88ca-b309bf6e8cac-model-cache\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:40:14.206124 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:14.205888    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bea54895-501a-4e04-88ca-b309bf6e8cac-tls-certs\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:40:14.206124 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:14.205896    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bea54895-501a-4e04-88ca-b309bf6e8cac-dshm\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:40:14.206124 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:14.205903    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bea54895-501a-4e04-88ca-b309bf6e8cac-home\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:40:14.459586 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:14.459494    2559 generic.go:356] "Generic (PLEG): container finished" podID="bea54895-501a-4e04-88ca-b309bf6e8cac" containerID="3436be19f43cd08d6708411dfb4bcf0f3448351b0de9afa6868ccaab95e6ba3b" exitCode=0
Jul 29 14:40:14.459586 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:14.459553    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff"
Jul 29 14:40:14.459789 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:14.459586    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff" event={"ID":"bea54895-501a-4e04-88ca-b309bf6e8cac","Type":"ContainerDied","Data":"3436be19f43cd08d6708411dfb4bcf0f3448351b0de9afa6868ccaab95e6ba3b"}
Jul 29 14:40:14.459789 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:14.459632    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff" event={"ID":"bea54895-501a-4e04-88ca-b309bf6e8cac","Type":"ContainerDied","Data":"6e71fb8ba0d80ce1a65a0721f4de4fb695b5284eb2dbada1d71df0f9a3f771aa"}
Jul 29 14:40:14.459789 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:14.459656    2559 scope.go:117] "RemoveContainer" containerID="3436be19f43cd08d6708411dfb4bcf0f3448351b0de9afa6868ccaab95e6ba3b"
Jul 29 14:40:14.468050 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:14.468020    2559 scope.go:117] "RemoveContainer" containerID="3436be19f43cd08d6708411dfb4bcf0f3448351b0de9afa6868ccaab95e6ba3b"
Jul 29 14:40:14.468318 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:40:14.468299    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3436be19f43cd08d6708411dfb4bcf0f3448351b0de9afa6868ccaab95e6ba3b\": container with ID starting with 3436be19f43cd08d6708411dfb4bcf0f3448351b0de9afa6868ccaab95e6ba3b not found: ID does not exist" containerID="3436be19f43cd08d6708411dfb4bcf0f3448351b0de9afa6868ccaab95e6ba3b"
Jul 29 14:40:14.468402 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:14.468325    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3436be19f43cd08d6708411dfb4bcf0f3448351b0de9afa6868ccaab95e6ba3b"} err="failed to get container status \"3436be19f43cd08d6708411dfb4bcf0f3448351b0de9afa6868ccaab95e6ba3b\": rpc error: code = NotFound desc = could not find container \"3436be19f43cd08d6708411dfb4bcf0f3448351b0de9afa6868ccaab95e6ba3b\": container with ID starting with 3436be19f43cd08d6708411dfb4bcf0f3448351b0de9afa6868ccaab95e6ba3b not found: ID does not exist"
Jul 29 14:40:14.483368 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:14.483336    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff"]
Jul 29 14:40:14.488790 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:14.488769    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-02f676/test-model-02f676-kserve-b7857b6cf-jtsff"]
Jul 29 14:40:14.637381 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:14.637344    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a900204-9a2f-4327-9fc2-a20b23629928" path="/var/lib/kubelet/pods/8a900204-9a2f-4327-9fc2-a20b23629928/volumes"
Jul 29 14:40:14.637745 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:14.637732    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bea54895-501a-4e04-88ca-b309bf6e8cac" path="/var/lib/kubelet/pods/bea54895-501a-4e04-88ca-b309bf6e8cac/volumes"
Jul 29 14:40:15.976692 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:15.976664    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:40:16.077002 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:16.076970    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:40:16.224336 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:16.224300    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:40:16.446898 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:16.446856    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:40:25.819987 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:25.819951    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:40:28.028180 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:28.028138    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:40:28.098853 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:28.098828    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:40:33.869230 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:33.869189    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:40:33.956787 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:33.956751    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:40:34.046932 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:34.046899    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:40:34.201254 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:34.201166    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"]
Jul 29 14:40:34.201417 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:34.201369    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5" podUID="aafde015-1d18-4771-b22a-723d039fc0e2" containerName="istio-proxy" containerID="cri-o://575b18b076e3e9909c37603773213b81f21286032fd4fed9dd5f455b4aca0d98" gracePeriod=30
Jul 29 14:40:34.204823 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:34.204794    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:40:34.432494 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:34.432456    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:40:34.777216 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:34.777181    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:40:35.108739 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:35.108658    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:40:35.382372 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:35.382284    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:40:38.568123 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:38.568087    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:40:38.639708 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:38.639673    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f"]
Jul 29 14:40:38.640151 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:38.640136    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a900204-9a2f-4327-9fc2-a20b23629928" containerName="main"
Jul 29 14:40:38.640210 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:38.640155    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a900204-9a2f-4327-9fc2-a20b23629928" containerName="main"
Jul 29 14:40:38.640210 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:38.640167    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bea54895-501a-4e04-88ca-b309bf6e8cac" containerName="main"
Jul 29 14:40:38.640210 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:38.640175    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bea54895-501a-4e04-88ca-b309bf6e8cac" containerName="main"
Jul 29 14:40:38.640210 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:38.640196    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0d73721-ea56-4540-9cff-827db21cc230" containerName="maas-api"
Jul 29 14:40:38.640210 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:38.640205    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0d73721-ea56-4540-9cff-827db21cc230" containerName="maas-api"
Jul 29 14:40:38.640365 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:38.640275    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0d73721-ea56-4540-9cff-827db21cc230" containerName="maas-api"
Jul 29 14:40:38.640365 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:38.640286    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="bea54895-501a-4e04-88ca-b309bf6e8cac" containerName="main"
Jul 29 14:40:38.640365 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:38.640299    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a900204-9a2f-4327-9fc2-a20b23629928" containerName="main"
Jul 29 14:40:38.643421 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:38.643404    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f"
Jul 29 14:40:38.646327 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:38.646304    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-a20e0b-serving-cert\""
Jul 29 14:40:38.646439 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:38.646357    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7jqfw\""
Jul 29 14:40:38.652767 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:38.652745    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f"]
Jul 29 14:40:38.661746 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:38.661722    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:40:38.761861 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:38.761832    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:40:38.813491 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:38.813452    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91821ee9-02c6-4c05-9938-f78b7ce5bf75-maas-api-tls\") pod \"maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f\" (UID: \"91821ee9-02c6-4c05-9938-f78b7ce5bf75\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f"
Jul 29 14:40:38.813652 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:38.813517    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91821ee9-02c6-4c05-9938-f78b7ce5bf75-openshift-service-ca\") pod \"maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f\" (UID: \"91821ee9-02c6-4c05-9938-f78b7ce5bf75\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f"
Jul 29 14:40:38.813652 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:38.813620    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5fcv\" (UniqueName: \"kubernetes.io/projected/91821ee9-02c6-4c05-9938-f78b7ce5bf75-kube-api-access-q5fcv\") pod \"maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f\" (UID: \"91821ee9-02c6-4c05-9938-f78b7ce5bf75\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f"
Jul 29 14:40:38.870785 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:38.870699    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:40:38.914766 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:38.914737    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5fcv\" (UniqueName: \"kubernetes.io/projected/91821ee9-02c6-4c05-9938-f78b7ce5bf75-kube-api-access-q5fcv\") pod \"maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f\" (UID: \"91821ee9-02c6-4c05-9938-f78b7ce5bf75\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f"
Jul 29 14:40:38.914927 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:38.914778    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91821ee9-02c6-4c05-9938-f78b7ce5bf75-maas-api-tls\") pod \"maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f\" (UID: \"91821ee9-02c6-4c05-9938-f78b7ce5bf75\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f"
Jul 29 14:40:38.914927 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:38.914813    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91821ee9-02c6-4c05-9938-f78b7ce5bf75-openshift-service-ca\") pod \"maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f\" (UID: \"91821ee9-02c6-4c05-9938-f78b7ce5bf75\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f"
Jul 29 14:40:38.915478 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:38.915459    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91821ee9-02c6-4c05-9938-f78b7ce5bf75-openshift-service-ca\") pod \"maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f\" (UID: \"91821ee9-02c6-4c05-9938-f78b7ce5bf75\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f"
Jul 29 14:40:38.917317 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:38.917296    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91821ee9-02c6-4c05-9938-f78b7ce5bf75-maas-api-tls\") pod \"maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f\" (UID: \"91821ee9-02c6-4c05-9938-f78b7ce5bf75\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f"
Jul 29 14:40:38.923751 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:38.923726    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5fcv\" (UniqueName: \"kubernetes.io/projected/91821ee9-02c6-4c05-9938-f78b7ce5bf75-kube-api-access-q5fcv\") pod \"maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f\" (UID: \"91821ee9-02c6-4c05-9938-f78b7ce5bf75\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f"
Jul 29 14:40:38.954662 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:38.954630    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f"
Jul 29 14:40:39.090495 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.090468    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f"]
Jul 29 14:40:39.092622 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:40:39.092593    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91821ee9_02c6_4c05_9938_f78b7ce5bf75.slice/crio-680ba550f1d901ab14d4781216e1f2feaafb416f0c2acda46adc62718a9eff28 WatchSource:0}: Error finding container 680ba550f1d901ab14d4781216e1f2feaafb416f0c2acda46adc62718a9eff28: Status 404 returned error can't find the container with id 680ba550f1d901ab14d4781216e1f2feaafb416f0c2acda46adc62718a9eff28
Jul 29 14:40:39.452309 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.452283    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:40:39.480429 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.480356    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:40:39.565440 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.565405    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f" event={"ID":"91821ee9-02c6-4c05-9938-f78b7ce5bf75","Type":"ContainerStarted","Data":"680ba550f1d901ab14d4781216e1f2feaafb416f0c2acda46adc62718a9eff28"}
Jul 29 14:40:39.566894 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.566868    2559 generic.go:356] "Generic (PLEG): container finished" podID="aafde015-1d18-4771-b22a-723d039fc0e2" containerID="575b18b076e3e9909c37603773213b81f21286032fd4fed9dd5f455b4aca0d98" exitCode=0
Jul 29 14:40:39.567022 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.566907    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5" event={"ID":"aafde015-1d18-4771-b22a-723d039fc0e2","Type":"ContainerDied","Data":"575b18b076e3e9909c37603773213b81f21286032fd4fed9dd5f455b4aca0d98"}
Jul 29 14:40:39.567022 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.566920    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"
Jul 29 14:40:39.567022 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.566956    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5" event={"ID":"aafde015-1d18-4771-b22a-723d039fc0e2","Type":"ContainerDied","Data":"6133058918dd80be9d77424216a31944688f8fb203752477e8de6fac7032fc36"}
Jul 29 14:40:39.567022 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.566980    2559 scope.go:117] "RemoveContainer" containerID="575b18b076e3e9909c37603773213b81f21286032fd4fed9dd5f455b4aca0d98"
Jul 29 14:40:39.577884 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.577865    2559 scope.go:117] "RemoveContainer" containerID="4f4075e2628b762d5fa9aa6756b9420c0ea0c9f4d7478eef3c7a112ff4a30c05"
Jul 29 14:40:39.583744 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.583726    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:40:39.585241 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.585226    2559 scope.go:117] "RemoveContainer" containerID="575b18b076e3e9909c37603773213b81f21286032fd4fed9dd5f455b4aca0d98"
Jul 29 14:40:39.585485 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:40:39.585467    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"575b18b076e3e9909c37603773213b81f21286032fd4fed9dd5f455b4aca0d98\": container with ID starting with 575b18b076e3e9909c37603773213b81f21286032fd4fed9dd5f455b4aca0d98 not found: ID does not exist" containerID="575b18b076e3e9909c37603773213b81f21286032fd4fed9dd5f455b4aca0d98"
Jul 29 14:40:39.585536 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.585493    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"575b18b076e3e9909c37603773213b81f21286032fd4fed9dd5f455b4aca0d98"} err="failed to get container status \"575b18b076e3e9909c37603773213b81f21286032fd4fed9dd5f455b4aca0d98\": rpc error: code = NotFound desc = could not find container \"575b18b076e3e9909c37603773213b81f21286032fd4fed9dd5f455b4aca0d98\": container with ID starting with 575b18b076e3e9909c37603773213b81f21286032fd4fed9dd5f455b4aca0d98 not found: ID does not exist"
Jul 29 14:40:39.585536 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.585509    2559 scope.go:117] "RemoveContainer" containerID="4f4075e2628b762d5fa9aa6756b9420c0ea0c9f4d7478eef3c7a112ff4a30c05"
Jul 29 14:40:39.585749 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:40:39.585734    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f4075e2628b762d5fa9aa6756b9420c0ea0c9f4d7478eef3c7a112ff4a30c05\": container with ID starting with 4f4075e2628b762d5fa9aa6756b9420c0ea0c9f4d7478eef3c7a112ff4a30c05 not found: ID does not exist" containerID="4f4075e2628b762d5fa9aa6756b9420c0ea0c9f4d7478eef3c7a112ff4a30c05"
Jul 29 14:40:39.585797 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.585751    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f4075e2628b762d5fa9aa6756b9420c0ea0c9f4d7478eef3c7a112ff4a30c05"} err="failed to get container status \"4f4075e2628b762d5fa9aa6756b9420c0ea0c9f4d7478eef3c7a112ff4a30c05\": rpc error: code = NotFound desc = could not find container \"4f4075e2628b762d5fa9aa6756b9420c0ea0c9f4d7478eef3c7a112ff4a30c05\": container with ID starting with 4f4075e2628b762d5fa9aa6756b9420c0ea0c9f4d7478eef3c7a112ff4a30c05 not found: ID does not exist"
Jul 29 14:40:39.619110 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.619076    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j474d\" (UniqueName: \"kubernetes.io/projected/aafde015-1d18-4771-b22a-723d039fc0e2-kube-api-access-j474d\") pod \"aafde015-1d18-4771-b22a-723d039fc0e2\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") "
Jul 29 14:40:39.619270 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.619116    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aafde015-1d18-4771-b22a-723d039fc0e2-istio-data\") pod \"aafde015-1d18-4771-b22a-723d039fc0e2\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") "
Jul 29 14:40:39.619270 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.619132    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aafde015-1d18-4771-b22a-723d039fc0e2-credential-socket\") pod \"aafde015-1d18-4771-b22a-723d039fc0e2\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") "
Jul 29 14:40:39.619270 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.619169    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aafde015-1d18-4771-b22a-723d039fc0e2-workload-certs\") pod \"aafde015-1d18-4771-b22a-723d039fc0e2\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") "
Jul 29 14:40:39.619270 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.619221    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aafde015-1d18-4771-b22a-723d039fc0e2-istiod-ca-cert\") pod \"aafde015-1d18-4771-b22a-723d039fc0e2\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") "
Jul 29 14:40:39.619270 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.619261    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aafde015-1d18-4771-b22a-723d039fc0e2-istio-podinfo\") pod \"aafde015-1d18-4771-b22a-723d039fc0e2\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") "
Jul 29 14:40:39.619520 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.619290    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aafde015-1d18-4771-b22a-723d039fc0e2-istio-envoy\") pod \"aafde015-1d18-4771-b22a-723d039fc0e2\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") "
Jul 29 14:40:39.619520 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.619318    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aafde015-1d18-4771-b22a-723d039fc0e2-istio-token\") pod \"aafde015-1d18-4771-b22a-723d039fc0e2\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") "
Jul 29 14:40:39.619520 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.619346    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aafde015-1d18-4771-b22a-723d039fc0e2-workload-socket\") pod \"aafde015-1d18-4771-b22a-723d039fc0e2\" (UID: \"aafde015-1d18-4771-b22a-723d039fc0e2\") "
Jul 29 14:40:39.619520 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.619446    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aafde015-1d18-4771-b22a-723d039fc0e2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "aafde015-1d18-4771-b22a-723d039fc0e2" (UID: "aafde015-1d18-4771-b22a-723d039fc0e2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:40:39.619716 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.619531    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aafde015-1d18-4771-b22a-723d039fc0e2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "aafde015-1d18-4771-b22a-723d039fc0e2" (UID: "aafde015-1d18-4771-b22a-723d039fc0e2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:40:39.619716 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.619664    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aafde015-1d18-4771-b22a-723d039fc0e2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "aafde015-1d18-4771-b22a-723d039fc0e2" (UID: "aafde015-1d18-4771-b22a-723d039fc0e2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:40:39.619716 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.619692    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aafde015-1d18-4771-b22a-723d039fc0e2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "aafde015-1d18-4771-b22a-723d039fc0e2" (UID: "aafde015-1d18-4771-b22a-723d039fc0e2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 14:40:39.619859 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.619746    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aafde015-1d18-4771-b22a-723d039fc0e2-workload-socket\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:40:39.619859 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.619761    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aafde015-1d18-4771-b22a-723d039fc0e2-credential-socket\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:40:39.619859 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.619773    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aafde015-1d18-4771-b22a-723d039fc0e2-workload-certs\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:40:39.620079 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.620036    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aafde015-1d18-4771-b22a-723d039fc0e2-istio-data" (OuterVolumeSpecName: "istio-data") pod "aafde015-1d18-4771-b22a-723d039fc0e2" (UID: "aafde015-1d18-4771-b22a-723d039fc0e2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:40:39.621684 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.621658    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aafde015-1d18-4771-b22a-723d039fc0e2-istio-token" (OuterVolumeSpecName: "istio-token") pod "aafde015-1d18-4771-b22a-723d039fc0e2" (UID: "aafde015-1d18-4771-b22a-723d039fc0e2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:40:39.621811 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.621679    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aafde015-1d18-4771-b22a-723d039fc0e2-kube-api-access-j474d" (OuterVolumeSpecName: "kube-api-access-j474d") pod "aafde015-1d18-4771-b22a-723d039fc0e2" (UID: "aafde015-1d18-4771-b22a-723d039fc0e2"). InnerVolumeSpecName "kube-api-access-j474d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:40:39.621960 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.621930    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aafde015-1d18-4771-b22a-723d039fc0e2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "aafde015-1d18-4771-b22a-723d039fc0e2" (UID: "aafde015-1d18-4771-b22a-723d039fc0e2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 14:40:39.622046 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.621941    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aafde015-1d18-4771-b22a-723d039fc0e2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "aafde015-1d18-4771-b22a-723d039fc0e2" (UID: "aafde015-1d18-4771-b22a-723d039fc0e2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:40:39.720600 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.720566    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aafde015-1d18-4771-b22a-723d039fc0e2-istio-podinfo\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:40:39.720600 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.720601    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aafde015-1d18-4771-b22a-723d039fc0e2-istio-envoy\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:40:39.720797 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.720614    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aafde015-1d18-4771-b22a-723d039fc0e2-istio-token\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:40:39.720797 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.720628    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j474d\" (UniqueName: \"kubernetes.io/projected/aafde015-1d18-4771-b22a-723d039fc0e2-kube-api-access-j474d\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:40:39.720797 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.720642    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aafde015-1d18-4771-b22a-723d039fc0e2-istio-data\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:40:39.720797 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.720654    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aafde015-1d18-4771-b22a-723d039fc0e2-istiod-ca-cert\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:40:39.891094 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.891048    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"]
Jul 29 14:40:39.895284 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:39.895259    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-02f676-openshift-default-5f4f468dc4-w2qj5"]
Jul 29 14:40:40.636464 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:40.636432    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aafde015-1d18-4771-b22a-723d039fc0e2" path="/var/lib/kubelet/pods/aafde015-1d18-4771-b22a-723d039fc0e2/volumes"
Jul 29 14:40:41.584196 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:41.584157    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f" event={"ID":"91821ee9-02c6-4c05-9938-f78b7ce5bf75","Type":"ContainerStarted","Data":"9740d3a9392b8884f86967594f984dfa2d41fdb89722b475b624218dc8105736"}
Jul 29 14:40:41.584377 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:41.584293    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f"
Jul 29 14:40:41.602603 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:41.602562    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f" podStartSLOduration=2.109020574 podStartE2EDuration="3.60254749s" totalImagesPullingTime="1.493526916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:40:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:40:41.600549032 +0000 UTC m=+2371.491220367" watchObservedRunningTime="2026-07-29 14:40:41.60254749 +0000 UTC m=+2371.493218825"
Jul 29 14:40:47.594713 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:40:47.594684    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f"
Jul 29 14:41:01.307963 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:01.307932    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:41:03.565004 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:03.564963    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:41:04.073858 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.073829    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:41:04.363643 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.363552    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"]
Jul 29 14:41:04.363971 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.363952    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aafde015-1d18-4771-b22a-723d039fc0e2" containerName="istio-proxy"
Jul 29 14:41:04.364079 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.363974    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="aafde015-1d18-4771-b22a-723d039fc0e2" containerName="istio-proxy"
Jul 29 14:41:04.364079 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.364012    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aafde015-1d18-4771-b22a-723d039fc0e2" containerName="istio-proxy"
Jul 29 14:41:04.364079 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.364022    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="aafde015-1d18-4771-b22a-723d039fc0e2" containerName="istio-proxy"
Jul 29 14:41:04.364243 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.364152    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="aafde015-1d18-4771-b22a-723d039fc0e2" containerName="istio-proxy"
Jul 29 14:41:04.364243 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.364164    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="aafde015-1d18-4771-b22a-723d039fc0e2" containerName="istio-proxy"
Jul 29 14:41:04.367016 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.366995    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:04.369729 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.369701    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-ffdf30-openshift-default-dockercfg-5mp6w\""
Jul 29 14:41:04.380110 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.380086    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"]
Jul 29 14:41:04.396161 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.396128    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:41:04.425489 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.425466    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c24f5fcf-9c4d-4cba-bc80-de298505458f-istio-data\") pod \"e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") " pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:04.425596 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.425502    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c24f5fcf-9c4d-4cba-bc80-de298505458f-istio-podinfo\") pod \"e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") " pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:04.425596 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.425520    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c24f5fcf-9c4d-4cba-bc80-de298505458f-istiod-ca-cert\") pod \"e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") " pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:04.425596 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.425551    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c24f5fcf-9c4d-4cba-bc80-de298505458f-istio-envoy\") pod \"e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") " pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:04.425739 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.425622    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c24f5fcf-9c4d-4cba-bc80-de298505458f-istio-token\") pod \"e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") " pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:04.425739 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.425644    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kj89\" (UniqueName: \"kubernetes.io/projected/c24f5fcf-9c4d-4cba-bc80-de298505458f-kube-api-access-5kj89\") pod \"e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") " pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:04.425739 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.425667    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c24f5fcf-9c4d-4cba-bc80-de298505458f-credential-socket\") pod \"e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") " pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:04.425739 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.425682    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c24f5fcf-9c4d-4cba-bc80-de298505458f-workload-socket\") pod \"e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") " pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:04.425739 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.425721    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c24f5fcf-9c4d-4cba-bc80-de298505458f-workload-certs\") pod \"e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") " pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:04.526915 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.526885    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c24f5fcf-9c4d-4cba-bc80-de298505458f-istio-token\") pod \"e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") " pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:04.526915 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.526917    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kj89\" (UniqueName: \"kubernetes.io/projected/c24f5fcf-9c4d-4cba-bc80-de298505458f-kube-api-access-5kj89\") pod \"e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") " pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:04.527168 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.526951    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c24f5fcf-9c4d-4cba-bc80-de298505458f-credential-socket\") pod \"e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") " pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:04.527168 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.526981    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c24f5fcf-9c4d-4cba-bc80-de298505458f-workload-socket\") pod \"e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") " pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:04.527168 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.527004    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c24f5fcf-9c4d-4cba-bc80-de298505458f-workload-certs\") pod \"e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") " pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:04.527168 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.527116    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c24f5fcf-9c4d-4cba-bc80-de298505458f-istio-data\") pod \"e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") " pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:04.527363 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.527175    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c24f5fcf-9c4d-4cba-bc80-de298505458f-istio-podinfo\") pod \"e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") " pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:04.527363 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.527195    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c24f5fcf-9c4d-4cba-bc80-de298505458f-istiod-ca-cert\") pod \"e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") " pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:04.527363 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.527232    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c24f5fcf-9c4d-4cba-bc80-de298505458f-istio-envoy\") pod \"e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") " pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:04.527510 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.527428    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c24f5fcf-9c4d-4cba-bc80-de298505458f-workload-socket\") pod \"e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") " pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:04.527510 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.527459    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c24f5fcf-9c4d-4cba-bc80-de298505458f-workload-certs\") pod \"e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") " pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:04.527607 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.527547    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c24f5fcf-9c4d-4cba-bc80-de298505458f-credential-socket\") pod \"e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") " pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:04.527659 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.527643    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c24f5fcf-9c4d-4cba-bc80-de298505458f-istio-data\") pod \"e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") " pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:04.527885 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.527838    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c24f5fcf-9c4d-4cba-bc80-de298505458f-istiod-ca-cert\") pod \"e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") " pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:04.529528 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.529511    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c24f5fcf-9c4d-4cba-bc80-de298505458f-istio-envoy\") pod \"e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") " pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:04.529770 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.529752    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c24f5fcf-9c4d-4cba-bc80-de298505458f-istio-podinfo\") pod \"e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") " pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:04.534842 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.534812    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kj89\" (UniqueName: \"kubernetes.io/projected/c24f5fcf-9c4d-4cba-bc80-de298505458f-kube-api-access-5kj89\") pod \"e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") " pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:04.534956 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.534882    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c24f5fcf-9c4d-4cba-bc80-de298505458f-istio-token\") pod \"e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") " pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:04.542788 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.542767    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:41:04.677930 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.677830    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:04.812121 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.812092    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"]
Jul 29 14:41:04.814118 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:41:04.814091    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc24f5fcf_9c4d_4cba_bc80_de298505458f.slice/crio-ac51e1f0e899e81f3ee1d12f8166dbcdf908bf386e751ac2449dde06e2ecfb91 WatchSource:0}: Error finding container ac51e1f0e899e81f3ee1d12f8166dbcdf908bf386e751ac2449dde06e2ecfb91: Status 404 returned error can't find the container with id ac51e1f0e899e81f3ee1d12f8166dbcdf908bf386e751ac2449dde06e2ecfb91
Jul 29 14:41:04.816570 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.816539    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:41:04.816643 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.816602    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:41:04.816643 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:04.816632    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:41:05.689040 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:05.689003    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq" event={"ID":"c24f5fcf-9c4d-4cba-bc80-de298505458f","Type":"ContainerStarted","Data":"75000fc999218eb17b846671fe55b0f352056b04c55609eaaed2432bf8f314ef"}
Jul 29 14:41:05.689555 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:05.689084    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq" event={"ID":"c24f5fcf-9c4d-4cba-bc80-de298505458f","Type":"ContainerStarted","Data":"ac51e1f0e899e81f3ee1d12f8166dbcdf908bf386e751ac2449dde06e2ecfb91"}
Jul 29 14:41:05.712716 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:05.712666    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq" podStartSLOduration=1.7126509030000001 podStartE2EDuration="1.712650903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:41:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:41:05.710450329 +0000 UTC m=+2395.601121701" watchObservedRunningTime="2026-07-29 14:41:05.712650903 +0000 UTC m=+2395.603322238"
Jul 29 14:41:06.678188 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:06.678154    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:06.682971 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:06.682946    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:06.692856 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:06.692834    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:06.693572 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:06.693552    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:07.668908 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:07.668848    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:41:07.747108 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:07.747070    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:41:07.753050 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:07.753023    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g"]
Jul 29 14:41:07.756546 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:07.756527    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g"
Jul 29 14:41:07.760036 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:07.760012    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-ffdf30-serving-cert\""
Jul 29 14:41:07.772951 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:07.772918    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g"]
Jul 29 14:41:07.837173 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:07.837136    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:41:07.856886 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:07.856850    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmvfx\" (UniqueName: \"kubernetes.io/projected/fa1ea6ed-8459-4c8e-82f8-0f081480538c-kube-api-access-vmvfx\") pod \"maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g\" (UID: \"fa1ea6ed-8459-4c8e-82f8-0f081480538c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g"
Jul 29 14:41:07.857046 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:07.856913    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa1ea6ed-8459-4c8e-82f8-0f081480538c-openshift-service-ca\") pod \"maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g\" (UID: \"fa1ea6ed-8459-4c8e-82f8-0f081480538c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g"
Jul 29 14:41:07.857046 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:07.856980    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa1ea6ed-8459-4c8e-82f8-0f081480538c-maas-api-tls\") pod \"maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g\" (UID: \"fa1ea6ed-8459-4c8e-82f8-0f081480538c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g"
Jul 29 14:41:07.958210 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:07.958114    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa1ea6ed-8459-4c8e-82f8-0f081480538c-maas-api-tls\") pod \"maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g\" (UID: \"fa1ea6ed-8459-4c8e-82f8-0f081480538c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g"
Jul 29 14:41:07.958210 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:07.958200    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmvfx\" (UniqueName: \"kubernetes.io/projected/fa1ea6ed-8459-4c8e-82f8-0f081480538c-kube-api-access-vmvfx\") pod \"maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g\" (UID: \"fa1ea6ed-8459-4c8e-82f8-0f081480538c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g"
Jul 29 14:41:07.958403 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:07.958228    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa1ea6ed-8459-4c8e-82f8-0f081480538c-openshift-service-ca\") pod \"maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g\" (UID: \"fa1ea6ed-8459-4c8e-82f8-0f081480538c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g"
Jul 29 14:41:07.958838 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:07.958818    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa1ea6ed-8459-4c8e-82f8-0f081480538c-openshift-service-ca\") pod \"maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g\" (UID: \"fa1ea6ed-8459-4c8e-82f8-0f081480538c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g"
Jul 29 14:41:07.960585 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:07.960562    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa1ea6ed-8459-4c8e-82f8-0f081480538c-maas-api-tls\") pod \"maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g\" (UID: \"fa1ea6ed-8459-4c8e-82f8-0f081480538c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g"
Jul 29 14:41:07.974208 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:07.974178    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmvfx\" (UniqueName: \"kubernetes.io/projected/fa1ea6ed-8459-4c8e-82f8-0f081480538c-kube-api-access-vmvfx\") pod \"maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g\" (UID: \"fa1ea6ed-8459-4c8e-82f8-0f081480538c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g"
Jul 29 14:41:07.999168 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:07.999142    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:41:08.068309 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:08.068278    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g"
Jul 29 14:41:08.245739 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:08.245712    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g"]
Jul 29 14:41:08.247434 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:41:08.247403    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa1ea6ed_8459_4c8e_82f8_0f081480538c.slice/crio-ef23aff70e57ab2f2b93f746dc1a8a2ee0621abb1fa834e6b90fd1af9380e919 WatchSource:0}: Error finding container ef23aff70e57ab2f2b93f746dc1a8a2ee0621abb1fa834e6b90fd1af9380e919: Status 404 returned error can't find the container with id ef23aff70e57ab2f2b93f746dc1a8a2ee0621abb1fa834e6b90fd1af9380e919
Jul 29 14:41:08.580408 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:08.580291    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:41:08.665162 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:08.665131    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:41:08.702001 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:08.701969    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g" event={"ID":"fa1ea6ed-8459-4c8e-82f8-0f081480538c","Type":"ContainerStarted","Data":"ef23aff70e57ab2f2b93f746dc1a8a2ee0621abb1fa834e6b90fd1af9380e919"}
Jul 29 14:41:09.708027 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:09.707987    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g" event={"ID":"fa1ea6ed-8459-4c8e-82f8-0f081480538c","Type":"ContainerStarted","Data":"644d7b525d19aa4d364f718cbf9dcf6ccfb896cefddb0e30b942c96f29106404"}
Jul 29 14:41:09.708475 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:09.708079    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g"
Jul 29 14:41:09.728689 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:09.728638    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g" podStartSLOduration=2.248690898 podStartE2EDuration="2.728618967s" totalImagesPullingTime="479.928069ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:41:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:41:09.727563395 +0000 UTC m=+2399.618234730" watchObservedRunningTime="2026-07-29 14:41:09.728618967 +0000 UTC m=+2399.619290323"
Jul 29 14:41:10.724651 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:10.724619    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdw8l_b93f6faa-c8bf-43e9-94e0-b3dcd97b7296/ovn-acl-logging/0.log"
Jul 29 14:41:10.731029 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:10.731009    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdw8l_b93f6faa-c8bf-43e9-94e0-b3dcd97b7296/ovn-acl-logging/0.log"
Jul 29 14:41:15.717551 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:15.717518    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g"
Jul 29 14:41:22.735268 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:22.735176    2559 patch_prober.go:28] interesting pod/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 29 14:41:22.735268 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:22.735235    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq" podUID="c24f5fcf-9c4d-4cba-bc80-de298505458f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 29 14:41:23.768704 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:23.768671    2559 generic.go:356] "Generic (PLEG): container finished" podID="c24f5fcf-9c4d-4cba-bc80-de298505458f" containerID="75000fc999218eb17b846671fe55b0f352056b04c55609eaaed2432bf8f314ef" exitCode=0
Jul 29 14:41:23.769269 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:23.768749    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq" event={"ID":"c24f5fcf-9c4d-4cba-bc80-de298505458f","Type":"ContainerDied","Data":"75000fc999218eb17b846671fe55b0f352056b04c55609eaaed2432bf8f314ef"}
Jul 29 14:41:23.769269 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:23.769257    2559 scope.go:117] "RemoveContainer" containerID="75000fc999218eb17b846671fe55b0f352056b04c55609eaaed2432bf8f314ef"
Jul 29 14:41:23.770328 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:23.770298    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:41:23.770449 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:23.770379    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:41:23.770449 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:23.770427    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:41:24.774846 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:24.774815    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq" event={"ID":"c24f5fcf-9c4d-4cba-bc80-de298505458f","Type":"ContainerStarted","Data":"082ab44b95a9b23eb36a4f5372f986bcefc419ddd86178577e8c34696a8bcf9f"}
Jul 29 14:41:25.678048 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:25.678001    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:25.678315 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:25.678291    2559 patch_prober.go:28] interesting pod/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 29 14:41:25.678403 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:25.678349    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq" podUID="c24f5fcf-9c4d-4cba-bc80-de298505458f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 29 14:41:26.679156 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:26.679116    2559 patch_prober.go:28] interesting pod/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 29 14:41:26.679568 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:26.679168    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq" podUID="c24f5fcf-9c4d-4cba-bc80-de298505458f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 29 14:41:27.679108 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:27.679044    2559 patch_prober.go:28] interesting pod/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 29 14:41:27.679318 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:27.679124    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq" podUID="c24f5fcf-9c4d-4cba-bc80-de298505458f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 29 14:41:28.678567 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:28.678529    2559 patch_prober.go:28] interesting pod/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 29 14:41:28.678775 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:28.678588    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq" podUID="c24f5fcf-9c4d-4cba-bc80-de298505458f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 29 14:41:29.682250 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:29.682219    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:29.682750 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:29.682547    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:29.683112 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:29.683092    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:41:32.826408 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:32.826326    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:41:51.653762 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:51.653727    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f"]
Jul 29 14:41:51.654252 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:51.653928    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f" podUID="91821ee9-02c6-4c05-9938-f78b7ce5bf75" containerName="maas-api" containerID="cri-o://9740d3a9392b8884f86967594f984dfa2d41fdb89722b475b624218dc8105736" gracePeriod=30
Jul 29 14:41:51.852666 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:51.852619    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:41:51.891110 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:51.891082    2559 generic.go:356] "Generic (PLEG): container finished" podID="91821ee9-02c6-4c05-9938-f78b7ce5bf75" containerID="9740d3a9392b8884f86967594f984dfa2d41fdb89722b475b624218dc8105736" exitCode=0
Jul 29 14:41:51.891221 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:51.891155    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f" event={"ID":"91821ee9-02c6-4c05-9938-f78b7ce5bf75","Type":"ContainerDied","Data":"9740d3a9392b8884f86967594f984dfa2d41fdb89722b475b624218dc8105736"}
Jul 29 14:41:51.891264 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:51.891219    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f" event={"ID":"91821ee9-02c6-4c05-9938-f78b7ce5bf75","Type":"ContainerDied","Data":"680ba550f1d901ab14d4781216e1f2feaafb416f0c2acda46adc62718a9eff28"}
Jul 29 14:41:51.891264 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:51.891243    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="680ba550f1d901ab14d4781216e1f2feaafb416f0c2acda46adc62718a9eff28"
Jul 29 14:41:51.898247 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:51.898226    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f"
Jul 29 14:41:51.917498 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:51.917429    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:41:51.948593 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:51.948568    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91821ee9-02c6-4c05-9938-f78b7ce5bf75-maas-api-tls\") pod \"91821ee9-02c6-4c05-9938-f78b7ce5bf75\" (UID: \"91821ee9-02c6-4c05-9938-f78b7ce5bf75\") "
Jul 29 14:41:51.948745 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:51.948633    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91821ee9-02c6-4c05-9938-f78b7ce5bf75-openshift-service-ca\") pod \"91821ee9-02c6-4c05-9938-f78b7ce5bf75\" (UID: \"91821ee9-02c6-4c05-9938-f78b7ce5bf75\") "
Jul 29 14:41:51.948745 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:51.948668    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q5fcv\" (UniqueName: \"kubernetes.io/projected/91821ee9-02c6-4c05-9938-f78b7ce5bf75-kube-api-access-q5fcv\") pod \"91821ee9-02c6-4c05-9938-f78b7ce5bf75\" (UID: \"91821ee9-02c6-4c05-9938-f78b7ce5bf75\") "
Jul 29 14:41:51.949017 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:51.948988    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91821ee9-02c6-4c05-9938-f78b7ce5bf75-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "91821ee9-02c6-4c05-9938-f78b7ce5bf75" (UID: "91821ee9-02c6-4c05-9938-f78b7ce5bf75"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 14:41:51.950564 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:51.950544    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91821ee9-02c6-4c05-9938-f78b7ce5bf75-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "91821ee9-02c6-4c05-9938-f78b7ce5bf75" (UID: "91821ee9-02c6-4c05-9938-f78b7ce5bf75"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 14:41:51.950649 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:51.950632    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91821ee9-02c6-4c05-9938-f78b7ce5bf75-kube-api-access-q5fcv" (OuterVolumeSpecName: "kube-api-access-q5fcv") pod "91821ee9-02c6-4c05-9938-f78b7ce5bf75" (UID: "91821ee9-02c6-4c05-9938-f78b7ce5bf75"). InnerVolumeSpecName "kube-api-access-q5fcv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:41:52.050014 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:52.049973    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91821ee9-02c6-4c05-9938-f78b7ce5bf75-openshift-service-ca\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:41:52.050014 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:52.050014    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5fcv\" (UniqueName: \"kubernetes.io/projected/91821ee9-02c6-4c05-9938-f78b7ce5bf75-kube-api-access-q5fcv\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:41:52.050210 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:52.050030    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91821ee9-02c6-4c05-9938-f78b7ce5bf75-maas-api-tls\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:41:52.296257 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:52.296225    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:41:52.901511 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:52.901428    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f"
Jul 29 14:41:52.920713 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:52.920681    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f"]
Jul 29 14:41:52.927427 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:52.927394    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a20e0b-6ff5f65cdb-f689f"]
Jul 29 14:41:54.636093 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:54.636042    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91821ee9-02c6-4c05-9938-f78b7ce5bf75" path="/var/lib/kubelet/pods/91821ee9-02c6-4c05-9938-f78b7ce5bf75/volumes"
Jul 29 14:41:57.896299 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:57.896269    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:41:57.996428 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:57.996397    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:41:58.164102 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:58.164001    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:41:58.404915 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:41:58.404881    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:42:14.146699 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:14.146667    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g"]
Jul 29 14:42:14.147161 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:14.146853    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g" podUID="fa1ea6ed-8459-4c8e-82f8-0f081480538c" containerName="maas-api" containerID="cri-o://644d7b525d19aa4d364f718cbf9dcf6ccfb896cefddb0e30b942c96f29106404" gracePeriod=30
Jul 29 14:42:14.178713 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:14.178678    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:42:14.257098 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:14.257047    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:42:14.325956 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:14.325917    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:42:14.390089 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:14.390039    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:42:14.394806 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:14.394779    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g"
Jul 29 14:42:14.550569 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:14.550538    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmvfx\" (UniqueName: \"kubernetes.io/projected/fa1ea6ed-8459-4c8e-82f8-0f081480538c-kube-api-access-vmvfx\") pod \"fa1ea6ed-8459-4c8e-82f8-0f081480538c\" (UID: \"fa1ea6ed-8459-4c8e-82f8-0f081480538c\") "
Jul 29 14:42:14.550743 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:14.550590    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa1ea6ed-8459-4c8e-82f8-0f081480538c-maas-api-tls\") pod \"fa1ea6ed-8459-4c8e-82f8-0f081480538c\" (UID: \"fa1ea6ed-8459-4c8e-82f8-0f081480538c\") "
Jul 29 14:42:14.550743 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:14.550700    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa1ea6ed-8459-4c8e-82f8-0f081480538c-openshift-service-ca\") pod \"fa1ea6ed-8459-4c8e-82f8-0f081480538c\" (UID: \"fa1ea6ed-8459-4c8e-82f8-0f081480538c\") "
Jul 29 14:42:14.551137 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:14.551105    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa1ea6ed-8459-4c8e-82f8-0f081480538c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fa1ea6ed-8459-4c8e-82f8-0f081480538c" (UID: "fa1ea6ed-8459-4c8e-82f8-0f081480538c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 14:42:14.552804 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:14.552780    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa1ea6ed-8459-4c8e-82f8-0f081480538c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fa1ea6ed-8459-4c8e-82f8-0f081480538c" (UID: "fa1ea6ed-8459-4c8e-82f8-0f081480538c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 14:42:14.552804 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:14.552781    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa1ea6ed-8459-4c8e-82f8-0f081480538c-kube-api-access-vmvfx" (OuterVolumeSpecName: "kube-api-access-vmvfx") pod "fa1ea6ed-8459-4c8e-82f8-0f081480538c" (UID: "fa1ea6ed-8459-4c8e-82f8-0f081480538c"). InnerVolumeSpecName "kube-api-access-vmvfx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:42:14.651456 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:14.651432    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa1ea6ed-8459-4c8e-82f8-0f081480538c-openshift-service-ca\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:42:14.651456 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:14.651455    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmvfx\" (UniqueName: \"kubernetes.io/projected/fa1ea6ed-8459-4c8e-82f8-0f081480538c-kube-api-access-vmvfx\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:42:14.651611 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:14.651465    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa1ea6ed-8459-4c8e-82f8-0f081480538c-maas-api-tls\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:42:14.825044 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:14.824952    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:42:14.999512 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:14.999478    2559 generic.go:356] "Generic (PLEG): container finished" podID="fa1ea6ed-8459-4c8e-82f8-0f081480538c" containerID="644d7b525d19aa4d364f718cbf9dcf6ccfb896cefddb0e30b942c96f29106404" exitCode=0
Jul 29 14:42:14.999676 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:14.999522    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g"
Jul 29 14:42:14.999676 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:14.999537    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g" event={"ID":"fa1ea6ed-8459-4c8e-82f8-0f081480538c","Type":"ContainerDied","Data":"644d7b525d19aa4d364f718cbf9dcf6ccfb896cefddb0e30b942c96f29106404"}
Jul 29 14:42:14.999676 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:14.999578    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g" event={"ID":"fa1ea6ed-8459-4c8e-82f8-0f081480538c","Type":"ContainerDied","Data":"ef23aff70e57ab2f2b93f746dc1a8a2ee0621abb1fa834e6b90fd1af9380e919"}
Jul 29 14:42:14.999676 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:14.999600    2559 scope.go:117] "RemoveContainer" containerID="644d7b525d19aa4d364f718cbf9dcf6ccfb896cefddb0e30b942c96f29106404"
Jul 29 14:42:15.007856 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:15.007841    2559 scope.go:117] "RemoveContainer" containerID="644d7b525d19aa4d364f718cbf9dcf6ccfb896cefddb0e30b942c96f29106404"
Jul 29 14:42:15.008111 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:42:15.008093    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"644d7b525d19aa4d364f718cbf9dcf6ccfb896cefddb0e30b942c96f29106404\": container with ID starting with 644d7b525d19aa4d364f718cbf9dcf6ccfb896cefddb0e30b942c96f29106404 not found: ID does not exist" containerID="644d7b525d19aa4d364f718cbf9dcf6ccfb896cefddb0e30b942c96f29106404"
Jul 29 14:42:15.008157 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:15.008119    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"644d7b525d19aa4d364f718cbf9dcf6ccfb896cefddb0e30b942c96f29106404"} err="failed to get container status \"644d7b525d19aa4d364f718cbf9dcf6ccfb896cefddb0e30b942c96f29106404\": rpc error: code = NotFound desc = could not find container \"644d7b525d19aa4d364f718cbf9dcf6ccfb896cefddb0e30b942c96f29106404\": container with ID starting with 644d7b525d19aa4d364f718cbf9dcf6ccfb896cefddb0e30b942c96f29106404 not found: ID does not exist"
Jul 29 14:42:15.017698 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:15.017677    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g"]
Jul 29 14:42:15.022813 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:15.022793    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ffdf30-5998879c94-cjc4g"]
Jul 29 14:42:16.636347 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:16.636314    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa1ea6ed-8459-4c8e-82f8-0f081480538c" path="/var/lib/kubelet/pods/fa1ea6ed-8459-4c8e-82f8-0f081480538c/volumes"
Jul 29 14:42:20.748325 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:20.748280    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:42:20.843462 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:20.843427    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:42:20.999873 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:20.999793    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"]
Jul 29 14:42:21.000017 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:20.999997    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq" podUID="c24f5fcf-9c4d-4cba-bc80-de298505458f" containerName="istio-proxy" containerID="cri-o://082ab44b95a9b23eb36a4f5372f986bcefc419ddd86178577e8c34696a8bcf9f" gracePeriod=30
Jul 29 14:42:21.025976 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:21.025939    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:42:21.247972 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:21.247941    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:42:21.620472 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:21.620438    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:42:21.922250 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:21.922161    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:42:22.682497 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:22.682464    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:42:25.304421 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:25.304377    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:42:25.413746 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:25.413715    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x"]
Jul 29 14:42:25.414121 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:25.414104    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91821ee9-02c6-4c05-9938-f78b7ce5bf75" containerName="maas-api"
Jul 29 14:42:25.414220 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:25.414123    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="91821ee9-02c6-4c05-9938-f78b7ce5bf75" containerName="maas-api"
Jul 29 14:42:25.414220 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:25.414137    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa1ea6ed-8459-4c8e-82f8-0f081480538c" containerName="maas-api"
Jul 29 14:42:25.414220 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:25.414144    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa1ea6ed-8459-4c8e-82f8-0f081480538c" containerName="maas-api"
Jul 29 14:42:25.414375 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:25.414252    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="91821ee9-02c6-4c05-9938-f78b7ce5bf75" containerName="maas-api"
Jul 29 14:42:25.414375 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:25.414266    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa1ea6ed-8459-4c8e-82f8-0f081480538c" containerName="maas-api"
Jul 29 14:42:25.415981 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:25.415960    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x"
Jul 29 14:42:25.418789 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:25.418768    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7jqfw\""
Jul 29 14:42:25.418886 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:25.418792    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-61bc0a-serving-cert\""
Jul 29 14:42:25.423151 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:25.423129    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:42:25.429861 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:25.429839    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x"]
Jul 29 14:42:25.512431 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:25.512375    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:42:25.549530 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:25.549493    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbts8\" (UniqueName: \"kubernetes.io/projected/11db5568-30e2-4d34-9a60-2b7d4e5670e3-kube-api-access-mbts8\") pod \"maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x\" (UID: \"11db5568-30e2-4d34-9a60-2b7d4e5670e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x"
Jul 29 14:42:25.549530 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:25.549531    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11db5568-30e2-4d34-9a60-2b7d4e5670e3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x\" (UID: \"11db5568-30e2-4d34-9a60-2b7d4e5670e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x"
Jul 29 14:42:25.549792 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:25.549559    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11db5568-30e2-4d34-9a60-2b7d4e5670e3-maas-api-tls\") pod \"maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x\" (UID: \"11db5568-30e2-4d34-9a60-2b7d4e5670e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x"
Jul 29 14:42:25.644657 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:25.644557    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:42:25.650907 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:25.650884    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbts8\" (UniqueName: \"kubernetes.io/projected/11db5568-30e2-4d34-9a60-2b7d4e5670e3-kube-api-access-mbts8\") pod \"maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x\" (UID: \"11db5568-30e2-4d34-9a60-2b7d4e5670e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x"
Jul 29 14:42:25.651001 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:25.650917    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11db5568-30e2-4d34-9a60-2b7d4e5670e3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x\" (UID: \"11db5568-30e2-4d34-9a60-2b7d4e5670e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x"
Jul 29 14:42:25.651001 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:25.650948    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11db5568-30e2-4d34-9a60-2b7d4e5670e3-maas-api-tls\") pod \"maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x\" (UID: \"11db5568-30e2-4d34-9a60-2b7d4e5670e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x"
Jul 29 14:42:25.651534 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:25.651512    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11db5568-30e2-4d34-9a60-2b7d4e5670e3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x\" (UID: \"11db5568-30e2-4d34-9a60-2b7d4e5670e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x"
Jul 29 14:42:25.653506 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:25.653482    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11db5568-30e2-4d34-9a60-2b7d4e5670e3-maas-api-tls\") pod \"maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x\" (UID: \"11db5568-30e2-4d34-9a60-2b7d4e5670e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x"
Jul 29 14:42:25.665387 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:25.665362    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbts8\" (UniqueName: \"kubernetes.io/projected/11db5568-30e2-4d34-9a60-2b7d4e5670e3-kube-api-access-mbts8\") pod \"maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x\" (UID: \"11db5568-30e2-4d34-9a60-2b7d4e5670e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x"
Jul 29 14:42:25.726383 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:25.726350    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x"
Jul 29 14:42:25.875701 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:25.875674    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x"]
Jul 29 14:42:25.877654 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:42:25.877628    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11db5568_30e2_4d34_9a60_2b7d4e5670e3.slice/crio-abfbc3993cdde70ea808568ae8f50ded8ff6063ebba35d3f2904bee06d256e2d WatchSource:0}: Error finding container abfbc3993cdde70ea808568ae8f50ded8ff6063ebba35d3f2904bee06d256e2d: Status 404 returned error can't find the container with id abfbc3993cdde70ea808568ae8f50ded8ff6063ebba35d3f2904bee06d256e2d
Jul 29 14:42:26.049889 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:26.049852    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x" event={"ID":"11db5568-30e2-4d34-9a60-2b7d4e5670e3","Type":"ContainerStarted","Data":"abfbc3993cdde70ea808568ae8f50ded8ff6063ebba35d3f2904bee06d256e2d"}
Jul 29 14:42:26.160547 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:26.160506    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:42:26.242389 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:26.242364    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:42:26.262364 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:26.262271    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:42:26.355867 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:26.355840    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c24f5fcf-9c4d-4cba-bc80-de298505458f-istio-envoy\") pod \"c24f5fcf-9c4d-4cba-bc80-de298505458f\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") "
Jul 29 14:42:26.356303 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:26.355896    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c24f5fcf-9c4d-4cba-bc80-de298505458f-istio-podinfo\") pod \"c24f5fcf-9c4d-4cba-bc80-de298505458f\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") "
Jul 29 14:42:26.356303 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:26.355945    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c24f5fcf-9c4d-4cba-bc80-de298505458f-istiod-ca-cert\") pod \"c24f5fcf-9c4d-4cba-bc80-de298505458f\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") "
Jul 29 14:42:26.356303 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:26.355960    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c24f5fcf-9c4d-4cba-bc80-de298505458f-istio-token\") pod \"c24f5fcf-9c4d-4cba-bc80-de298505458f\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") "
Jul 29 14:42:26.356303 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:26.355992    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5kj89\" (UniqueName: \"kubernetes.io/projected/c24f5fcf-9c4d-4cba-bc80-de298505458f-kube-api-access-5kj89\") pod \"c24f5fcf-9c4d-4cba-bc80-de298505458f\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") "
Jul 29 14:42:26.356303 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:26.356035    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c24f5fcf-9c4d-4cba-bc80-de298505458f-workload-socket\") pod \"c24f5fcf-9c4d-4cba-bc80-de298505458f\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") "
Jul 29 14:42:26.356303 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:26.356088    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c24f5fcf-9c4d-4cba-bc80-de298505458f-workload-certs\") pod \"c24f5fcf-9c4d-4cba-bc80-de298505458f\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") "
Jul 29 14:42:26.356303 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:26.356133    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c24f5fcf-9c4d-4cba-bc80-de298505458f-credential-socket\") pod \"c24f5fcf-9c4d-4cba-bc80-de298505458f\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") "
Jul 29 14:42:26.356303 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:26.356178    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c24f5fcf-9c4d-4cba-bc80-de298505458f-istio-data\") pod \"c24f5fcf-9c4d-4cba-bc80-de298505458f\" (UID: \"c24f5fcf-9c4d-4cba-bc80-de298505458f\") "
Jul 29 14:42:26.356723 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:26.356625    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c24f5fcf-9c4d-4cba-bc80-de298505458f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c24f5fcf-9c4d-4cba-bc80-de298505458f" (UID: "c24f5fcf-9c4d-4cba-bc80-de298505458f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:42:26.356837 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:26.356753    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c24f5fcf-9c4d-4cba-bc80-de298505458f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c24f5fcf-9c4d-4cba-bc80-de298505458f" (UID: "c24f5fcf-9c4d-4cba-bc80-de298505458f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:42:26.357656 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:26.357232    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c24f5fcf-9c4d-4cba-bc80-de298505458f-istio-data" (OuterVolumeSpecName: "istio-data") pod "c24f5fcf-9c4d-4cba-bc80-de298505458f" (UID: "c24f5fcf-9c4d-4cba-bc80-de298505458f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:42:26.357656 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:26.357231    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c24f5fcf-9c4d-4cba-bc80-de298505458f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c24f5fcf-9c4d-4cba-bc80-de298505458f" (UID: "c24f5fcf-9c4d-4cba-bc80-de298505458f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 14:42:26.357656 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:26.357609    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c24f5fcf-9c4d-4cba-bc80-de298505458f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c24f5fcf-9c4d-4cba-bc80-de298505458f" (UID: "c24f5fcf-9c4d-4cba-bc80-de298505458f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:42:26.358591 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:26.358554    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c24f5fcf-9c4d-4cba-bc80-de298505458f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c24f5fcf-9c4d-4cba-bc80-de298505458f" (UID: "c24f5fcf-9c4d-4cba-bc80-de298505458f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:42:26.358992 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:26.358954    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c24f5fcf-9c4d-4cba-bc80-de298505458f-istio-token" (OuterVolumeSpecName: "istio-token") pod "c24f5fcf-9c4d-4cba-bc80-de298505458f" (UID: "c24f5fcf-9c4d-4cba-bc80-de298505458f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:42:26.359841 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:26.359817    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c24f5fcf-9c4d-4cba-bc80-de298505458f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c24f5fcf-9c4d-4cba-bc80-de298505458f" (UID: "c24f5fcf-9c4d-4cba-bc80-de298505458f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 14:42:26.360194 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:26.360166    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c24f5fcf-9c4d-4cba-bc80-de298505458f-kube-api-access-5kj89" (OuterVolumeSpecName: "kube-api-access-5kj89") pod "c24f5fcf-9c4d-4cba-bc80-de298505458f" (UID: "c24f5fcf-9c4d-4cba-bc80-de298505458f"). InnerVolumeSpecName "kube-api-access-5kj89". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:42:26.457008 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:26.456975    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c24f5fcf-9c4d-4cba-bc80-de298505458f-istio-data\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:42:26.457008 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:26.457004    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c24f5fcf-9c4d-4cba-bc80-de298505458f-istio-envoy\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:42:26.457008 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:26.457013    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c24f5fcf-9c4d-4cba-bc80-de298505458f-istio-podinfo\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:42:26.457249 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:26.457021    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c24f5fcf-9c4d-4cba-bc80-de298505458f-istiod-ca-cert\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:42:26.457249 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:26.457029    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c24f5fcf-9c4d-4cba-bc80-de298505458f-istio-token\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:42:26.457249 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:26.457037    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5kj89\" (UniqueName: \"kubernetes.io/projected/c24f5fcf-9c4d-4cba-bc80-de298505458f-kube-api-access-5kj89\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:42:26.457249 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:26.457046    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c24f5fcf-9c4d-4cba-bc80-de298505458f-workload-socket\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:42:26.457249 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:26.457076    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c24f5fcf-9c4d-4cba-bc80-de298505458f-workload-certs\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:42:26.457249 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:26.457085    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c24f5fcf-9c4d-4cba-bc80-de298505458f-credential-socket\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:42:27.056120 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:27.056079    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x" event={"ID":"11db5568-30e2-4d34-9a60-2b7d4e5670e3","Type":"ContainerStarted","Data":"7b12eec7d782bcb3c46eb062c59c80ede193a0d2a197db5badebf10f5691338b"}
Jul 29 14:42:27.056384 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:27.056151    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x"
Jul 29 14:42:27.057664 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:27.057638    2559 generic.go:356] "Generic (PLEG): container finished" podID="c24f5fcf-9c4d-4cba-bc80-de298505458f" containerID="082ab44b95a9b23eb36a4f5372f986bcefc419ddd86178577e8c34696a8bcf9f" exitCode=0
Jul 29 14:42:27.057794 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:27.057675    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq" event={"ID":"c24f5fcf-9c4d-4cba-bc80-de298505458f","Type":"ContainerDied","Data":"082ab44b95a9b23eb36a4f5372f986bcefc419ddd86178577e8c34696a8bcf9f"}
Jul 29 14:42:27.057794 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:27.057683    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"
Jul 29 14:42:27.057794 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:27.057706    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq" event={"ID":"c24f5fcf-9c4d-4cba-bc80-de298505458f","Type":"ContainerDied","Data":"ac51e1f0e899e81f3ee1d12f8166dbcdf908bf386e751ac2449dde06e2ecfb91"}
Jul 29 14:42:27.057794 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:27.057727    2559 scope.go:117] "RemoveContainer" containerID="082ab44b95a9b23eb36a4f5372f986bcefc419ddd86178577e8c34696a8bcf9f"
Jul 29 14:42:27.066149 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:27.066120    2559 scope.go:117] "RemoveContainer" containerID="75000fc999218eb17b846671fe55b0f352056b04c55609eaaed2432bf8f314ef"
Jul 29 14:42:27.073639 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:27.073601    2559 scope.go:117] "RemoveContainer" containerID="082ab44b95a9b23eb36a4f5372f986bcefc419ddd86178577e8c34696a8bcf9f"
Jul 29 14:42:27.073916 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:42:27.073886    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"082ab44b95a9b23eb36a4f5372f986bcefc419ddd86178577e8c34696a8bcf9f\": container with ID starting with 082ab44b95a9b23eb36a4f5372f986bcefc419ddd86178577e8c34696a8bcf9f not found: ID does not exist" containerID="082ab44b95a9b23eb36a4f5372f986bcefc419ddd86178577e8c34696a8bcf9f"
Jul 29 14:42:27.074138 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:27.073923    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"082ab44b95a9b23eb36a4f5372f986bcefc419ddd86178577e8c34696a8bcf9f"} err="failed to get container status \"082ab44b95a9b23eb36a4f5372f986bcefc419ddd86178577e8c34696a8bcf9f\": rpc error: code = NotFound desc = could not find container \"082ab44b95a9b23eb36a4f5372f986bcefc419ddd86178577e8c34696a8bcf9f\": container with ID starting with 082ab44b95a9b23eb36a4f5372f986bcefc419ddd86178577e8c34696a8bcf9f not found: ID does not exist"
Jul 29 14:42:27.074138 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:27.073947    2559 scope.go:117] "RemoveContainer" containerID="75000fc999218eb17b846671fe55b0f352056b04c55609eaaed2432bf8f314ef"
Jul 29 14:42:27.074277 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:42:27.074251    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75000fc999218eb17b846671fe55b0f352056b04c55609eaaed2432bf8f314ef\": container with ID starting with 75000fc999218eb17b846671fe55b0f352056b04c55609eaaed2432bf8f314ef not found: ID does not exist" containerID="75000fc999218eb17b846671fe55b0f352056b04c55609eaaed2432bf8f314ef"
Jul 29 14:42:27.074331 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:27.074286    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75000fc999218eb17b846671fe55b0f352056b04c55609eaaed2432bf8f314ef"} err="failed to get container status \"75000fc999218eb17b846671fe55b0f352056b04c55609eaaed2432bf8f314ef\": rpc error: code = NotFound desc = could not find container \"75000fc999218eb17b846671fe55b0f352056b04c55609eaaed2432bf8f314ef\": container with ID starting with 75000fc999218eb17b846671fe55b0f352056b04c55609eaaed2432bf8f314ef not found: ID does not exist"
Jul 29 14:42:27.074885 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:27.074847    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x" podStartSLOduration=1.6118611710000001 podStartE2EDuration="2.074835005s" totalImagesPullingTime="462.973834ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:42:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:42:27.07398117 +0000 UTC m=+2476.964652583" watchObservedRunningTime="2026-07-29 14:42:27.074835005 +0000 UTC m=+2476.965506357"
Jul 29 14:42:27.088974 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:27.088943    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"]
Jul 29 14:42:27.096178 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:27.096157    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-ffdf30-openshift-default-9494c9448-fpvsq"]
Jul 29 14:42:28.636503 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:28.636472    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c24f5fcf-9c4d-4cba-bc80-de298505458f" path="/var/lib/kubelet/pods/c24f5fcf-9c4d-4cba-bc80-de298505458f/volumes"
Jul 29 14:42:33.068948 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:33.068911    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x"
Jul 29 14:42:50.239011 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:50.238975    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:42:50.648894 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:50.648859    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:42:50.933804 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:50.933699    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"]
Jul 29 14:42:50.934038 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:50.934025    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c24f5fcf-9c4d-4cba-bc80-de298505458f" containerName="istio-proxy"
Jul 29 14:42:50.934104 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:50.934038    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c24f5fcf-9c4d-4cba-bc80-de298505458f" containerName="istio-proxy"
Jul 29 14:42:50.934104 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:50.934048    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c24f5fcf-9c4d-4cba-bc80-de298505458f" containerName="istio-proxy"
Jul 29 14:42:50.934104 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:50.934053    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c24f5fcf-9c4d-4cba-bc80-de298505458f" containerName="istio-proxy"
Jul 29 14:42:50.934206 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:50.934116    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="c24f5fcf-9c4d-4cba-bc80-de298505458f" containerName="istio-proxy"
Jul 29 14:42:50.934206 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:50.934126    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="c24f5fcf-9c4d-4cba-bc80-de298505458f" containerName="istio-proxy"
Jul 29 14:42:50.936896 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:50.936880    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:42:50.940445 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:50.940427    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-532c20-openshift-default-dockercfg-wr4bs\""
Jul 29 14:42:50.954816 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:50.954793    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"]
Jul 29 14:42:50.968802 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:50.968775    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:42:51.065357 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:51.065323    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tghcv\" (UniqueName: \"kubernetes.io/projected/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-kube-api-access-tghcv\") pod \"e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") " pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:42:51.065526 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:51.065392    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-istio-data\") pod \"e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") " pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:42:51.065526 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:51.065464    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-istio-token\") pod \"e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") " pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:42:51.065526 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:51.065513    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-workload-certs\") pod \"e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") " pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:42:51.065657 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:51.065558    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-credential-socket\") pod \"e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") " pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:42:51.065657 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:51.065618    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-workload-socket\") pod \"e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") " pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:42:51.065657 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:51.065638    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-istio-envoy\") pod \"e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") " pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:42:51.065657 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:51.065653    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-istio-podinfo\") pod \"e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") " pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:42:51.065806 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:51.065668    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-istiod-ca-cert\") pod \"e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") " pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:42:51.066187 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:51.066167    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:42:51.166366 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:51.166335    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-istio-data\") pod \"e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") " pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:42:51.166366 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:51.166376    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-istio-token\") pod \"e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") " pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:42:51.166633 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:51.166404    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-workload-certs\") pod \"e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") " pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:42:51.166633 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:51.166436    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-credential-socket\") pod \"e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") " pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:42:51.166633 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:51.166464    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-workload-socket\") pod \"e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") " pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:42:51.166633 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:51.166491    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-istio-envoy\") pod \"e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") " pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:42:51.166633 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:51.166510    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-istio-podinfo\") pod \"e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") " pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:42:51.166633 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:51.166534    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-istiod-ca-cert\") pod \"e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") " pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:42:51.166955 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:51.166657    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tghcv\" (UniqueName: \"kubernetes.io/projected/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-kube-api-access-tghcv\") pod \"e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") " pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:42:51.166955 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:51.166827    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-istio-data\") pod \"e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") " pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:42:51.166955 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:51.166918    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-credential-socket\") pod \"e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") " pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:42:51.167166 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:51.167142    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-workload-socket\") pod \"e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") " pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:42:51.167292 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:51.167269    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-workload-certs\") pod \"e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") " pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:42:51.167521 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:51.167493    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-istiod-ca-cert\") pod \"e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") " pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:42:51.168958 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:51.168934    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-istio-envoy\") pod \"e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") " pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:42:51.169128 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:51.169108    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-istio-podinfo\") pod \"e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") " pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:42:51.176179 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:51.176151    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-istio-token\") pod \"e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") " pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:42:51.176289 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:51.176196    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tghcv\" (UniqueName: \"kubernetes.io/projected/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-kube-api-access-tghcv\") pod \"e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") " pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:42:51.246994 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:51.246965    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:42:51.381560 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:51.381521    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"]
Jul 29 14:42:51.383773 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:42:51.383738    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee8a4c85_69ec_44a7_88a8_8f3e3984c33c.slice/crio-8035dc4b8bc5e5be20569a92d346bd739e4d3b41cebb8b4cfcc07610cdc7d4c5 WatchSource:0}: Error finding container 8035dc4b8bc5e5be20569a92d346bd739e4d3b41cebb8b4cfcc07610cdc7d4c5: Status 404 returned error can't find the container with id 8035dc4b8bc5e5be20569a92d346bd739e4d3b41cebb8b4cfcc07610cdc7d4c5
Jul 29 14:42:51.385889 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:51.385851    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:42:51.385979 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:51.385922    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:42:51.385979 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:51.385953    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:42:52.174595 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:52.174563    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8" event={"ID":"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c","Type":"ContainerStarted","Data":"bb1e48a49013d02a22f2c917a7dfce3478fb3ba6ab6148983d767ac45b840bcb"}
Jul 29 14:42:52.174772 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:52.174612    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8" event={"ID":"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c","Type":"ContainerStarted","Data":"8035dc4b8bc5e5be20569a92d346bd739e4d3b41cebb8b4cfcc07610cdc7d4c5"}
Jul 29 14:42:52.197577 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:52.197523    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8" podStartSLOduration=2.197506389 podStartE2EDuration="2.197506389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:42:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:42:52.195480878 +0000 UTC m=+2502.086152213" watchObservedRunningTime="2026-07-29 14:42:52.197506389 +0000 UTC m=+2502.088177723"
Jul 29 14:42:52.247871 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:52.247829    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:42:52.252802 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:52.252776    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:42:53.179500 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:53.179468    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:42:53.180562 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:53.180541    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:42:54.130249 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:54.130217    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:42:54.211553 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:54.211520    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:42:54.283299 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:54.283266    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:42:54.834189 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:54.834151    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:42:54.934274 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:54.934241    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:42:55.024447 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:42:55.024412    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:43:09.222242 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:09.222199    2559 patch_prober.go:28] interesting pod/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 29 14:43:09.222668 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:09.222282    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8" podUID="ee8a4c85-69ec-44a7-88a8-8f3e3984c33c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 29 14:43:11.261614 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:11.261583    2559 generic.go:356] "Generic (PLEG): container finished" podID="ee8a4c85-69ec-44a7-88a8-8f3e3984c33c" containerID="bb1e48a49013d02a22f2c917a7dfce3478fb3ba6ab6148983d767ac45b840bcb" exitCode=0
Jul 29 14:43:11.262025 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:11.261653    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8" event={"ID":"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c","Type":"ContainerDied","Data":"bb1e48a49013d02a22f2c917a7dfce3478fb3ba6ab6148983d767ac45b840bcb"}
Jul 29 14:43:11.262025 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:11.262015    2559 scope.go:117] "RemoveContainer" containerID="bb1e48a49013d02a22f2c917a7dfce3478fb3ba6ab6148983d767ac45b840bcb"
Jul 29 14:43:11.262973 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:11.262946    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:43:11.263028 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:11.263014    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:43:11.263083 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:11.263050    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:43:12.271175 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:12.271136    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8" event={"ID":"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c","Type":"ContainerStarted","Data":"0001c77f222d4e4acd31d8bc636d2fa9e83731793e4cb0fc6eab3ec4c0364bde"}
Jul 29 14:43:13.247262 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:13.247217    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:43:13.247563 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:13.247540    2559 patch_prober.go:28] interesting pod/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 29 14:43:13.247624 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:13.247602    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8" podUID="ee8a4c85-69ec-44a7-88a8-8f3e3984c33c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 29 14:43:14.248601 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:14.248556    2559 patch_prober.go:28] interesting pod/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 29 14:43:14.249274 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:14.249233    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8" podUID="ee8a4c85-69ec-44a7-88a8-8f3e3984c33c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 29 14:43:15.248173 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:15.248132    2559 patch_prober.go:28] interesting pod/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 29 14:43:15.248468 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:15.248437    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8" podUID="ee8a4c85-69ec-44a7-88a8-8f3e3984c33c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 29 14:43:16.251897 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:16.251857    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:43:16.252322 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:16.252153    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:43:16.253005 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:16.252985    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:43:19.138776 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:19.138744    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:43:31.267757 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:31.267720    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:43:31.368426 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:31.368391    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:43:34.947400 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:34.947360    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr"]
Jul 29 14:43:34.951043 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:34.951020    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr"
Jul 29 14:43:34.955185 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:34.955165    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-61bc0a\"/\"default-dockercfg-9vxjs\""
Jul 29 14:43:34.955349 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:34.955326    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-61bc0a\"/\"openshift-service-ca.crt\""
Jul 29 14:43:34.955418 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:34.955345    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-61bc0a\"/\"auth-test-model-61bc0a-kserve-self-signed-certs\""
Jul 29 14:43:34.955569 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:34.955543    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-61bc0a\"/\"kube-root-ca.crt\""
Jul 29 14:43:34.960606 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:34.960583    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr"]
Jul 29 14:43:34.981342 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:34.981316    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:43:35.047479 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:35.047451    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68150e83-ad19-42f6-aea3-5ce299c146de-home\") pod \"auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr\" (UID: \"68150e83-ad19-42f6-aea3-5ce299c146de\") " pod="ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr"
Jul 29 14:43:35.047647 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:35.047492    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68150e83-ad19-42f6-aea3-5ce299c146de-tmp-dir\") pod \"auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr\" (UID: \"68150e83-ad19-42f6-aea3-5ce299c146de\") " pod="ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr"
Jul 29 14:43:35.047647 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:35.047527    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68150e83-ad19-42f6-aea3-5ce299c146de-dshm\") pod \"auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr\" (UID: \"68150e83-ad19-42f6-aea3-5ce299c146de\") " pod="ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr"
Jul 29 14:43:35.047647 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:35.047563    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68150e83-ad19-42f6-aea3-5ce299c146de-tls-certs\") pod \"auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr\" (UID: \"68150e83-ad19-42f6-aea3-5ce299c146de\") " pod="ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr"
Jul 29 14:43:35.047647 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:35.047587    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f56nw\" (UniqueName: \"kubernetes.io/projected/68150e83-ad19-42f6-aea3-5ce299c146de-kube-api-access-f56nw\") pod \"auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr\" (UID: \"68150e83-ad19-42f6-aea3-5ce299c146de\") " pod="ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr"
Jul 29 14:43:35.047647 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:35.047634    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68150e83-ad19-42f6-aea3-5ce299c146de-model-cache\") pod \"auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr\" (UID: \"68150e83-ad19-42f6-aea3-5ce299c146de\") " pod="ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr"
Jul 29 14:43:35.148359 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:35.148325    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68150e83-ad19-42f6-aea3-5ce299c146de-model-cache\") pod \"auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr\" (UID: \"68150e83-ad19-42f6-aea3-5ce299c146de\") " pod="ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr"
Jul 29 14:43:35.148560 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:35.148385    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68150e83-ad19-42f6-aea3-5ce299c146de-home\") pod \"auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr\" (UID: \"68150e83-ad19-42f6-aea3-5ce299c146de\") " pod="ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr"
Jul 29 14:43:35.148560 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:35.148414    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68150e83-ad19-42f6-aea3-5ce299c146de-tmp-dir\") pod \"auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr\" (UID: \"68150e83-ad19-42f6-aea3-5ce299c146de\") " pod="ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr"
Jul 29 14:43:35.148560 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:35.148438    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68150e83-ad19-42f6-aea3-5ce299c146de-dshm\") pod \"auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr\" (UID: \"68150e83-ad19-42f6-aea3-5ce299c146de\") " pod="ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr"
Jul 29 14:43:35.148560 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:35.148468    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68150e83-ad19-42f6-aea3-5ce299c146de-tls-certs\") pod \"auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr\" (UID: \"68150e83-ad19-42f6-aea3-5ce299c146de\") " pod="ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr"
Jul 29 14:43:35.148560 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:35.148490    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f56nw\" (UniqueName: \"kubernetes.io/projected/68150e83-ad19-42f6-aea3-5ce299c146de-kube-api-access-f56nw\") pod \"auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr\" (UID: \"68150e83-ad19-42f6-aea3-5ce299c146de\") " pod="ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr"
Jul 29 14:43:35.148858 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:35.148834    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68150e83-ad19-42f6-aea3-5ce299c146de-home\") pod \"auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr\" (UID: \"68150e83-ad19-42f6-aea3-5ce299c146de\") " pod="ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr"
Jul 29 14:43:35.148918 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:35.148871    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68150e83-ad19-42f6-aea3-5ce299c146de-model-cache\") pod \"auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr\" (UID: \"68150e83-ad19-42f6-aea3-5ce299c146de\") " pod="ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr"
Jul 29 14:43:35.148918 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:35.148897    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68150e83-ad19-42f6-aea3-5ce299c146de-tmp-dir\") pod \"auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr\" (UID: \"68150e83-ad19-42f6-aea3-5ce299c146de\") " pod="ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr"
Jul 29 14:43:35.150776 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:35.150748    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68150e83-ad19-42f6-aea3-5ce299c146de-dshm\") pod \"auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr\" (UID: \"68150e83-ad19-42f6-aea3-5ce299c146de\") " pod="ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr"
Jul 29 14:43:35.150954 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:35.150936    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68150e83-ad19-42f6-aea3-5ce299c146de-tls-certs\") pod \"auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr\" (UID: \"68150e83-ad19-42f6-aea3-5ce299c146de\") " pod="ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr"
Jul 29 14:43:35.157184 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:35.157164    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f56nw\" (UniqueName: \"kubernetes.io/projected/68150e83-ad19-42f6-aea3-5ce299c146de-kube-api-access-f56nw\") pod \"auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr\" (UID: \"68150e83-ad19-42f6-aea3-5ce299c146de\") " pod="ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr"
Jul 29 14:43:35.261043 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:35.260999    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr"
Jul 29 14:43:35.391843 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:35.391818    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr"]
Jul 29 14:43:35.394280 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:43:35.394254    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod68150e83_ad19_42f6_aea3_5ce299c146de.slice/crio-7786a6f57ae3e48142ada8dcf2e2cb19de8646c848053efe4de705234790bb7f WatchSource:0}: Error finding container 7786a6f57ae3e48142ada8dcf2e2cb19de8646c848053efe4de705234790bb7f: Status 404 returned error can't find the container with id 7786a6f57ae3e48142ada8dcf2e2cb19de8646c848053efe4de705234790bb7f
Jul 29 14:43:36.379786 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:36.379754    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr" event={"ID":"68150e83-ad19-42f6-aea3-5ce299c146de","Type":"ContainerStarted","Data":"573a9ba098bd7371b1a7640b8a3e9272aecb5180219f97493167a9f58c16be54"}
Jul 29 14:43:36.380184 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:36.379795    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr" event={"ID":"68150e83-ad19-42f6-aea3-5ce299c146de","Type":"ContainerStarted","Data":"7786a6f57ae3e48142ada8dcf2e2cb19de8646c848053efe4de705234790bb7f"}
Jul 29 14:43:36.398017 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:36.397975    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr" podStartSLOduration=2.397960868 podStartE2EDuration="2.397960868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:43:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:43:36.396607185 +0000 UTC m=+2546.287278514" watchObservedRunningTime="2026-07-29 14:43:36.397960868 +0000 UTC m=+2546.288632203"
Jul 29 14:43:45.261135 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:45.261098    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr"
Jul 29 14:43:45.261717 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:45.261154    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr"
Jul 29 14:43:45.274051 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:45.274022    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr"
Jul 29 14:43:45.431438 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:43:45.431400    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr"
Jul 29 14:44:07.488404 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:07.488362    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:44:08.350870 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:08.350835    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2"]
Jul 29 14:44:08.354149 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:08.354124    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2"
Jul 29 14:44:08.357471 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:08.357447    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-532c20\"/\"openshift-service-ca.crt\""
Jul 29 14:44:08.358902 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:08.358879    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-532c20\"/\"auth-test-model-532c20-kserve-self-signed-certs\""
Jul 29 14:44:08.358902 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:08.358902    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-532c20\"/\"default-dockercfg-cjwft\""
Jul 29 14:44:08.359081 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:08.358927    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-532c20\"/\"kube-root-ca.crt\""
Jul 29 14:44:08.363656 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:08.363632    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2"]
Jul 29 14:44:08.378179 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:08.378150    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:44:08.441525 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:08.441491    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65db1672-804f-4f6b-bc4c-ea267bd35b7b-tls-certs\") pod \"auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2\" (UID: \"65db1672-804f-4f6b-bc4c-ea267bd35b7b\") " pod="ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2"
Jul 29 14:44:08.441656 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:08.441546    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65db1672-804f-4f6b-bc4c-ea267bd35b7b-home\") pod \"auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2\" (UID: \"65db1672-804f-4f6b-bc4c-ea267bd35b7b\") " pod="ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2"
Jul 29 14:44:08.441656 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:08.441611    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65db1672-804f-4f6b-bc4c-ea267bd35b7b-tmp-dir\") pod \"auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2\" (UID: \"65db1672-804f-4f6b-bc4c-ea267bd35b7b\") " pod="ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2"
Jul 29 14:44:08.441656 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:08.441643    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l2qx\" (UniqueName: \"kubernetes.io/projected/65db1672-804f-4f6b-bc4c-ea267bd35b7b-kube-api-access-5l2qx\") pod \"auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2\" (UID: \"65db1672-804f-4f6b-bc4c-ea267bd35b7b\") " pod="ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2"
Jul 29 14:44:08.441764 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:08.441666    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65db1672-804f-4f6b-bc4c-ea267bd35b7b-model-cache\") pod \"auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2\" (UID: \"65db1672-804f-4f6b-bc4c-ea267bd35b7b\") " pod="ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2"
Jul 29 14:44:08.441764 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:08.441685    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65db1672-804f-4f6b-bc4c-ea267bd35b7b-dshm\") pod \"auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2\" (UID: \"65db1672-804f-4f6b-bc4c-ea267bd35b7b\") " pod="ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2"
Jul 29 14:44:08.542316 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:08.542283    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65db1672-804f-4f6b-bc4c-ea267bd35b7b-home\") pod \"auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2\" (UID: \"65db1672-804f-4f6b-bc4c-ea267bd35b7b\") " pod="ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2"
Jul 29 14:44:08.542742 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:08.542328    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65db1672-804f-4f6b-bc4c-ea267bd35b7b-tmp-dir\") pod \"auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2\" (UID: \"65db1672-804f-4f6b-bc4c-ea267bd35b7b\") " pod="ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2"
Jul 29 14:44:08.542742 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:08.542451    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l2qx\" (UniqueName: \"kubernetes.io/projected/65db1672-804f-4f6b-bc4c-ea267bd35b7b-kube-api-access-5l2qx\") pod \"auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2\" (UID: \"65db1672-804f-4f6b-bc4c-ea267bd35b7b\") " pod="ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2"
Jul 29 14:44:08.542742 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:08.542489    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65db1672-804f-4f6b-bc4c-ea267bd35b7b-model-cache\") pod \"auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2\" (UID: \"65db1672-804f-4f6b-bc4c-ea267bd35b7b\") " pod="ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2"
Jul 29 14:44:08.542742 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:08.542520    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65db1672-804f-4f6b-bc4c-ea267bd35b7b-dshm\") pod \"auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2\" (UID: \"65db1672-804f-4f6b-bc4c-ea267bd35b7b\") " pod="ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2"
Jul 29 14:44:08.542742 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:08.542602    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65db1672-804f-4f6b-bc4c-ea267bd35b7b-tls-certs\") pod \"auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2\" (UID: \"65db1672-804f-4f6b-bc4c-ea267bd35b7b\") " pod="ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2"
Jul 29 14:44:08.542742 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:08.542669    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65db1672-804f-4f6b-bc4c-ea267bd35b7b-home\") pod \"auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2\" (UID: \"65db1672-804f-4f6b-bc4c-ea267bd35b7b\") " pod="ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2"
Jul 29 14:44:08.542742 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:08.542708    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65db1672-804f-4f6b-bc4c-ea267bd35b7b-tmp-dir\") pod \"auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2\" (UID: \"65db1672-804f-4f6b-bc4c-ea267bd35b7b\") " pod="ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2"
Jul 29 14:44:08.543038 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:08.542836    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65db1672-804f-4f6b-bc4c-ea267bd35b7b-model-cache\") pod \"auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2\" (UID: \"65db1672-804f-4f6b-bc4c-ea267bd35b7b\") " pod="ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2"
Jul 29 14:44:08.544858 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:08.544827    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65db1672-804f-4f6b-bc4c-ea267bd35b7b-dshm\") pod \"auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2\" (UID: \"65db1672-804f-4f6b-bc4c-ea267bd35b7b\") " pod="ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2"
Jul 29 14:44:08.545097 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:08.545077    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65db1672-804f-4f6b-bc4c-ea267bd35b7b-tls-certs\") pod \"auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2\" (UID: \"65db1672-804f-4f6b-bc4c-ea267bd35b7b\") " pod="ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2"
Jul 29 14:44:08.551940 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:08.551917    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l2qx\" (UniqueName: \"kubernetes.io/projected/65db1672-804f-4f6b-bc4c-ea267bd35b7b-kube-api-access-5l2qx\") pod \"auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2\" (UID: \"65db1672-804f-4f6b-bc4c-ea267bd35b7b\") " pod="ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2"
Jul 29 14:44:08.664679 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:08.664601    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2"
Jul 29 14:44:08.793500 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:08.793375    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2"]
Jul 29 14:44:08.796222 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:44:08.796197    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod65db1672_804f_4f6b_bc4c_ea267bd35b7b.slice/crio-880dc931ffef98db689989136c6e8126312745db3654d7c21c6999388131a897 WatchSource:0}: Error finding container 880dc931ffef98db689989136c6e8126312745db3654d7c21c6999388131a897: Status 404 returned error can't find the container with id 880dc931ffef98db689989136c6e8126312745db3654d7c21c6999388131a897
Jul 29 14:44:09.521777 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:09.521736    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2" event={"ID":"65db1672-804f-4f6b-bc4c-ea267bd35b7b","Type":"ContainerStarted","Data":"309bbec09ec837cb27e968626e065372785612e681583f8a5faf3143a0e36072"}
Jul 29 14:44:09.521777 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:09.521781    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2" event={"ID":"65db1672-804f-4f6b-bc4c-ea267bd35b7b","Type":"ContainerStarted","Data":"880dc931ffef98db689989136c6e8126312745db3654d7c21c6999388131a897"}
Jul 29 14:44:09.540626 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:09.540579    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2" podStartSLOduration=1.540565169 podStartE2EDuration="1.540565169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:44:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:44:09.538540351 +0000 UTC m=+2579.429211687" watchObservedRunningTime="2026-07-29 14:44:09.540565169 +0000 UTC m=+2579.431236555"
Jul 29 14:44:18.665671 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:18.665635    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2"
Jul 29 14:44:18.666215 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:18.665689    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2"
Jul 29 14:44:18.677978 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:18.677949    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2"
Jul 29 14:44:19.575645 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:44:19.575618    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2"
Jul 29 14:45:27.259128 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:45:27.259090    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:45:27.358890 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:45:27.358856    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:46:10.749324 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:46:10.749247    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdw8l_b93f6faa-c8bf-43e9-94e0-b3dcd97b7296/ovn-acl-logging/0.log"
Jul 29 14:46:10.756407 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:46:10.756385    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdw8l_b93f6faa-c8bf-43e9-94e0-b3dcd97b7296/ovn-acl-logging/0.log"
Jul 29 14:47:10.877761 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:47:10.877730    2559 scope.go:117] "RemoveContainer" containerID="9740d3a9392b8884f86967594f984dfa2d41fdb89722b475b624218dc8105736"
Jul 29 14:48:39.286162 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:48:39.286127    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:48:50.931855 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:48:50.931823    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr"]
Jul 29 14:48:50.955634 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:48:50.955608    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:48:52.054783 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:48:52.054746    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2"]
Jul 29 14:48:52.070108 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:48:52.070081    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:49:02.961941 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:02.961858    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6"]
Jul 29 14:49:02.965041 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:02.965023    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6"
Jul 29 14:49:02.967730 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:02.967704    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-61bc0a\"/\"sub-test-model-61bc0a-kserve-self-signed-certs\""
Jul 29 14:49:02.975633 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:02.975611    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6"]
Jul 29 14:49:02.984933 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:02.984858    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:49:03.015918 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:03.015892    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13919ccb-c701-4721-8ff5-a715e9399413-dshm\") pod \"sub-test-model-61bc0a-kserve-66758b5669-t84h6\" (UID: \"13919ccb-c701-4721-8ff5-a715e9399413\") " pod="ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6"
Jul 29 14:49:03.016012 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:03.015934    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64dls\" (UniqueName: \"kubernetes.io/projected/13919ccb-c701-4721-8ff5-a715e9399413-kube-api-access-64dls\") pod \"sub-test-model-61bc0a-kserve-66758b5669-t84h6\" (UID: \"13919ccb-c701-4721-8ff5-a715e9399413\") " pod="ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6"
Jul 29 14:49:03.016012 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:03.015956    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13919ccb-c701-4721-8ff5-a715e9399413-model-cache\") pod \"sub-test-model-61bc0a-kserve-66758b5669-t84h6\" (UID: \"13919ccb-c701-4721-8ff5-a715e9399413\") " pod="ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6"
Jul 29 14:49:03.016114 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:03.016022    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13919ccb-c701-4721-8ff5-a715e9399413-home\") pod \"sub-test-model-61bc0a-kserve-66758b5669-t84h6\" (UID: \"13919ccb-c701-4721-8ff5-a715e9399413\") " pod="ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6"
Jul 29 14:49:03.016114 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:03.016038    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13919ccb-c701-4721-8ff5-a715e9399413-tls-certs\") pod \"sub-test-model-61bc0a-kserve-66758b5669-t84h6\" (UID: \"13919ccb-c701-4721-8ff5-a715e9399413\") " pod="ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6"
Jul 29 14:49:03.016114 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:03.016099    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13919ccb-c701-4721-8ff5-a715e9399413-tmp-dir\") pod \"sub-test-model-61bc0a-kserve-66758b5669-t84h6\" (UID: \"13919ccb-c701-4721-8ff5-a715e9399413\") " pod="ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6"
Jul 29 14:49:03.116754 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:03.116722    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13919ccb-c701-4721-8ff5-a715e9399413-dshm\") pod \"sub-test-model-61bc0a-kserve-66758b5669-t84h6\" (UID: \"13919ccb-c701-4721-8ff5-a715e9399413\") " pod="ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6"
Jul 29 14:49:03.116893 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:03.116770    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64dls\" (UniqueName: \"kubernetes.io/projected/13919ccb-c701-4721-8ff5-a715e9399413-kube-api-access-64dls\") pod \"sub-test-model-61bc0a-kserve-66758b5669-t84h6\" (UID: \"13919ccb-c701-4721-8ff5-a715e9399413\") " pod="ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6"
Jul 29 14:49:03.116893 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:03.116864    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13919ccb-c701-4721-8ff5-a715e9399413-model-cache\") pod \"sub-test-model-61bc0a-kserve-66758b5669-t84h6\" (UID: \"13919ccb-c701-4721-8ff5-a715e9399413\") " pod="ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6"
Jul 29 14:49:03.116982 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:03.116899    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13919ccb-c701-4721-8ff5-a715e9399413-home\") pod \"sub-test-model-61bc0a-kserve-66758b5669-t84h6\" (UID: \"13919ccb-c701-4721-8ff5-a715e9399413\") " pod="ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6"
Jul 29 14:49:03.116982 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:03.116914    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13919ccb-c701-4721-8ff5-a715e9399413-tls-certs\") pod \"sub-test-model-61bc0a-kserve-66758b5669-t84h6\" (UID: \"13919ccb-c701-4721-8ff5-a715e9399413\") " pod="ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6"
Jul 29 14:49:03.116982 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:03.116951    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13919ccb-c701-4721-8ff5-a715e9399413-tmp-dir\") pod \"sub-test-model-61bc0a-kserve-66758b5669-t84h6\" (UID: \"13919ccb-c701-4721-8ff5-a715e9399413\") " pod="ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6"
Jul 29 14:49:03.117316 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:03.117294    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13919ccb-c701-4721-8ff5-a715e9399413-model-cache\") pod \"sub-test-model-61bc0a-kserve-66758b5669-t84h6\" (UID: \"13919ccb-c701-4721-8ff5-a715e9399413\") " pod="ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6"
Jul 29 14:49:03.117418 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:03.117329    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13919ccb-c701-4721-8ff5-a715e9399413-tmp-dir\") pod \"sub-test-model-61bc0a-kserve-66758b5669-t84h6\" (UID: \"13919ccb-c701-4721-8ff5-a715e9399413\") " pod="ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6"
Jul 29 14:49:03.117418 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:03.117364    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13919ccb-c701-4721-8ff5-a715e9399413-home\") pod \"sub-test-model-61bc0a-kserve-66758b5669-t84h6\" (UID: \"13919ccb-c701-4721-8ff5-a715e9399413\") " pod="ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6"
Jul 29 14:49:03.118921 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:03.118904    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13919ccb-c701-4721-8ff5-a715e9399413-dshm\") pod \"sub-test-model-61bc0a-kserve-66758b5669-t84h6\" (UID: \"13919ccb-c701-4721-8ff5-a715e9399413\") " pod="ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6"
Jul 29 14:49:03.119316 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:03.119295    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13919ccb-c701-4721-8ff5-a715e9399413-tls-certs\") pod \"sub-test-model-61bc0a-kserve-66758b5669-t84h6\" (UID: \"13919ccb-c701-4721-8ff5-a715e9399413\") " pod="ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6"
Jul 29 14:49:03.125531 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:03.125504    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64dls\" (UniqueName: \"kubernetes.io/projected/13919ccb-c701-4721-8ff5-a715e9399413-kube-api-access-64dls\") pod \"sub-test-model-61bc0a-kserve-66758b5669-t84h6\" (UID: \"13919ccb-c701-4721-8ff5-a715e9399413\") " pod="ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6"
Jul 29 14:49:03.273973 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:03.273941    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6"
Jul 29 14:49:03.431256 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:03.431221    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6"]
Jul 29 14:49:03.432805 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:49:03.432774    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod13919ccb_c701_4721_8ff5_a715e9399413.slice/crio-33a79dab0e54a5542644bda948f977368c68a6202d34eff177e1eb9b179d09f2 WatchSource:0}: Error finding container 33a79dab0e54a5542644bda948f977368c68a6202d34eff177e1eb9b179d09f2: Status 404 returned error can't find the container with id 33a79dab0e54a5542644bda948f977368c68a6202d34eff177e1eb9b179d09f2
Jul 29 14:49:03.794769 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:03.794738    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6" event={"ID":"13919ccb-c701-4721-8ff5-a715e9399413","Type":"ContainerStarted","Data":"a1a05fa4ce4d14a730bdbe2b88977fcfadba4527fd58fbe052489007fd6fbf03"}
Jul 29 14:49:03.794934 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:03.794775    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6" event={"ID":"13919ccb-c701-4721-8ff5-a715e9399413","Type":"ContainerStarted","Data":"33a79dab0e54a5542644bda948f977368c68a6202d34eff177e1eb9b179d09f2"}
Jul 29 14:49:03.811642 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:03.811587    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6" podStartSLOduration=1.811568303 podStartE2EDuration="1.811568303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:49:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:49:03.811475517 +0000 UTC m=+2873.702146852" watchObservedRunningTime="2026-07-29 14:49:03.811568303 +0000 UTC m=+2873.702239639"
Jul 29 14:49:05.950461 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:05.950394    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr" podUID="68150e83-ad19-42f6-aea3-5ce299c146de" containerName="main" containerID="cri-o://573a9ba098bd7371b1a7640b8a3e9272aecb5180219f97493167a9f58c16be54" gracePeriod=45
Jul 29 14:49:06.195199 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:06.195175    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr"
Jul 29 14:49:06.245546 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:06.245517    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68150e83-ad19-42f6-aea3-5ce299c146de-home\") pod \"68150e83-ad19-42f6-aea3-5ce299c146de\" (UID: \"68150e83-ad19-42f6-aea3-5ce299c146de\") "
Jul 29 14:49:06.245724 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:06.245570    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68150e83-ad19-42f6-aea3-5ce299c146de-model-cache\") pod \"68150e83-ad19-42f6-aea3-5ce299c146de\" (UID: \"68150e83-ad19-42f6-aea3-5ce299c146de\") "
Jul 29 14:49:06.245724 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:06.245599    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f56nw\" (UniqueName: \"kubernetes.io/projected/68150e83-ad19-42f6-aea3-5ce299c146de-kube-api-access-f56nw\") pod \"68150e83-ad19-42f6-aea3-5ce299c146de\" (UID: \"68150e83-ad19-42f6-aea3-5ce299c146de\") "
Jul 29 14:49:06.245724 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:06.245662    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68150e83-ad19-42f6-aea3-5ce299c146de-tls-certs\") pod \"68150e83-ad19-42f6-aea3-5ce299c146de\" (UID: \"68150e83-ad19-42f6-aea3-5ce299c146de\") "
Jul 29 14:49:06.245724 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:06.245692    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68150e83-ad19-42f6-aea3-5ce299c146de-dshm\") pod \"68150e83-ad19-42f6-aea3-5ce299c146de\" (UID: \"68150e83-ad19-42f6-aea3-5ce299c146de\") "
Jul 29 14:49:06.245933 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:06.245734    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68150e83-ad19-42f6-aea3-5ce299c146de-tmp-dir\") pod \"68150e83-ad19-42f6-aea3-5ce299c146de\" (UID: \"68150e83-ad19-42f6-aea3-5ce299c146de\") "
Jul 29 14:49:06.245933 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:06.245765    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68150e83-ad19-42f6-aea3-5ce299c146de-home" (OuterVolumeSpecName: "home") pod "68150e83-ad19-42f6-aea3-5ce299c146de" (UID: "68150e83-ad19-42f6-aea3-5ce299c146de"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:49:06.245933 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:06.245859    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68150e83-ad19-42f6-aea3-5ce299c146de-model-cache" (OuterVolumeSpecName: "model-cache") pod "68150e83-ad19-42f6-aea3-5ce299c146de" (UID: "68150e83-ad19-42f6-aea3-5ce299c146de"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:49:06.246104 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:06.246027    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68150e83-ad19-42f6-aea3-5ce299c146de-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "68150e83-ad19-42f6-aea3-5ce299c146de" (UID: "68150e83-ad19-42f6-aea3-5ce299c146de"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:49:06.246168 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:06.246105    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68150e83-ad19-42f6-aea3-5ce299c146de-home\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:49:06.246168 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:06.246122    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68150e83-ad19-42f6-aea3-5ce299c146de-model-cache\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:49:06.247970 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:06.247947    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68150e83-ad19-42f6-aea3-5ce299c146de-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "68150e83-ad19-42f6-aea3-5ce299c146de" (UID: "68150e83-ad19-42f6-aea3-5ce299c146de"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 14:49:06.248077 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:06.247974    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68150e83-ad19-42f6-aea3-5ce299c146de-kube-api-access-f56nw" (OuterVolumeSpecName: "kube-api-access-f56nw") pod "68150e83-ad19-42f6-aea3-5ce299c146de" (UID: "68150e83-ad19-42f6-aea3-5ce299c146de"). InnerVolumeSpecName "kube-api-access-f56nw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:49:06.248077 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:06.247983    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68150e83-ad19-42f6-aea3-5ce299c146de-dshm" (OuterVolumeSpecName: "dshm") pod "68150e83-ad19-42f6-aea3-5ce299c146de" (UID: "68150e83-ad19-42f6-aea3-5ce299c146de"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:49:06.346949 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:06.346914    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f56nw\" (UniqueName: \"kubernetes.io/projected/68150e83-ad19-42f6-aea3-5ce299c146de-kube-api-access-f56nw\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:49:06.346949 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:06.346943    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68150e83-ad19-42f6-aea3-5ce299c146de-tls-certs\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:49:06.346949 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:06.346953    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68150e83-ad19-42f6-aea3-5ce299c146de-dshm\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:49:06.347197 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:06.346961    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68150e83-ad19-42f6-aea3-5ce299c146de-tmp-dir\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:49:06.809161 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:06.809127    2559 generic.go:356] "Generic (PLEG): container finished" podID="68150e83-ad19-42f6-aea3-5ce299c146de" containerID="573a9ba098bd7371b1a7640b8a3e9272aecb5180219f97493167a9f58c16be54" exitCode=0
Jul 29 14:49:06.809327 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:06.809168    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr"
Jul 29 14:49:06.809327 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:06.809204    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr" event={"ID":"68150e83-ad19-42f6-aea3-5ce299c146de","Type":"ContainerDied","Data":"573a9ba098bd7371b1a7640b8a3e9272aecb5180219f97493167a9f58c16be54"}
Jul 29 14:49:06.809327 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:06.809249    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr" event={"ID":"68150e83-ad19-42f6-aea3-5ce299c146de","Type":"ContainerDied","Data":"7786a6f57ae3e48142ada8dcf2e2cb19de8646c848053efe4de705234790bb7f"}
Jul 29 14:49:06.809327 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:06.809272    2559 scope.go:117] "RemoveContainer" containerID="573a9ba098bd7371b1a7640b8a3e9272aecb5180219f97493167a9f58c16be54"
Jul 29 14:49:06.817543 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:06.817522    2559 scope.go:117] "RemoveContainer" containerID="573a9ba098bd7371b1a7640b8a3e9272aecb5180219f97493167a9f58c16be54"
Jul 29 14:49:06.817800 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:49:06.817782    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"573a9ba098bd7371b1a7640b8a3e9272aecb5180219f97493167a9f58c16be54\": container with ID starting with 573a9ba098bd7371b1a7640b8a3e9272aecb5180219f97493167a9f58c16be54 not found: ID does not exist" containerID="573a9ba098bd7371b1a7640b8a3e9272aecb5180219f97493167a9f58c16be54"
Jul 29 14:49:06.817846 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:06.817808    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"573a9ba098bd7371b1a7640b8a3e9272aecb5180219f97493167a9f58c16be54"} err="failed to get container status \"573a9ba098bd7371b1a7640b8a3e9272aecb5180219f97493167a9f58c16be54\": rpc error: code = NotFound desc = could not find container \"573a9ba098bd7371b1a7640b8a3e9272aecb5180219f97493167a9f58c16be54\": container with ID starting with 573a9ba098bd7371b1a7640b8a3e9272aecb5180219f97493167a9f58c16be54 not found: ID does not exist"
Jul 29 14:49:06.825209 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:06.825185    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr"]
Jul 29 14:49:06.831102 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:06.831082    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-61bc0a/auth-test-model-61bc0a-kserve-85c94bd5ff-7twsr"]
Jul 29 14:49:07.073524 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:07.073485    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2" podUID="65db1672-804f-4f6b-bc4c-ea267bd35b7b" containerName="main" containerID="cri-o://309bbec09ec837cb27e968626e065372785612e681583f8a5faf3143a0e36072" gracePeriod=45
Jul 29 14:49:07.321026 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:07.321000    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2"
Jul 29 14:49:07.358597 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:07.358535    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65db1672-804f-4f6b-bc4c-ea267bd35b7b-tls-certs\") pod \"65db1672-804f-4f6b-bc4c-ea267bd35b7b\" (UID: \"65db1672-804f-4f6b-bc4c-ea267bd35b7b\") "
Jul 29 14:49:07.358597 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:07.358574    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5l2qx\" (UniqueName: \"kubernetes.io/projected/65db1672-804f-4f6b-bc4c-ea267bd35b7b-kube-api-access-5l2qx\") pod \"65db1672-804f-4f6b-bc4c-ea267bd35b7b\" (UID: \"65db1672-804f-4f6b-bc4c-ea267bd35b7b\") "
Jul 29 14:49:07.358597 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:07.358592    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65db1672-804f-4f6b-bc4c-ea267bd35b7b-dshm\") pod \"65db1672-804f-4f6b-bc4c-ea267bd35b7b\" (UID: \"65db1672-804f-4f6b-bc4c-ea267bd35b7b\") "
Jul 29 14:49:07.358848 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:07.358662    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65db1672-804f-4f6b-bc4c-ea267bd35b7b-home\") pod \"65db1672-804f-4f6b-bc4c-ea267bd35b7b\" (UID: \"65db1672-804f-4f6b-bc4c-ea267bd35b7b\") "
Jul 29 14:49:07.358848 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:07.358688    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65db1672-804f-4f6b-bc4c-ea267bd35b7b-tmp-dir\") pod \"65db1672-804f-4f6b-bc4c-ea267bd35b7b\" (UID: \"65db1672-804f-4f6b-bc4c-ea267bd35b7b\") "
Jul 29 14:49:07.358848 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:07.358723    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65db1672-804f-4f6b-bc4c-ea267bd35b7b-model-cache\") pod \"65db1672-804f-4f6b-bc4c-ea267bd35b7b\" (UID: \"65db1672-804f-4f6b-bc4c-ea267bd35b7b\") "
Jul 29 14:49:07.359016 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:07.358941    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65db1672-804f-4f6b-bc4c-ea267bd35b7b-home" (OuterVolumeSpecName: "home") pod "65db1672-804f-4f6b-bc4c-ea267bd35b7b" (UID: "65db1672-804f-4f6b-bc4c-ea267bd35b7b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:49:07.359092 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:07.358956    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65db1672-804f-4f6b-bc4c-ea267bd35b7b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "65db1672-804f-4f6b-bc4c-ea267bd35b7b" (UID: "65db1672-804f-4f6b-bc4c-ea267bd35b7b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:49:07.359144 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:07.359127    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65db1672-804f-4f6b-bc4c-ea267bd35b7b-home\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:49:07.359201 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:07.359162    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65db1672-804f-4f6b-bc4c-ea267bd35b7b-model-cache" (OuterVolumeSpecName: "model-cache") pod "65db1672-804f-4f6b-bc4c-ea267bd35b7b" (UID: "65db1672-804f-4f6b-bc4c-ea267bd35b7b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:49:07.360816 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:07.360788    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65db1672-804f-4f6b-bc4c-ea267bd35b7b-kube-api-access-5l2qx" (OuterVolumeSpecName: "kube-api-access-5l2qx") pod "65db1672-804f-4f6b-bc4c-ea267bd35b7b" (UID: "65db1672-804f-4f6b-bc4c-ea267bd35b7b"). InnerVolumeSpecName "kube-api-access-5l2qx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:49:07.360816 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:07.360802    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65db1672-804f-4f6b-bc4c-ea267bd35b7b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "65db1672-804f-4f6b-bc4c-ea267bd35b7b" (UID: "65db1672-804f-4f6b-bc4c-ea267bd35b7b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 14:49:07.360948 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:07.360859    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65db1672-804f-4f6b-bc4c-ea267bd35b7b-dshm" (OuterVolumeSpecName: "dshm") pod "65db1672-804f-4f6b-bc4c-ea267bd35b7b" (UID: "65db1672-804f-4f6b-bc4c-ea267bd35b7b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:49:07.460552 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:07.460511    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65db1672-804f-4f6b-bc4c-ea267bd35b7b-tmp-dir\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:49:07.460552 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:07.460549    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65db1672-804f-4f6b-bc4c-ea267bd35b7b-model-cache\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:49:07.460552 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:07.460558    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65db1672-804f-4f6b-bc4c-ea267bd35b7b-tls-certs\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:49:07.460774 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:07.460567    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5l2qx\" (UniqueName: \"kubernetes.io/projected/65db1672-804f-4f6b-bc4c-ea267bd35b7b-kube-api-access-5l2qx\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:49:07.460774 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:07.460575    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65db1672-804f-4f6b-bc4c-ea267bd35b7b-dshm\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:49:07.815172 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:07.815137    2559 generic.go:356] "Generic (PLEG): container finished" podID="65db1672-804f-4f6b-bc4c-ea267bd35b7b" containerID="309bbec09ec837cb27e968626e065372785612e681583f8a5faf3143a0e36072" exitCode=0
Jul 29 14:49:07.815349 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:07.815180    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2"
Jul 29 14:49:07.815349 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:07.815177    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2" event={"ID":"65db1672-804f-4f6b-bc4c-ea267bd35b7b","Type":"ContainerDied","Data":"309bbec09ec837cb27e968626e065372785612e681583f8a5faf3143a0e36072"}
Jul 29 14:49:07.815349 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:07.815294    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2" event={"ID":"65db1672-804f-4f6b-bc4c-ea267bd35b7b","Type":"ContainerDied","Data":"880dc931ffef98db689989136c6e8126312745db3654d7c21c6999388131a897"}
Jul 29 14:49:07.815349 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:07.815314    2559 scope.go:117] "RemoveContainer" containerID="309bbec09ec837cb27e968626e065372785612e681583f8a5faf3143a0e36072"
Jul 29 14:49:07.824285 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:07.824268    2559 scope.go:117] "RemoveContainer" containerID="309bbec09ec837cb27e968626e065372785612e681583f8a5faf3143a0e36072"
Jul 29 14:49:07.824532 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:49:07.824513    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"309bbec09ec837cb27e968626e065372785612e681583f8a5faf3143a0e36072\": container with ID starting with 309bbec09ec837cb27e968626e065372785612e681583f8a5faf3143a0e36072 not found: ID does not exist" containerID="309bbec09ec837cb27e968626e065372785612e681583f8a5faf3143a0e36072"
Jul 29 14:49:07.824609 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:07.824550    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"309bbec09ec837cb27e968626e065372785612e681583f8a5faf3143a0e36072"} err="failed to get container status \"309bbec09ec837cb27e968626e065372785612e681583f8a5faf3143a0e36072\": rpc error: code = NotFound desc = could not find container \"309bbec09ec837cb27e968626e065372785612e681583f8a5faf3143a0e36072\": container with ID starting with 309bbec09ec837cb27e968626e065372785612e681583f8a5faf3143a0e36072 not found: ID does not exist"
Jul 29 14:49:07.839343 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:07.839323    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2"]
Jul 29 14:49:07.842619 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:07.842596    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-532c20/auth-test-model-532c20-kserve-6f8b59b9b7-dm4f2"]
Jul 29 14:49:08.637834 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:08.637800    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65db1672-804f-4f6b-bc4c-ea267bd35b7b" path="/var/lib/kubelet/pods/65db1672-804f-4f6b-bc4c-ea267bd35b7b/volumes"
Jul 29 14:49:08.638352 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:08.638233    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68150e83-ad19-42f6-aea3-5ce299c146de" path="/var/lib/kubelet/pods/68150e83-ad19-42f6-aea3-5ce299c146de/volumes"
Jul 29 14:49:13.274644 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:13.274605    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6"
Jul 29 14:49:13.274644 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:13.274653    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6"
Jul 29 14:49:13.286756 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:13.286731    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6"
Jul 29 14:49:13.851462 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:13.851436    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6"
Jul 29 14:49:33.646825 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:33.646790    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5"]
Jul 29 14:49:33.647411 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:33.647130    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68150e83-ad19-42f6-aea3-5ce299c146de" containerName="main"
Jul 29 14:49:33.647411 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:33.647143    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="68150e83-ad19-42f6-aea3-5ce299c146de" containerName="main"
Jul 29 14:49:33.647411 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:33.647169    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65db1672-804f-4f6b-bc4c-ea267bd35b7b" containerName="main"
Jul 29 14:49:33.647411 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:33.647174    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="65db1672-804f-4f6b-bc4c-ea267bd35b7b" containerName="main"
Jul 29 14:49:33.647411 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:33.647222    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="65db1672-804f-4f6b-bc4c-ea267bd35b7b" containerName="main"
Jul 29 14:49:33.647411 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:33.647231    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="68150e83-ad19-42f6-aea3-5ce299c146de" containerName="main"
Jul 29 14:49:33.650132 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:33.650111    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5"
Jul 29 14:49:33.653459 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:33.653432    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-532c20\"/\"kube-root-ca.crt\""
Jul 29 14:49:33.653589 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:33.653432    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-532c20\"/\"openshift-service-ca.crt\""
Jul 29 14:49:33.653589 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:33.653494    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-532c20\"/\"sub-test-model-532c20-kserve-self-signed-certs\""
Jul 29 14:49:33.654889 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:33.654866    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-532c20\"/\"default-dockercfg-cjwft\""
Jul 29 14:49:33.660201 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:33.660180    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5"]
Jul 29 14:49:33.686401 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:33.686369    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:49:33.699610 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:33.699587    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52a74b31-99b4-4c0e-a111-103388200d1a-tmp-dir\") pod \"sub-test-model-532c20-kserve-845c956d5-j8zf5\" (UID: \"52a74b31-99b4-4c0e-a111-103388200d1a\") " pod="ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5"
Jul 29 14:49:33.699685 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:33.699618    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52a74b31-99b4-4c0e-a111-103388200d1a-tls-certs\") pod \"sub-test-model-532c20-kserve-845c956d5-j8zf5\" (UID: \"52a74b31-99b4-4c0e-a111-103388200d1a\") " pod="ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5"
Jul 29 14:49:33.699685 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:33.699636    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svf9r\" (UniqueName: \"kubernetes.io/projected/52a74b31-99b4-4c0e-a111-103388200d1a-kube-api-access-svf9r\") pod \"sub-test-model-532c20-kserve-845c956d5-j8zf5\" (UID: \"52a74b31-99b4-4c0e-a111-103388200d1a\") " pod="ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5"
Jul 29 14:49:33.699685 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:33.699665    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52a74b31-99b4-4c0e-a111-103388200d1a-home\") pod \"sub-test-model-532c20-kserve-845c956d5-j8zf5\" (UID: \"52a74b31-99b4-4c0e-a111-103388200d1a\") " pod="ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5"
Jul 29 14:49:33.699836 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:33.699707    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52a74b31-99b4-4c0e-a111-103388200d1a-dshm\") pod \"sub-test-model-532c20-kserve-845c956d5-j8zf5\" (UID: \"52a74b31-99b4-4c0e-a111-103388200d1a\") " pod="ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5"
Jul 29 14:49:33.699836 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:33.699728    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52a74b31-99b4-4c0e-a111-103388200d1a-model-cache\") pod \"sub-test-model-532c20-kserve-845c956d5-j8zf5\" (UID: \"52a74b31-99b4-4c0e-a111-103388200d1a\") " pod="ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5"
Jul 29 14:49:33.800967 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:33.800934    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52a74b31-99b4-4c0e-a111-103388200d1a-tmp-dir\") pod \"sub-test-model-532c20-kserve-845c956d5-j8zf5\" (UID: \"52a74b31-99b4-4c0e-a111-103388200d1a\") " pod="ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5"
Jul 29 14:49:33.800967 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:33.800972    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52a74b31-99b4-4c0e-a111-103388200d1a-tls-certs\") pod \"sub-test-model-532c20-kserve-845c956d5-j8zf5\" (UID: \"52a74b31-99b4-4c0e-a111-103388200d1a\") " pod="ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5"
Jul 29 14:49:33.801218 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:33.800989    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svf9r\" (UniqueName: \"kubernetes.io/projected/52a74b31-99b4-4c0e-a111-103388200d1a-kube-api-access-svf9r\") pod \"sub-test-model-532c20-kserve-845c956d5-j8zf5\" (UID: \"52a74b31-99b4-4c0e-a111-103388200d1a\") " pod="ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5"
Jul 29 14:49:33.801218 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:33.801016    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52a74b31-99b4-4c0e-a111-103388200d1a-home\") pod \"sub-test-model-532c20-kserve-845c956d5-j8zf5\" (UID: \"52a74b31-99b4-4c0e-a111-103388200d1a\") " pod="ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5"
Jul 29 14:49:33.801218 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:33.801036    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52a74b31-99b4-4c0e-a111-103388200d1a-dshm\") pod \"sub-test-model-532c20-kserve-845c956d5-j8zf5\" (UID: \"52a74b31-99b4-4c0e-a111-103388200d1a\") " pod="ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5"
Jul 29 14:49:33.801218 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:33.801074    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52a74b31-99b4-4c0e-a111-103388200d1a-model-cache\") pod \"sub-test-model-532c20-kserve-845c956d5-j8zf5\" (UID: \"52a74b31-99b4-4c0e-a111-103388200d1a\") " pod="ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5"
Jul 29 14:49:33.801447 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:33.801422    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52a74b31-99b4-4c0e-a111-103388200d1a-tmp-dir\") pod \"sub-test-model-532c20-kserve-845c956d5-j8zf5\" (UID: \"52a74b31-99b4-4c0e-a111-103388200d1a\") " pod="ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5"
Jul 29 14:49:33.801508 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:33.801473    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52a74b31-99b4-4c0e-a111-103388200d1a-home\") pod \"sub-test-model-532c20-kserve-845c956d5-j8zf5\" (UID: \"52a74b31-99b4-4c0e-a111-103388200d1a\") " pod="ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5"
Jul 29 14:49:33.801561 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:33.801515    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52a74b31-99b4-4c0e-a111-103388200d1a-model-cache\") pod \"sub-test-model-532c20-kserve-845c956d5-j8zf5\" (UID: \"52a74b31-99b4-4c0e-a111-103388200d1a\") " pod="ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5"
Jul 29 14:49:33.803358 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:33.803335    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52a74b31-99b4-4c0e-a111-103388200d1a-dshm\") pod \"sub-test-model-532c20-kserve-845c956d5-j8zf5\" (UID: \"52a74b31-99b4-4c0e-a111-103388200d1a\") " pod="ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5"
Jul 29 14:49:33.803507 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:33.803490    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52a74b31-99b4-4c0e-a111-103388200d1a-tls-certs\") pod \"sub-test-model-532c20-kserve-845c956d5-j8zf5\" (UID: \"52a74b31-99b4-4c0e-a111-103388200d1a\") " pod="ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5"
Jul 29 14:49:33.809089 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:33.809052    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svf9r\" (UniqueName: \"kubernetes.io/projected/52a74b31-99b4-4c0e-a111-103388200d1a-kube-api-access-svf9r\") pod \"sub-test-model-532c20-kserve-845c956d5-j8zf5\" (UID: \"52a74b31-99b4-4c0e-a111-103388200d1a\") " pod="ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5"
Jul 29 14:49:33.960024 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:33.959932    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5"
Jul 29 14:49:34.092130 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:34.092097    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5"]
Jul 29 14:49:34.095342 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:49:34.095313    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod52a74b31_99b4_4c0e_a111_103388200d1a.slice/crio-1f8ad9f3f5f0689fd57b2d1946d01b3626247fac5c05d927666efa2b64a06f64 WatchSource:0}: Error finding container 1f8ad9f3f5f0689fd57b2d1946d01b3626247fac5c05d927666efa2b64a06f64: Status 404 returned error can't find the container with id 1f8ad9f3f5f0689fd57b2d1946d01b3626247fac5c05d927666efa2b64a06f64
Jul 29 14:49:34.928335 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:34.928298    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5" event={"ID":"52a74b31-99b4-4c0e-a111-103388200d1a","Type":"ContainerStarted","Data":"48831ef2c3bd7738139a1f97df4e0021632e8ccad9e73e26f123cdd55e5c8062"}
Jul 29 14:49:34.928335 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:34.928340    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5" event={"ID":"52a74b31-99b4-4c0e-a111-103388200d1a","Type":"ContainerStarted","Data":"1f8ad9f3f5f0689fd57b2d1946d01b3626247fac5c05d927666efa2b64a06f64"}
Jul 29 14:49:34.946619 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:34.946567    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5" podStartSLOduration=1.946551093 podStartE2EDuration="1.946551093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:49:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:49:34.945229781 +0000 UTC m=+2904.835901115" watchObservedRunningTime="2026-07-29 14:49:34.946551093 +0000 UTC m=+2904.837222427"
Jul 29 14:49:43.961080 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:43.961011    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5"
Jul 29 14:49:43.961759 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:43.961099    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5"
Jul 29 14:49:43.973881 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:43.973851    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5"
Jul 29 14:49:44.980317 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:49:44.980287    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5"
Jul 29 14:50:05.328320 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:05.328281    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:50:38.530922 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:38.530841    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6"]
Jul 29 14:50:38.554745 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:38.554717    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:50:39.456136 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:39.456092    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5"]
Jul 29 14:50:39.478177 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:39.478146    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:50:46.770040 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:46.769987    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k"]
Jul 29 14:50:46.774524 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:46.774494    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k"
Jul 29 14:50:46.777861 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:46.777832    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-61bc0a\"/\"rate-test-model-61bc0a-kserve-self-signed-certs\""
Jul 29 14:50:46.778634 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:46.778604    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k"]
Jul 29 14:50:46.780452 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:46.780429    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:50:46.835467 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:46.835443    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-model-cache\") pod \"rate-test-model-61bc0a-kserve-74f866fb59-nwg5k\" (UID: \"d040d6bf-9f96-4a6f-bcda-f541a3937d9a\") " pod="ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k"
Jul 29 14:50:46.835591 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:46.835495    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-home\") pod \"rate-test-model-61bc0a-kserve-74f866fb59-nwg5k\" (UID: \"d040d6bf-9f96-4a6f-bcda-f541a3937d9a\") " pod="ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k"
Jul 29 14:50:46.835591 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:46.835517    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdtnw\" (UniqueName: \"kubernetes.io/projected/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-kube-api-access-gdtnw\") pod \"rate-test-model-61bc0a-kserve-74f866fb59-nwg5k\" (UID: \"d040d6bf-9f96-4a6f-bcda-f541a3937d9a\") " pod="ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k"
Jul 29 14:50:46.835591 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:46.835577    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-dshm\") pod \"rate-test-model-61bc0a-kserve-74f866fb59-nwg5k\" (UID: \"d040d6bf-9f96-4a6f-bcda-f541a3937d9a\") " pod="ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k"
Jul 29 14:50:46.835769 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:46.835652    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-tls-certs\") pod \"rate-test-model-61bc0a-kserve-74f866fb59-nwg5k\" (UID: \"d040d6bf-9f96-4a6f-bcda-f541a3937d9a\") " pod="ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k"
Jul 29 14:50:46.835769 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:46.835685    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-tmp-dir\") pod \"rate-test-model-61bc0a-kserve-74f866fb59-nwg5k\" (UID: \"d040d6bf-9f96-4a6f-bcda-f541a3937d9a\") " pod="ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k"
Jul 29 14:50:46.936715 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:46.936681    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-home\") pod \"rate-test-model-61bc0a-kserve-74f866fb59-nwg5k\" (UID: \"d040d6bf-9f96-4a6f-bcda-f541a3937d9a\") " pod="ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k"
Jul 29 14:50:46.936715 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:46.936717    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdtnw\" (UniqueName: \"kubernetes.io/projected/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-kube-api-access-gdtnw\") pod \"rate-test-model-61bc0a-kserve-74f866fb59-nwg5k\" (UID: \"d040d6bf-9f96-4a6f-bcda-f541a3937d9a\") " pod="ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k"
Jul 29 14:50:46.936929 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:46.936757    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-dshm\") pod \"rate-test-model-61bc0a-kserve-74f866fb59-nwg5k\" (UID: \"d040d6bf-9f96-4a6f-bcda-f541a3937d9a\") " pod="ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k"
Jul 29 14:50:46.936929 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:46.936776    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-tls-certs\") pod \"rate-test-model-61bc0a-kserve-74f866fb59-nwg5k\" (UID: \"d040d6bf-9f96-4a6f-bcda-f541a3937d9a\") " pod="ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k"
Jul 29 14:50:46.936929 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:46.936790    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-tmp-dir\") pod \"rate-test-model-61bc0a-kserve-74f866fb59-nwg5k\" (UID: \"d040d6bf-9f96-4a6f-bcda-f541a3937d9a\") " pod="ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k"
Jul 29 14:50:46.936929 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:46.936817    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-model-cache\") pod \"rate-test-model-61bc0a-kserve-74f866fb59-nwg5k\" (UID: \"d040d6bf-9f96-4a6f-bcda-f541a3937d9a\") " pod="ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k"
Jul 29 14:50:46.937152 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:46.937132    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-home\") pod \"rate-test-model-61bc0a-kserve-74f866fb59-nwg5k\" (UID: \"d040d6bf-9f96-4a6f-bcda-f541a3937d9a\") " pod="ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k"
Jul 29 14:50:46.937260 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:46.937237    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-model-cache\") pod \"rate-test-model-61bc0a-kserve-74f866fb59-nwg5k\" (UID: \"d040d6bf-9f96-4a6f-bcda-f541a3937d9a\") " pod="ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k"
Jul 29 14:50:46.937304 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:46.937273    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-tmp-dir\") pod \"rate-test-model-61bc0a-kserve-74f866fb59-nwg5k\" (UID: \"d040d6bf-9f96-4a6f-bcda-f541a3937d9a\") " pod="ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k"
Jul 29 14:50:46.939026 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:46.939002    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-dshm\") pod \"rate-test-model-61bc0a-kserve-74f866fb59-nwg5k\" (UID: \"d040d6bf-9f96-4a6f-bcda-f541a3937d9a\") " pod="ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k"
Jul 29 14:50:46.939209 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:46.939194    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-tls-certs\") pod \"rate-test-model-61bc0a-kserve-74f866fb59-nwg5k\" (UID: \"d040d6bf-9f96-4a6f-bcda-f541a3937d9a\") " pod="ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k"
Jul 29 14:50:46.947210 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:46.947183    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdtnw\" (UniqueName: \"kubernetes.io/projected/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-kube-api-access-gdtnw\") pod \"rate-test-model-61bc0a-kserve-74f866fb59-nwg5k\" (UID: \"d040d6bf-9f96-4a6f-bcda-f541a3937d9a\") " pod="ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k"
Jul 29 14:50:47.084982 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:47.084893    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k"
Jul 29 14:50:47.219197 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:47.219140    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k"]
Jul 29 14:50:47.221636 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:50:47.221606    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd040d6bf_9f96_4a6f_bcda_f541a3937d9a.slice/crio-f85bba08e02e34c9a429045a31225bb0174122b3046a89f0464e1f1a0e2bf893 WatchSource:0}: Error finding container f85bba08e02e34c9a429045a31225bb0174122b3046a89f0464e1f1a0e2bf893: Status 404 returned error can't find the container with id f85bba08e02e34c9a429045a31225bb0174122b3046a89f0464e1f1a0e2bf893
Jul 29 14:50:47.239110 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:47.239052    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k" event={"ID":"d040d6bf-9f96-4a6f-bcda-f541a3937d9a","Type":"ContainerStarted","Data":"f85bba08e02e34c9a429045a31225bb0174122b3046a89f0464e1f1a0e2bf893"}
Jul 29 14:50:48.245135 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:48.245095    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k" event={"ID":"d040d6bf-9f96-4a6f-bcda-f541a3937d9a","Type":"ContainerStarted","Data":"3d7a262a5183982a87a0a367365956202595b3715a61219544a2c11c85521a05"}
Jul 29 14:50:48.263457 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:48.263408    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k" podStartSLOduration=2.263394638 podStartE2EDuration="2.263394638s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:50:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:50:48.262135869 +0000 UTC m=+2978.152807204" watchObservedRunningTime="2026-07-29 14:50:48.263394638 +0000 UTC m=+2978.154065999"
Jul 29 14:50:53.549299 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:53.549237    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6" podUID="13919ccb-c701-4721-8ff5-a715e9399413" containerName="main" containerID="cri-o://a1a05fa4ce4d14a730bdbe2b88977fcfadba4527fd58fbe052489007fd6fbf03" gracePeriod=45
Jul 29 14:50:53.800635 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:53.800575    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6"
Jul 29 14:50:53.898331 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:53.898298    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13919ccb-c701-4721-8ff5-a715e9399413-dshm\") pod \"13919ccb-c701-4721-8ff5-a715e9399413\" (UID: \"13919ccb-c701-4721-8ff5-a715e9399413\") "
Jul 29 14:50:53.898331 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:53.898334    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13919ccb-c701-4721-8ff5-a715e9399413-model-cache\") pod \"13919ccb-c701-4721-8ff5-a715e9399413\" (UID: \"13919ccb-c701-4721-8ff5-a715e9399413\") "
Jul 29 14:50:53.898553 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:53.898377    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13919ccb-c701-4721-8ff5-a715e9399413-home\") pod \"13919ccb-c701-4721-8ff5-a715e9399413\" (UID: \"13919ccb-c701-4721-8ff5-a715e9399413\") "
Jul 29 14:50:53.898553 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:53.898435    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64dls\" (UniqueName: \"kubernetes.io/projected/13919ccb-c701-4721-8ff5-a715e9399413-kube-api-access-64dls\") pod \"13919ccb-c701-4721-8ff5-a715e9399413\" (UID: \"13919ccb-c701-4721-8ff5-a715e9399413\") "
Jul 29 14:50:53.898553 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:53.898453    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13919ccb-c701-4721-8ff5-a715e9399413-tls-certs\") pod \"13919ccb-c701-4721-8ff5-a715e9399413\" (UID: \"13919ccb-c701-4721-8ff5-a715e9399413\") "
Jul 29 14:50:53.898553 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:53.898479    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13919ccb-c701-4721-8ff5-a715e9399413-tmp-dir\") pod \"13919ccb-c701-4721-8ff5-a715e9399413\" (UID: \"13919ccb-c701-4721-8ff5-a715e9399413\") "
Jul 29 14:50:53.898779 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:53.898604    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13919ccb-c701-4721-8ff5-a715e9399413-model-cache" (OuterVolumeSpecName: "model-cache") pod "13919ccb-c701-4721-8ff5-a715e9399413" (UID: "13919ccb-c701-4721-8ff5-a715e9399413"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:50:53.898779 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:53.898613    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13919ccb-c701-4721-8ff5-a715e9399413-home" (OuterVolumeSpecName: "home") pod "13919ccb-c701-4721-8ff5-a715e9399413" (UID: "13919ccb-c701-4721-8ff5-a715e9399413"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:50:53.898779 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:53.898720    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13919ccb-c701-4721-8ff5-a715e9399413-model-cache\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:50:53.898779 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:53.898735    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13919ccb-c701-4721-8ff5-a715e9399413-home\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:50:53.898962 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:53.898806    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13919ccb-c701-4721-8ff5-a715e9399413-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "13919ccb-c701-4721-8ff5-a715e9399413" (UID: "13919ccb-c701-4721-8ff5-a715e9399413"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:50:53.900527 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:53.900500    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13919ccb-c701-4721-8ff5-a715e9399413-dshm" (OuterVolumeSpecName: "dshm") pod "13919ccb-c701-4721-8ff5-a715e9399413" (UID: "13919ccb-c701-4721-8ff5-a715e9399413"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:50:53.900620 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:53.900531    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13919ccb-c701-4721-8ff5-a715e9399413-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "13919ccb-c701-4721-8ff5-a715e9399413" (UID: "13919ccb-c701-4721-8ff5-a715e9399413"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 14:50:53.900620 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:53.900559    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13919ccb-c701-4721-8ff5-a715e9399413-kube-api-access-64dls" (OuterVolumeSpecName: "kube-api-access-64dls") pod "13919ccb-c701-4721-8ff5-a715e9399413" (UID: "13919ccb-c701-4721-8ff5-a715e9399413"). InnerVolumeSpecName "kube-api-access-64dls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:50:53.999276 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:53.999241    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64dls\" (UniqueName: \"kubernetes.io/projected/13919ccb-c701-4721-8ff5-a715e9399413-kube-api-access-64dls\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:50:53.999276 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:53.999272    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13919ccb-c701-4721-8ff5-a715e9399413-tls-certs\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:50:53.999276 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:53.999283    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13919ccb-c701-4721-8ff5-a715e9399413-tmp-dir\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:50:53.999506 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:53.999291    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13919ccb-c701-4721-8ff5-a715e9399413-dshm\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:50:54.273529 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:54.273497    2559 generic.go:356] "Generic (PLEG): container finished" podID="13919ccb-c701-4721-8ff5-a715e9399413" containerID="a1a05fa4ce4d14a730bdbe2b88977fcfadba4527fd58fbe052489007fd6fbf03" exitCode=0
Jul 29 14:50:54.273696 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:54.273543    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6"
Jul 29 14:50:54.273696 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:54.273540    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6" event={"ID":"13919ccb-c701-4721-8ff5-a715e9399413","Type":"ContainerDied","Data":"a1a05fa4ce4d14a730bdbe2b88977fcfadba4527fd58fbe052489007fd6fbf03"}
Jul 29 14:50:54.273696 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:54.273656    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6" event={"ID":"13919ccb-c701-4721-8ff5-a715e9399413","Type":"ContainerDied","Data":"33a79dab0e54a5542644bda948f977368c68a6202d34eff177e1eb9b179d09f2"}
Jul 29 14:50:54.273696 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:54.273678    2559 scope.go:117] "RemoveContainer" containerID="a1a05fa4ce4d14a730bdbe2b88977fcfadba4527fd58fbe052489007fd6fbf03"
Jul 29 14:50:54.282596 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:54.282570    2559 scope.go:117] "RemoveContainer" containerID="a1a05fa4ce4d14a730bdbe2b88977fcfadba4527fd58fbe052489007fd6fbf03"
Jul 29 14:50:54.282860 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:50:54.282840    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1a05fa4ce4d14a730bdbe2b88977fcfadba4527fd58fbe052489007fd6fbf03\": container with ID starting with a1a05fa4ce4d14a730bdbe2b88977fcfadba4527fd58fbe052489007fd6fbf03 not found: ID does not exist" containerID="a1a05fa4ce4d14a730bdbe2b88977fcfadba4527fd58fbe052489007fd6fbf03"
Jul 29 14:50:54.282919 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:54.282868    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1a05fa4ce4d14a730bdbe2b88977fcfadba4527fd58fbe052489007fd6fbf03"} err="failed to get container status \"a1a05fa4ce4d14a730bdbe2b88977fcfadba4527fd58fbe052489007fd6fbf03\": rpc error: code = NotFound desc = could not find container \"a1a05fa4ce4d14a730bdbe2b88977fcfadba4527fd58fbe052489007fd6fbf03\": container with ID starting with a1a05fa4ce4d14a730bdbe2b88977fcfadba4527fd58fbe052489007fd6fbf03 not found: ID does not exist"
Jul 29 14:50:54.295662 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:54.295638    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6"]
Jul 29 14:50:54.299972 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:54.299952    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-61bc0a/sub-test-model-61bc0a-kserve-66758b5669-t84h6"]
Jul 29 14:50:54.474734 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:54.474675    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5" podUID="52a74b31-99b4-4c0e-a111-103388200d1a" containerName="main" containerID="cri-o://48831ef2c3bd7738139a1f97df4e0021632e8ccad9e73e26f123cdd55e5c8062" gracePeriod=45
Jul 29 14:50:54.637606 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:54.637573    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13919ccb-c701-4721-8ff5-a715e9399413" path="/var/lib/kubelet/pods/13919ccb-c701-4721-8ff5-a715e9399413/volumes"
Jul 29 14:50:54.724637 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:54.724612    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5"
Jul 29 14:50:54.806106 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:54.806050    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52a74b31-99b4-4c0e-a111-103388200d1a-tls-certs\") pod \"52a74b31-99b4-4c0e-a111-103388200d1a\" (UID: \"52a74b31-99b4-4c0e-a111-103388200d1a\") "
Jul 29 14:50:54.806106 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:54.806107    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52a74b31-99b4-4c0e-a111-103388200d1a-home\") pod \"52a74b31-99b4-4c0e-a111-103388200d1a\" (UID: \"52a74b31-99b4-4c0e-a111-103388200d1a\") "
Jul 29 14:50:54.806345 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:54.806143    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52a74b31-99b4-4c0e-a111-103388200d1a-dshm\") pod \"52a74b31-99b4-4c0e-a111-103388200d1a\" (UID: \"52a74b31-99b4-4c0e-a111-103388200d1a\") "
Jul 29 14:50:54.806345 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:54.806162    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52a74b31-99b4-4c0e-a111-103388200d1a-model-cache\") pod \"52a74b31-99b4-4c0e-a111-103388200d1a\" (UID: \"52a74b31-99b4-4c0e-a111-103388200d1a\") "
Jul 29 14:50:54.806345 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:54.806210    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-svf9r\" (UniqueName: \"kubernetes.io/projected/52a74b31-99b4-4c0e-a111-103388200d1a-kube-api-access-svf9r\") pod \"52a74b31-99b4-4c0e-a111-103388200d1a\" (UID: \"52a74b31-99b4-4c0e-a111-103388200d1a\") "
Jul 29 14:50:54.806345 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:54.806245    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52a74b31-99b4-4c0e-a111-103388200d1a-tmp-dir\") pod \"52a74b31-99b4-4c0e-a111-103388200d1a\" (UID: \"52a74b31-99b4-4c0e-a111-103388200d1a\") "
Jul 29 14:50:54.806546 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:54.806460    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52a74b31-99b4-4c0e-a111-103388200d1a-home" (OuterVolumeSpecName: "home") pod "52a74b31-99b4-4c0e-a111-103388200d1a" (UID: "52a74b31-99b4-4c0e-a111-103388200d1a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:50:54.806546 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:54.806454    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52a74b31-99b4-4c0e-a111-103388200d1a-model-cache" (OuterVolumeSpecName: "model-cache") pod "52a74b31-99b4-4c0e-a111-103388200d1a" (UID: "52a74b31-99b4-4c0e-a111-103388200d1a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:50:54.806640 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:54.806612    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52a74b31-99b4-4c0e-a111-103388200d1a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "52a74b31-99b4-4c0e-a111-103388200d1a" (UID: "52a74b31-99b4-4c0e-a111-103388200d1a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:50:54.808326 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:54.808300    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52a74b31-99b4-4c0e-a111-103388200d1a-dshm" (OuterVolumeSpecName: "dshm") pod "52a74b31-99b4-4c0e-a111-103388200d1a" (UID: "52a74b31-99b4-4c0e-a111-103388200d1a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:50:54.808449 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:54.808395    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52a74b31-99b4-4c0e-a111-103388200d1a-kube-api-access-svf9r" (OuterVolumeSpecName: "kube-api-access-svf9r") pod "52a74b31-99b4-4c0e-a111-103388200d1a" (UID: "52a74b31-99b4-4c0e-a111-103388200d1a"). InnerVolumeSpecName "kube-api-access-svf9r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:50:54.808449 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:54.808412    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52a74b31-99b4-4c0e-a111-103388200d1a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "52a74b31-99b4-4c0e-a111-103388200d1a" (UID: "52a74b31-99b4-4c0e-a111-103388200d1a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 14:50:54.907532 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:54.907499    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52a74b31-99b4-4c0e-a111-103388200d1a-tls-certs\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:50:54.907532 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:54.907522    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52a74b31-99b4-4c0e-a111-103388200d1a-home\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:50:54.907532 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:54.907530    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52a74b31-99b4-4c0e-a111-103388200d1a-dshm\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:50:54.907532 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:54.907538    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52a74b31-99b4-4c0e-a111-103388200d1a-model-cache\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:50:54.907776 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:54.907546    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svf9r\" (UniqueName: \"kubernetes.io/projected/52a74b31-99b4-4c0e-a111-103388200d1a-kube-api-access-svf9r\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:50:54.907776 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:54.907553    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52a74b31-99b4-4c0e-a111-103388200d1a-tmp-dir\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:50:55.279683 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:55.279645    2559 generic.go:356] "Generic (PLEG): container finished" podID="52a74b31-99b4-4c0e-a111-103388200d1a" containerID="48831ef2c3bd7738139a1f97df4e0021632e8ccad9e73e26f123cdd55e5c8062" exitCode=0
Jul 29 14:50:55.279879 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:55.279687    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5"
Jul 29 14:50:55.279879 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:55.279729    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5" event={"ID":"52a74b31-99b4-4c0e-a111-103388200d1a","Type":"ContainerDied","Data":"48831ef2c3bd7738139a1f97df4e0021632e8ccad9e73e26f123cdd55e5c8062"}
Jul 29 14:50:55.279879 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:55.279770    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5" event={"ID":"52a74b31-99b4-4c0e-a111-103388200d1a","Type":"ContainerDied","Data":"1f8ad9f3f5f0689fd57b2d1946d01b3626247fac5c05d927666efa2b64a06f64"}
Jul 29 14:50:55.279879 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:55.279791    2559 scope.go:117] "RemoveContainer" containerID="48831ef2c3bd7738139a1f97df4e0021632e8ccad9e73e26f123cdd55e5c8062"
Jul 29 14:50:55.288659 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:55.288642    2559 scope.go:117] "RemoveContainer" containerID="48831ef2c3bd7738139a1f97df4e0021632e8ccad9e73e26f123cdd55e5c8062"
Jul 29 14:50:55.288884 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:50:55.288867    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48831ef2c3bd7738139a1f97df4e0021632e8ccad9e73e26f123cdd55e5c8062\": container with ID starting with 48831ef2c3bd7738139a1f97df4e0021632e8ccad9e73e26f123cdd55e5c8062 not found: ID does not exist" containerID="48831ef2c3bd7738139a1f97df4e0021632e8ccad9e73e26f123cdd55e5c8062"
Jul 29 14:50:55.288927 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:55.288892    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48831ef2c3bd7738139a1f97df4e0021632e8ccad9e73e26f123cdd55e5c8062"} err="failed to get container status \"48831ef2c3bd7738139a1f97df4e0021632e8ccad9e73e26f123cdd55e5c8062\": rpc error: code = NotFound desc = could not find container \"48831ef2c3bd7738139a1f97df4e0021632e8ccad9e73e26f123cdd55e5c8062\": container with ID starting with 48831ef2c3bd7738139a1f97df4e0021632e8ccad9e73e26f123cdd55e5c8062 not found: ID does not exist"
Jul 29 14:50:55.301672 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:55.301648    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5"]
Jul 29 14:50:55.306008 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:55.305986    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-532c20/sub-test-model-532c20-kserve-845c956d5-j8zf5"]
Jul 29 14:50:56.637229 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:56.637199    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52a74b31-99b4-4c0e-a111-103388200d1a" path="/var/lib/kubelet/pods/52a74b31-99b4-4c0e-a111-103388200d1a/volumes"
Jul 29 14:50:57.085204 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:57.085168    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k"
Jul 29 14:50:57.085375 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:57.085216    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k"
Jul 29 14:50:57.097252 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:57.097220    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k"
Jul 29 14:50:57.301619 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:50:57.301587    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k"
Jul 29 14:51:09.948369 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:09.948336    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2"]
Jul 29 14:51:09.948785 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:09.948657    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13919ccb-c701-4721-8ff5-a715e9399413" containerName="main"
Jul 29 14:51:09.948785 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:09.948670    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="13919ccb-c701-4721-8ff5-a715e9399413" containerName="main"
Jul 29 14:51:09.948785 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:09.948692    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52a74b31-99b4-4c0e-a111-103388200d1a" containerName="main"
Jul 29 14:51:09.948785 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:09.948697    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="52a74b31-99b4-4c0e-a111-103388200d1a" containerName="main"
Jul 29 14:51:09.948785 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:09.948765    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="13919ccb-c701-4721-8ff5-a715e9399413" containerName="main"
Jul 29 14:51:09.948785 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:09.948776    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="52a74b31-99b4-4c0e-a111-103388200d1a" containerName="main"
Jul 29 14:51:09.952963 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:09.952948    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2"
Jul 29 14:51:09.955787 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:09.955767    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-532c20\"/\"openshift-service-ca.crt\""
Jul 29 14:51:09.957294 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:09.957273    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-532c20\"/\"default-dockercfg-cjwft\""
Jul 29 14:51:09.957390 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:09.957299    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-532c20\"/\"kube-root-ca.crt\""
Jul 29 14:51:09.957390 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:09.957271    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-532c20\"/\"rate-test-model-532c20-kserve-self-signed-certs\""
Jul 29 14:51:09.962306 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:09.962284    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2"]
Jul 29 14:51:09.976385 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:09.976352    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:51:10.044860 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:10.044826    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-tls-certs\") pod \"rate-test-model-532c20-kserve-858c55f75-9j5b2\" (UID: \"536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a\") " pod="ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2"
Jul 29 14:51:10.045002 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:10.044865    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-dshm\") pod \"rate-test-model-532c20-kserve-858c55f75-9j5b2\" (UID: \"536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a\") " pod="ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2"
Jul 29 14:51:10.045002 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:10.044921    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-home\") pod \"rate-test-model-532c20-kserve-858c55f75-9j5b2\" (UID: \"536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a\") " pod="ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2"
Jul 29 14:51:10.045002 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:10.044951    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-model-cache\") pod \"rate-test-model-532c20-kserve-858c55f75-9j5b2\" (UID: \"536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a\") " pod="ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2"
Jul 29 14:51:10.045138 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:10.045047    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwjnv\" (UniqueName: \"kubernetes.io/projected/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-kube-api-access-kwjnv\") pod \"rate-test-model-532c20-kserve-858c55f75-9j5b2\" (UID: \"536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a\") " pod="ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2"
Jul 29 14:51:10.045138 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:10.045113    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-tmp-dir\") pod \"rate-test-model-532c20-kserve-858c55f75-9j5b2\" (UID: \"536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a\") " pod="ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2"
Jul 29 14:51:10.145824 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:10.145786    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwjnv\" (UniqueName: \"kubernetes.io/projected/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-kube-api-access-kwjnv\") pod \"rate-test-model-532c20-kserve-858c55f75-9j5b2\" (UID: \"536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a\") " pod="ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2"
Jul 29 14:51:10.146020 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:10.145843    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-tmp-dir\") pod \"rate-test-model-532c20-kserve-858c55f75-9j5b2\" (UID: \"536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a\") " pod="ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2"
Jul 29 14:51:10.146020 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:10.145884    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-tls-certs\") pod \"rate-test-model-532c20-kserve-858c55f75-9j5b2\" (UID: \"536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a\") " pod="ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2"
Jul 29 14:51:10.146020 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:10.145908    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-dshm\") pod \"rate-test-model-532c20-kserve-858c55f75-9j5b2\" (UID: \"536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a\") " pod="ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2"
Jul 29 14:51:10.146020 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:10.145942    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-home\") pod \"rate-test-model-532c20-kserve-858c55f75-9j5b2\" (UID: \"536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a\") " pod="ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2"
Jul 29 14:51:10.146020 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:10.145962    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-model-cache\") pod \"rate-test-model-532c20-kserve-858c55f75-9j5b2\" (UID: \"536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a\") " pod="ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2"
Jul 29 14:51:10.146341 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:10.146319    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-tmp-dir\") pod \"rate-test-model-532c20-kserve-858c55f75-9j5b2\" (UID: \"536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a\") " pod="ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2"
Jul 29 14:51:10.146413 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:10.146397    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-home\") pod \"rate-test-model-532c20-kserve-858c55f75-9j5b2\" (UID: \"536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a\") " pod="ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2"
Jul 29 14:51:10.146449 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:10.146437    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-model-cache\") pod \"rate-test-model-532c20-kserve-858c55f75-9j5b2\" (UID: \"536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a\") " pod="ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2"
Jul 29 14:51:10.148197 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:10.148181    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-dshm\") pod \"rate-test-model-532c20-kserve-858c55f75-9j5b2\" (UID: \"536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a\") " pod="ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2"
Jul 29 14:51:10.148470 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:10.148448    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-tls-certs\") pod \"rate-test-model-532c20-kserve-858c55f75-9j5b2\" (UID: \"536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a\") " pod="ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2"
Jul 29 14:51:10.154732 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:10.154708    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwjnv\" (UniqueName: \"kubernetes.io/projected/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-kube-api-access-kwjnv\") pod \"rate-test-model-532c20-kserve-858c55f75-9j5b2\" (UID: \"536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a\") " pod="ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2"
Jul 29 14:51:10.262382 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:10.262351    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2"
Jul 29 14:51:10.393813 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:10.393786    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2"]
Jul 29 14:51:10.395404 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:51:10.395374    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod536ff6eb_9fbb_4a8a_bf96_bfbb64cfb40a.slice/crio-352ba735dd60729b0f8fa2dbd560a7212b5eee56fff894d9d17bbaca5ae4b3c7 WatchSource:0}: Error finding container 352ba735dd60729b0f8fa2dbd560a7212b5eee56fff894d9d17bbaca5ae4b3c7: Status 404 returned error can't find the container with id 352ba735dd60729b0f8fa2dbd560a7212b5eee56fff894d9d17bbaca5ae4b3c7
Jul 29 14:51:10.774731 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:10.774699    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdw8l_b93f6faa-c8bf-43e9-94e0-b3dcd97b7296/ovn-acl-logging/0.log"
Jul 29 14:51:10.782664 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:10.782644    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdw8l_b93f6faa-c8bf-43e9-94e0-b3dcd97b7296/ovn-acl-logging/0.log"
Jul 29 14:51:11.357319 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:11.357283    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2" event={"ID":"536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a","Type":"ContainerStarted","Data":"c0c46301108f902181a24dc9e693fa655527954973be74b66962ca312259b7e5"}
Jul 29 14:51:11.357319 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:11.357325    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2" event={"ID":"536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a","Type":"ContainerStarted","Data":"352ba735dd60729b0f8fa2dbd560a7212b5eee56fff894d9d17bbaca5ae4b3c7"}
Jul 29 14:51:11.375791 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:11.375738    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2" podStartSLOduration=2.375723111 podStartE2EDuration="2.375723111s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:51:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:51:11.375118702 +0000 UTC m=+3001.265790037" watchObservedRunningTime="2026-07-29 14:51:11.375723111 +0000 UTC m=+3001.266394445"
Jul 29 14:51:20.262798 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:20.262758    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2"
Jul 29 14:51:20.263302 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:20.262811    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2"
Jul 29 14:51:20.275247 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:20.275217    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2"
Jul 29 14:51:20.407923 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:51:20.407889    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2"
Jul 29 14:52:43.653105 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:52:43.653052    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k"]
Jul 29 14:52:43.723328 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:52:43.723287    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:52:44.772327 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:52:44.772295    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2"]
Jul 29 14:52:44.796730 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:52:44.796691    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:52:58.671333 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:52:58.671295    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k" podUID="d040d6bf-9f96-4a6f-bcda-f541a3937d9a" containerName="main" containerID="cri-o://3d7a262a5183982a87a0a367365956202595b3715a61219544a2c11c85521a05" gracePeriod=45
Jul 29 14:52:58.825733 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:52:58.825699    2559 generic.go:356] "Generic (PLEG): container finished" podID="d040d6bf-9f96-4a6f-bcda-f541a3937d9a" containerID="3d7a262a5183982a87a0a367365956202595b3715a61219544a2c11c85521a05" exitCode=0
Jul 29 14:52:58.825886 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:52:58.825772    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k" event={"ID":"d040d6bf-9f96-4a6f-bcda-f541a3937d9a","Type":"ContainerDied","Data":"3d7a262a5183982a87a0a367365956202595b3715a61219544a2c11c85521a05"}
Jul 29 14:52:58.928679 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:52:58.928620    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k"
Jul 29 14:52:58.997383 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:52:58.997352    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-dshm\") pod \"d040d6bf-9f96-4a6f-bcda-f541a3937d9a\" (UID: \"d040d6bf-9f96-4a6f-bcda-f541a3937d9a\") "
Jul 29 14:52:58.997542 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:52:58.997398    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-tmp-dir\") pod \"d040d6bf-9f96-4a6f-bcda-f541a3937d9a\" (UID: \"d040d6bf-9f96-4a6f-bcda-f541a3937d9a\") "
Jul 29 14:52:58.997542 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:52:58.997452    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-tls-certs\") pod \"d040d6bf-9f96-4a6f-bcda-f541a3937d9a\" (UID: \"d040d6bf-9f96-4a6f-bcda-f541a3937d9a\") "
Jul 29 14:52:58.997542 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:52:58.997494    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-home\") pod \"d040d6bf-9f96-4a6f-bcda-f541a3937d9a\" (UID: \"d040d6bf-9f96-4a6f-bcda-f541a3937d9a\") "
Jul 29 14:52:58.997542 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:52:58.997532    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdtnw\" (UniqueName: \"kubernetes.io/projected/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-kube-api-access-gdtnw\") pod \"d040d6bf-9f96-4a6f-bcda-f541a3937d9a\" (UID: \"d040d6bf-9f96-4a6f-bcda-f541a3937d9a\") "
Jul 29 14:52:58.997735 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:52:58.997582    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-model-cache\") pod \"d040d6bf-9f96-4a6f-bcda-f541a3937d9a\" (UID: \"d040d6bf-9f96-4a6f-bcda-f541a3937d9a\") "
Jul 29 14:52:58.997735 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:52:58.997685    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d040d6bf-9f96-4a6f-bcda-f541a3937d9a" (UID: "d040d6bf-9f96-4a6f-bcda-f541a3937d9a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:52:58.997879 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:52:58.997856    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-home" (OuterVolumeSpecName: "home") pod "d040d6bf-9f96-4a6f-bcda-f541a3937d9a" (UID: "d040d6bf-9f96-4a6f-bcda-f541a3937d9a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:52:58.997944 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:52:58.997875    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-model-cache" (OuterVolumeSpecName: "model-cache") pod "d040d6bf-9f96-4a6f-bcda-f541a3937d9a" (UID: "d040d6bf-9f96-4a6f-bcda-f541a3937d9a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:52:58.997944 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:52:58.997867    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-tmp-dir\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:52:58.999485 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:52:58.999454    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d040d6bf-9f96-4a6f-bcda-f541a3937d9a" (UID: "d040d6bf-9f96-4a6f-bcda-f541a3937d9a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 14:52:58.999615 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:52:58.999483    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-dshm" (OuterVolumeSpecName: "dshm") pod "d040d6bf-9f96-4a6f-bcda-f541a3937d9a" (UID: "d040d6bf-9f96-4a6f-bcda-f541a3937d9a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:52:58.999615 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:52:58.999555    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-kube-api-access-gdtnw" (OuterVolumeSpecName: "kube-api-access-gdtnw") pod "d040d6bf-9f96-4a6f-bcda-f541a3937d9a" (UID: "d040d6bf-9f96-4a6f-bcda-f541a3937d9a"). InnerVolumeSpecName "kube-api-access-gdtnw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:52:59.098450 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:52:59.098420    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-tls-certs\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:52:59.098450 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:52:59.098444    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-home\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:52:59.098450 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:52:59.098454    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdtnw\" (UniqueName: \"kubernetes.io/projected/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-kube-api-access-gdtnw\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:52:59.098668 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:52:59.098463    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-model-cache\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:52:59.098668 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:52:59.098471    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d040d6bf-9f96-4a6f-bcda-f541a3937d9a-dshm\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:52:59.790959 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:52:59.790915    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2" podUID="536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a" containerName="main" containerID="cri-o://c0c46301108f902181a24dc9e693fa655527954973be74b66962ca312259b7e5" gracePeriod=45
Jul 29 14:52:59.831451 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:52:59.831420    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k"
Jul 29 14:52:59.831451 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:52:59.831439    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k" event={"ID":"d040d6bf-9f96-4a6f-bcda-f541a3937d9a","Type":"ContainerDied","Data":"f85bba08e02e34c9a429045a31225bb0174122b3046a89f0464e1f1a0e2bf893"}
Jul 29 14:52:59.831610 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:52:59.831490    2559 scope.go:117] "RemoveContainer" containerID="3d7a262a5183982a87a0a367365956202595b3715a61219544a2c11c85521a05"
Jul 29 14:52:59.858512 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:52:59.858484    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k"]
Jul 29 14:52:59.862905 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:52:59.862880    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-61bc0a/rate-test-model-61bc0a-kserve-74f866fb59-nwg5k"]
Jul 29 14:53:00.039864 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:00.039838    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2"
Jul 29 14:53:00.107099 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:00.106995    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-tls-certs\") pod \"536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a\" (UID: \"536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a\") "
Jul 29 14:53:00.107099 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:00.107038    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-model-cache\") pod \"536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a\" (UID: \"536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a\") "
Jul 29 14:53:00.107319 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:00.107103    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwjnv\" (UniqueName: \"kubernetes.io/projected/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-kube-api-access-kwjnv\") pod \"536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a\" (UID: \"536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a\") "
Jul 29 14:53:00.107319 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:00.107144    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-dshm\") pod \"536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a\" (UID: \"536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a\") "
Jul 29 14:53:00.107319 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:00.107174    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-tmp-dir\") pod \"536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a\" (UID: \"536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a\") "
Jul 29 14:53:00.107319 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:00.107201    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-home\") pod \"536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a\" (UID: \"536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a\") "
Jul 29 14:53:00.107511 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:00.107371    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-model-cache" (OuterVolumeSpecName: "model-cache") pod "536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a" (UID: "536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:53:00.107511 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:00.107489    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a" (UID: "536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:53:00.107613 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:00.107513    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-home" (OuterVolumeSpecName: "home") pod "536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a" (UID: "536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:53:00.107613 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:00.107535    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-model-cache\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:53:00.109120 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:00.109098    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a" (UID: "536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 14:53:00.109251 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:00.109233    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-kube-api-access-kwjnv" (OuterVolumeSpecName: "kube-api-access-kwjnv") pod "536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a" (UID: "536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a"). InnerVolumeSpecName "kube-api-access-kwjnv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:53:00.109314 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:00.109238    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-dshm" (OuterVolumeSpecName: "dshm") pod "536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a" (UID: "536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:53:00.208886 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:00.208837    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-dshm\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:53:00.208886 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:00.208882    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-tmp-dir\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:53:00.208886 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:00.208893    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-home\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:53:00.208886 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:00.208902    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-tls-certs\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:53:00.209189 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:00.208910    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwjnv\" (UniqueName: \"kubernetes.io/projected/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a-kube-api-access-kwjnv\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:53:00.635959 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:00.635924    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d040d6bf-9f96-4a6f-bcda-f541a3937d9a" path="/var/lib/kubelet/pods/d040d6bf-9f96-4a6f-bcda-f541a3937d9a/volumes"
Jul 29 14:53:00.836469 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:00.836434    2559 generic.go:356] "Generic (PLEG): container finished" podID="536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a" containerID="c0c46301108f902181a24dc9e693fa655527954973be74b66962ca312259b7e5" exitCode=0
Jul 29 14:53:00.836932 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:00.836477    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2"
Jul 29 14:53:00.836932 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:00.836525    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2" event={"ID":"536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a","Type":"ContainerDied","Data":"c0c46301108f902181a24dc9e693fa655527954973be74b66962ca312259b7e5"}
Jul 29 14:53:00.836932 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:00.836565    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2" event={"ID":"536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a","Type":"ContainerDied","Data":"352ba735dd60729b0f8fa2dbd560a7212b5eee56fff894d9d17bbaca5ae4b3c7"}
Jul 29 14:53:00.836932 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:00.836586    2559 scope.go:117] "RemoveContainer" containerID="c0c46301108f902181a24dc9e693fa655527954973be74b66962ca312259b7e5"
Jul 29 14:53:00.844774 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:00.844755    2559 scope.go:117] "RemoveContainer" containerID="c0c46301108f902181a24dc9e693fa655527954973be74b66962ca312259b7e5"
Jul 29 14:53:00.845013 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:53:00.844993    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0c46301108f902181a24dc9e693fa655527954973be74b66962ca312259b7e5\": container with ID starting with c0c46301108f902181a24dc9e693fa655527954973be74b66962ca312259b7e5 not found: ID does not exist" containerID="c0c46301108f902181a24dc9e693fa655527954973be74b66962ca312259b7e5"
Jul 29 14:53:00.845085 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:00.845021    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0c46301108f902181a24dc9e693fa655527954973be74b66962ca312259b7e5"} err="failed to get container status \"c0c46301108f902181a24dc9e693fa655527954973be74b66962ca312259b7e5\": rpc error: code = NotFound desc = could not find container \"c0c46301108f902181a24dc9e693fa655527954973be74b66962ca312259b7e5\": container with ID starting with c0c46301108f902181a24dc9e693fa655527954973be74b66962ca312259b7e5 not found: ID does not exist"
Jul 29 14:53:00.854726 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:00.854699    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2"]
Jul 29 14:53:00.858808 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:00.858788    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-532c20/rate-test-model-532c20-kserve-858c55f75-9j5b2"]
Jul 29 14:53:02.636070 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:02.636024    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a" path="/var/lib/kubelet/pods/536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a/volumes"
Jul 29 14:53:04.222978 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.222938    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:53:04.546787 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.546740    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"]
Jul 29 14:53:04.547154 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.547137    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a" containerName="main"
Jul 29 14:53:04.547236 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.547156    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a" containerName="main"
Jul 29 14:53:04.547236 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.547170    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d040d6bf-9f96-4a6f-bcda-f541a3937d9a" containerName="main"
Jul 29 14:53:04.547236 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.547175    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d040d6bf-9f96-4a6f-bcda-f541a3937d9a" containerName="main"
Jul 29 14:53:04.547236 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.547222    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d040d6bf-9f96-4a6f-bcda-f541a3937d9a" containerName="main"
Jul 29 14:53:04.547236 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.547236    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="536ff6eb-9fbb-4a8a-bf96-bfbb64cfb40a" containerName="main"
Jul 29 14:53:04.551475 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.551456    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:04.554597 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.554577    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-19afc1-openshift-default-dockercfg-c77k7\""
Jul 29 14:53:04.562354 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.562333    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"]
Jul 29 14:53:04.579022 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.578999    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:53:04.648668 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.648631    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18d0724a-eb28-4815-a8a4-48d958dda388-istio-data\") pod \"e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") " pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:04.648857 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.648724    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz595\" (UniqueName: \"kubernetes.io/projected/18d0724a-eb28-4815-a8a4-48d958dda388-kube-api-access-wz595\") pod \"e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") " pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:04.648857 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.648756    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18d0724a-eb28-4815-a8a4-48d958dda388-credential-socket\") pod \"e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") " pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:04.648857 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.648780    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18d0724a-eb28-4815-a8a4-48d958dda388-workload-certs\") pod \"e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") " pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:04.648857 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.648816    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18d0724a-eb28-4815-a8a4-48d958dda388-istiod-ca-cert\") pod \"e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") " pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:04.649108 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.648906    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18d0724a-eb28-4815-a8a4-48d958dda388-workload-socket\") pod \"e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") " pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:04.649108 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.648941    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18d0724a-eb28-4815-a8a4-48d958dda388-istio-envoy\") pod \"e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") " pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:04.649108 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.648996    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18d0724a-eb28-4815-a8a4-48d958dda388-istio-podinfo\") pod \"e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") " pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:04.649108 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.649019    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18d0724a-eb28-4815-a8a4-48d958dda388-istio-token\") pod \"e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") " pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:04.749546 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.749492    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18d0724a-eb28-4815-a8a4-48d958dda388-istio-envoy\") pod \"e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") " pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:04.749727 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.749667    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18d0724a-eb28-4815-a8a4-48d958dda388-istio-podinfo\") pod \"e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") " pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:04.749727 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.749709    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18d0724a-eb28-4815-a8a4-48d958dda388-istio-token\") pod \"e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") " pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:04.749884 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.749756    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18d0724a-eb28-4815-a8a4-48d958dda388-istio-data\") pod \"e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") " pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:04.749884 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.749818    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz595\" (UniqueName: \"kubernetes.io/projected/18d0724a-eb28-4815-a8a4-48d958dda388-kube-api-access-wz595\") pod \"e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") " pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:04.749884 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.749847    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18d0724a-eb28-4815-a8a4-48d958dda388-credential-socket\") pod \"e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") " pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:04.749884 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.749872    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18d0724a-eb28-4815-a8a4-48d958dda388-workload-certs\") pod \"e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") " pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:04.750163 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.749897    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18d0724a-eb28-4815-a8a4-48d958dda388-istiod-ca-cert\") pod \"e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") " pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:04.750163 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.749988    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18d0724a-eb28-4815-a8a4-48d958dda388-workload-socket\") pod \"e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") " pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:04.750295 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.750272    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18d0724a-eb28-4815-a8a4-48d958dda388-istio-data\") pod \"e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") " pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:04.750352 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.750291    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18d0724a-eb28-4815-a8a4-48d958dda388-credential-socket\") pod \"e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") " pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:04.750424 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.750402    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18d0724a-eb28-4815-a8a4-48d958dda388-workload-socket\") pod \"e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") " pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:04.750482 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.750462    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18d0724a-eb28-4815-a8a4-48d958dda388-workload-certs\") pod \"e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") " pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:04.750720 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.750700    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18d0724a-eb28-4815-a8a4-48d958dda388-istiod-ca-cert\") pod \"e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") " pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:04.751840 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.751820    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18d0724a-eb28-4815-a8a4-48d958dda388-istio-envoy\") pod \"e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") " pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:04.752237 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.752220    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18d0724a-eb28-4815-a8a4-48d958dda388-istio-podinfo\") pod \"e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") " pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:04.758166 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.758139    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18d0724a-eb28-4815-a8a4-48d958dda388-istio-token\") pod \"e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") " pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:04.758484 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.758466    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz595\" (UniqueName: \"kubernetes.io/projected/18d0724a-eb28-4815-a8a4-48d958dda388-kube-api-access-wz595\") pod \"e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") " pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:04.851015 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.850905    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:53:04.861370 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:04.861342    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:05.003870 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:05.003843    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"]
Jul 29 14:53:05.005675 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:53:05.005644    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18d0724a_eb28_4815_a8a4_48d958dda388.slice/crio-859b3cf8d1dd10c7730fd5b00ad1e770ec58711a3823f12da11fed0112876095 WatchSource:0}: Error finding container 859b3cf8d1dd10c7730fd5b00ad1e770ec58711a3823f12da11fed0112876095: Status 404 returned error can't find the container with id 859b3cf8d1dd10c7730fd5b00ad1e770ec58711a3823f12da11fed0112876095
Jul 29 14:53:05.007972 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:05.007935    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:53:05.008036 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:05.008015    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:53:05.008095 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:05.008080    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:53:05.861588 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:05.861550    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv" event={"ID":"18d0724a-eb28-4815-a8a4-48d958dda388","Type":"ContainerStarted","Data":"2d530ef2f60ddcb6a940ecd2b56aa1009b543bcb9bb0921a8e8b3a3244a206b1"}
Jul 29 14:53:05.861979 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:05.861610    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv" event={"ID":"18d0724a-eb28-4815-a8a4-48d958dda388","Type":"ContainerStarted","Data":"859b3cf8d1dd10c7730fd5b00ad1e770ec58711a3823f12da11fed0112876095"}
Jul 29 14:53:05.885748 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:05.885697    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv" podStartSLOduration=1.885683848 podStartE2EDuration="1.885683848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:53:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:53:05.884238979 +0000 UTC m=+3115.774910326" watchObservedRunningTime="2026-07-29 14:53:05.885683848 +0000 UTC m=+3115.776355184"
Jul 29 14:53:06.861469 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:06.861428    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:06.867265 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:06.867240    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:07.870514 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:07.870483    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:07.871377 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:07.871355    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:07.974521 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:07.974480    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:53:08.131939 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:08.131847    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:53:08.257027 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:08.256984    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:53:08.445427 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:08.445345    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:53:08.933770 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:08.933734    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:53:09.022946 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:09.022911    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:53:23.912219 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:23.912172    2559 patch_prober.go:28] interesting pod/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 29 14:53:23.912665 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:23.912254    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv" podUID="18d0724a-eb28-4815-a8a4-48d958dda388" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 29 14:53:25.951028 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:25.950991    2559 generic.go:356] "Generic (PLEG): container finished" podID="18d0724a-eb28-4815-a8a4-48d958dda388" containerID="2d530ef2f60ddcb6a940ecd2b56aa1009b543bcb9bb0921a8e8b3a3244a206b1" exitCode=0
Jul 29 14:53:25.951457 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:25.951076    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv" event={"ID":"18d0724a-eb28-4815-a8a4-48d958dda388","Type":"ContainerDied","Data":"2d530ef2f60ddcb6a940ecd2b56aa1009b543bcb9bb0921a8e8b3a3244a206b1"}
Jul 29 14:53:25.951457 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:25.951428    2559 scope.go:117] "RemoveContainer" containerID="2d530ef2f60ddcb6a940ecd2b56aa1009b543bcb9bb0921a8e8b3a3244a206b1"
Jul 29 14:53:25.952448 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:25.952422    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:53:25.952503 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:25.952491    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:53:25.952554 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:25.952530    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:53:26.956936 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:26.956902    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv" event={"ID":"18d0724a-eb28-4815-a8a4-48d958dda388","Type":"ContainerStarted","Data":"5f11f2db8ea90f209025188ab98a7177fd30d46511ebcd688a54a463bf84d74b"}
Jul 29 14:53:27.861913 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:27.861876    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:27.862133 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:27.862002    2559 patch_prober.go:28] interesting pod/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Jul 29 14:53:27.862133 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:27.862050    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv" podUID="18d0724a-eb28-4815-a8a4-48d958dda388" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Jul 29 14:53:28.862330 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:28.862291    2559 patch_prober.go:28] interesting pod/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Jul 29 14:53:28.863342 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:28.862886    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv" podUID="18d0724a-eb28-4815-a8a4-48d958dda388" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Jul 29 14:53:29.862188 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:29.862147    2559 patch_prober.go:28] interesting pod/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Jul 29 14:53:29.862405 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:29.862234    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv" podUID="18d0724a-eb28-4815-a8a4-48d958dda388" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Jul 29 14:53:30.862296 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:30.862249    2559 patch_prober.go:28] interesting pod/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Jul 29 14:53:30.862512 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:30.862335    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv" podUID="18d0724a-eb28-4815-a8a4-48d958dda388" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Jul 29 14:53:31.865572 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:31.865541    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:31.865996 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:31.865877    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:31.866496 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:31.866479    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:53:33.014008 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:33.013926    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:53:33.705657 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:33.705620    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:53:33.990226 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:33.990199    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"]
Jul 29 14:53:33.998074 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:33.998013    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:53:34.001226 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.001204    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-726aca-openshift-default-dockercfg-q4q8t\""
Jul 29 14:53:34.008785 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.008757    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"]
Jul 29 14:53:34.013759 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.013733    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7eb92074-59a2-4c68-8722-51c2e97a2d8e-credential-socket\") pod \"e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") " pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:53:34.013888 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.013766    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7eb92074-59a2-4c68-8722-51c2e97a2d8e-workload-certs\") pod \"e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") " pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:53:34.013888 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.013806    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7eb92074-59a2-4c68-8722-51c2e97a2d8e-istio-data\") pod \"e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") " pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:53:34.013888 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.013829    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7eb92074-59a2-4c68-8722-51c2e97a2d8e-workload-socket\") pod \"e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") " pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:53:34.013888 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.013845    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsz2n\" (UniqueName: \"kubernetes.io/projected/7eb92074-59a2-4c68-8722-51c2e97a2d8e-kube-api-access-qsz2n\") pod \"e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") " pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:53:34.014424 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.013940    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7eb92074-59a2-4c68-8722-51c2e97a2d8e-istio-podinfo\") pod \"e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") " pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:53:34.014424 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.013969    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7eb92074-59a2-4c68-8722-51c2e97a2d8e-istio-token\") pod \"e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") " pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:53:34.014424 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.013987    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7eb92074-59a2-4c68-8722-51c2e97a2d8e-istiod-ca-cert\") pod \"e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") " pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:53:34.014424 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.014019    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7eb92074-59a2-4c68-8722-51c2e97a2d8e-istio-envoy\") pod \"e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") " pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:53:34.029219 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.029189    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:53:34.115283 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.115239    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7eb92074-59a2-4c68-8722-51c2e97a2d8e-credential-socket\") pod \"e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") " pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:53:34.115494 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.115308    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7eb92074-59a2-4c68-8722-51c2e97a2d8e-workload-certs\") pod \"e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") " pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:53:34.115494 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.115380    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7eb92074-59a2-4c68-8722-51c2e97a2d8e-istio-data\") pod \"e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") " pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:53:34.115494 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.115422    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7eb92074-59a2-4c68-8722-51c2e97a2d8e-workload-socket\") pod \"e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") " pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:53:34.115494 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.115447    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsz2n\" (UniqueName: \"kubernetes.io/projected/7eb92074-59a2-4c68-8722-51c2e97a2d8e-kube-api-access-qsz2n\") pod \"e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") " pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:53:34.115756 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.115509    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7eb92074-59a2-4c68-8722-51c2e97a2d8e-istio-podinfo\") pod \"e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") " pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:53:34.115756 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.115534    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7eb92074-59a2-4c68-8722-51c2e97a2d8e-istio-token\") pod \"e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") " pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:53:34.115756 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.115565    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7eb92074-59a2-4c68-8722-51c2e97a2d8e-istiod-ca-cert\") pod \"e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") " pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:53:34.115756 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.115605    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7eb92074-59a2-4c68-8722-51c2e97a2d8e-istio-envoy\") pod \"e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") " pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:53:34.116027 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.115756    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7eb92074-59a2-4c68-8722-51c2e97a2d8e-credential-socket\") pod \"e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") " pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:53:34.116305 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.116280    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7eb92074-59a2-4c68-8722-51c2e97a2d8e-workload-certs\") pod \"e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") " pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:53:34.116477 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.116407    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7eb92074-59a2-4c68-8722-51c2e97a2d8e-istio-data\") pod \"e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") " pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:53:34.116671 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.116647    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7eb92074-59a2-4c68-8722-51c2e97a2d8e-workload-socket\") pod \"e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") " pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:53:34.117185 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.117162    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7eb92074-59a2-4c68-8722-51c2e97a2d8e-istiod-ca-cert\") pod \"e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") " pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:53:34.118660 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.118640    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7eb92074-59a2-4c68-8722-51c2e97a2d8e-istio-envoy\") pod \"e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") " pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:53:34.118827 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.118808    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7eb92074-59a2-4c68-8722-51c2e97a2d8e-istio-podinfo\") pod \"e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") " pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:53:34.126771 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.126749    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7eb92074-59a2-4c68-8722-51c2e97a2d8e-istio-token\") pod \"e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") " pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:53:34.127346 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.127325    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsz2n\" (UniqueName: \"kubernetes.io/projected/7eb92074-59a2-4c68-8722-51c2e97a2d8e-kube-api-access-qsz2n\") pod \"e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") " pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:53:34.253258 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.253166    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:53:34.309211 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.309176    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:53:34.444401 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.444369    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"]
Jul 29 14:53:34.448619 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:53:34.448590    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eb92074_59a2_4c68_8722_51c2e97a2d8e.slice/crio-723696965e7f7545689781d69fb043c8ddeb035de01bb8752611d9dd5c944f4a WatchSource:0}: Error finding container 723696965e7f7545689781d69fb043c8ddeb035de01bb8752611d9dd5c944f4a: Status 404 returned error can't find the container with id 723696965e7f7545689781d69fb043c8ddeb035de01bb8752611d9dd5c944f4a
Jul 29 14:53:34.450818 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.450787    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:53:34.450897 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.450851    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:53:34.450897 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:34.450880    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:53:35.003173 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:35.003136    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq" event={"ID":"7eb92074-59a2-4c68-8722-51c2e97a2d8e","Type":"ContainerStarted","Data":"835c811f0c0a081f405b0c1cb19391e7b59b955eb6ae282d71e58a0cbb82faaf"}
Jul 29 14:53:35.003340 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:35.003197    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq" event={"ID":"7eb92074-59a2-4c68-8722-51c2e97a2d8e","Type":"ContainerStarted","Data":"723696965e7f7545689781d69fb043c8ddeb035de01bb8752611d9dd5c944f4a"}
Jul 29 14:53:35.024640 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:35.024593    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq" podStartSLOduration=2.02457836 podStartE2EDuration="2.02457836s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:53:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:53:35.023042523 +0000 UTC m=+3144.913713857" watchObservedRunningTime="2026-07-29 14:53:35.02457836 +0000 UTC m=+3144.915249695"
Jul 29 14:53:35.309357 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:35.309269    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:53:35.314695 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:35.314662    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:53:36.007503 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:36.007474    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:53:36.008404 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:36.008383    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:53:36.980600 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:36.980548    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:53:37.077074 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:37.077028    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:53:37.259153 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:37.259114    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:53:37.435219 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:37.435188    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:53:37.791308 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:37.791277    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:53:37.891132 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:37.891099    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:53:55.098980 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:55.098949    2559 generic.go:356] "Generic (PLEG): container finished" podID="7eb92074-59a2-4c68-8722-51c2e97a2d8e" containerID="835c811f0c0a081f405b0c1cb19391e7b59b955eb6ae282d71e58a0cbb82faaf" exitCode=0
Jul 29 14:53:55.099483 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:55.098991    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq" event={"ID":"7eb92074-59a2-4c68-8722-51c2e97a2d8e","Type":"ContainerDied","Data":"835c811f0c0a081f405b0c1cb19391e7b59b955eb6ae282d71e58a0cbb82faaf"}
Jul 29 14:53:55.099554 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:55.099496    2559 scope.go:117] "RemoveContainer" containerID="835c811f0c0a081f405b0c1cb19391e7b59b955eb6ae282d71e58a0cbb82faaf"
Jul 29 14:53:55.100553 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:55.100506    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:53:55.100645 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:55.100583    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:53:55.100645 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:55.100617    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 29 14:53:56.105331 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:56.105298    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq" event={"ID":"7eb92074-59a2-4c68-8722-51c2e97a2d8e","Type":"ContainerStarted","Data":"6b5909b289f67880e6be7616889ebefc20deb1c870c89e9dea096969fd734bdf"}
Jul 29 14:53:56.309453 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:56.309416    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:53:56.309693 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:56.309671    2559 patch_prober.go:28] interesting pod/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 29 14:53:56.309740 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:56.309717    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq" podUID="7eb92074-59a2-4c68-8722-51c2e97a2d8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 29 14:53:57.310554 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:57.310515    2559 patch_prober.go:28] interesting pod/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 29 14:53:57.310974 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:57.310582    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq" podUID="7eb92074-59a2-4c68-8722-51c2e97a2d8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 29 14:53:58.309630 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:58.309589    2559 patch_prober.go:28] interesting pod/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 29 14:53:58.309823 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:58.309652    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq" podUID="7eb92074-59a2-4c68-8722-51c2e97a2d8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 29 14:53:59.310530 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:59.310489    2559 patch_prober.go:28] interesting pod/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 29 14:53:59.310951 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:53:59.310547    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq" podUID="7eb92074-59a2-4c68-8722-51c2e97a2d8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 29 14:54:00.310379 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:00.310339    2559 patch_prober.go:28] interesting pod/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 29 14:54:00.310560 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:00.310398    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq" podUID="7eb92074-59a2-4c68-8722-51c2e97a2d8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 29 14:54:01.314147 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:01.314114    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:54:01.314609 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:01.314395    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:54:01.315029 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:01.315014    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:54:02.000561 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:02.000521    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:54:11.447765 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:11.447730    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj"]
Jul 29 14:54:11.450114 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:11.450095    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj"
Jul 29 14:54:11.454585 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:11.454564    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-19afc1\"/\"ipp-route-19afc1-kserve-self-signed-certs\""
Jul 29 14:54:11.454585 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:11.454563    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-19afc1\"/\"kube-root-ca.crt\""
Jul 29 14:54:11.454760 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:11.454614    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-19afc1\"/\"default-dockercfg-6q4hh\""
Jul 29 14:54:11.454760 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:11.454631    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-19afc1\"/\"openshift-service-ca.crt\""
Jul 29 14:54:11.461604 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:11.461579    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj"]
Jul 29 14:54:11.541373 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:11.541338    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:54:11.553548 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:11.553521    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-home\") pod \"ipp-route-19afc1-kserve-7d7b8f5795-q42gj\" (UID: \"29c7c69b-9cbf-406e-ae51-744f2b27c5fa\") " pod="ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj"
Jul 29 14:54:11.553662 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:11.553642    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-tls-certs\") pod \"ipp-route-19afc1-kserve-7d7b8f5795-q42gj\" (UID: \"29c7c69b-9cbf-406e-ae51-744f2b27c5fa\") " pod="ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj"
Jul 29 14:54:11.553713 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:11.553689    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-model-cache\") pod \"ipp-route-19afc1-kserve-7d7b8f5795-q42gj\" (UID: \"29c7c69b-9cbf-406e-ae51-744f2b27c5fa\") " pod="ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj"
Jul 29 14:54:11.553753 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:11.553722    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xppb\" (UniqueName: \"kubernetes.io/projected/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-kube-api-access-7xppb\") pod \"ipp-route-19afc1-kserve-7d7b8f5795-q42gj\" (UID: \"29c7c69b-9cbf-406e-ae51-744f2b27c5fa\") " pod="ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj"
Jul 29 14:54:11.553800 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:11.553756    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-tmp-dir\") pod \"ipp-route-19afc1-kserve-7d7b8f5795-q42gj\" (UID: \"29c7c69b-9cbf-406e-ae51-744f2b27c5fa\") " pod="ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj"
Jul 29 14:54:11.553800 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:11.553780    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-dshm\") pod \"ipp-route-19afc1-kserve-7d7b8f5795-q42gj\" (UID: \"29c7c69b-9cbf-406e-ae51-744f2b27c5fa\") " pod="ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj"
Jul 29 14:54:11.654286 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:11.654238    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-home\") pod \"ipp-route-19afc1-kserve-7d7b8f5795-q42gj\" (UID: \"29c7c69b-9cbf-406e-ae51-744f2b27c5fa\") " pod="ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj"
Jul 29 14:54:11.654476 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:11.654384    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-tls-certs\") pod \"ipp-route-19afc1-kserve-7d7b8f5795-q42gj\" (UID: \"29c7c69b-9cbf-406e-ae51-744f2b27c5fa\") " pod="ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj"
Jul 29 14:54:11.654476 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:11.654406    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-model-cache\") pod \"ipp-route-19afc1-kserve-7d7b8f5795-q42gj\" (UID: \"29c7c69b-9cbf-406e-ae51-744f2b27c5fa\") " pod="ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj"
Jul 29 14:54:11.654476 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:11.654428    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xppb\" (UniqueName: \"kubernetes.io/projected/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-kube-api-access-7xppb\") pod \"ipp-route-19afc1-kserve-7d7b8f5795-q42gj\" (UID: \"29c7c69b-9cbf-406e-ae51-744f2b27c5fa\") " pod="ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj"
Jul 29 14:54:11.654476 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:11.654454    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-tmp-dir\") pod \"ipp-route-19afc1-kserve-7d7b8f5795-q42gj\" (UID: \"29c7c69b-9cbf-406e-ae51-744f2b27c5fa\") " pod="ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj"
Jul 29 14:54:11.654739 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:11.654716    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-home\") pod \"ipp-route-19afc1-kserve-7d7b8f5795-q42gj\" (UID: \"29c7c69b-9cbf-406e-ae51-744f2b27c5fa\") " pod="ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj"
Jul 29 14:54:11.654794 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:11.654730    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-dshm\") pod \"ipp-route-19afc1-kserve-7d7b8f5795-q42gj\" (UID: \"29c7c69b-9cbf-406e-ae51-744f2b27c5fa\") " pod="ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj"
Jul 29 14:54:11.654794 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:11.654777    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-model-cache\") pod \"ipp-route-19afc1-kserve-7d7b8f5795-q42gj\" (UID: \"29c7c69b-9cbf-406e-ae51-744f2b27c5fa\") " pod="ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj"
Jul 29 14:54:11.654903 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:11.654885    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-tmp-dir\") pod \"ipp-route-19afc1-kserve-7d7b8f5795-q42gj\" (UID: \"29c7c69b-9cbf-406e-ae51-744f2b27c5fa\") " pod="ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj"
Jul 29 14:54:11.656904 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:11.656883    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-dshm\") pod \"ipp-route-19afc1-kserve-7d7b8f5795-q42gj\" (UID: \"29c7c69b-9cbf-406e-ae51-744f2b27c5fa\") " pod="ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj"
Jul 29 14:54:11.656966 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:11.656924    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-tls-certs\") pod \"ipp-route-19afc1-kserve-7d7b8f5795-q42gj\" (UID: \"29c7c69b-9cbf-406e-ae51-744f2b27c5fa\") " pod="ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj"
Jul 29 14:54:11.663613 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:11.663591    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xppb\" (UniqueName: \"kubernetes.io/projected/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-kube-api-access-7xppb\") pod \"ipp-route-19afc1-kserve-7d7b8f5795-q42gj\" (UID: \"29c7c69b-9cbf-406e-ae51-744f2b27c5fa\") " pod="ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj"
Jul 29 14:54:11.758864 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:11.758827    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj"
Jul 29 14:54:11.892872 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:11.892844    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj"]
Jul 29 14:54:11.895164 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:54:11.895140    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod29c7c69b_9cbf_406e_ae51_744f2b27c5fa.slice/crio-e68a9eab4625e6044ad83b2d4ea4b56469d2c896dccf2156bca89815fffe8943 WatchSource:0}: Error finding container e68a9eab4625e6044ad83b2d4ea4b56469d2c896dccf2156bca89815fffe8943: Status 404 returned error can't find the container with id e68a9eab4625e6044ad83b2d4ea4b56469d2c896dccf2156bca89815fffe8943
Jul 29 14:54:12.177051 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:12.176953    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj" event={"ID":"29c7c69b-9cbf-406e-ae51-744f2b27c5fa","Type":"ContainerStarted","Data":"1ccb94da087da5e3c52d12327ec200f3a86c831636c0b066e6c118065a0c0dee"}
Jul 29 14:54:12.177051 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:12.177000    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj" event={"ID":"29c7c69b-9cbf-406e-ae51-744f2b27c5fa","Type":"ContainerStarted","Data":"e68a9eab4625e6044ad83b2d4ea4b56469d2c896dccf2156bca89815fffe8943"}
Jul 29 14:54:12.195797 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:12.195744    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj" podStartSLOduration=1.19572827 podStartE2EDuration="1.19572827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:54:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:54:12.194277992 +0000 UTC m=+3182.084949327" watchObservedRunningTime="2026-07-29 14:54:12.19572827 +0000 UTC m=+3182.086399605"
Jul 29 14:54:21.759856 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:21.759819    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj"
Jul 29 14:54:21.760258 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:21.759869    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj"
Jul 29 14:54:21.772182 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:21.772156    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj"
Jul 29 14:54:22.231579 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:22.231493    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj"
Jul 29 14:54:54.462355 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:54.462313    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:54:54.783415 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:54.783379    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:54:55.103583 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:55.103495    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:54:58.294107 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:58.294049    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:54:58.389742 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:58.389691    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:54:58.469035 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:58.469001    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:54:58.726793 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:58.726691    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:54:59.140469 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:59.140434    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:54:59.241130 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:54:59.241080    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:55:23.215179 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:55:23.215143    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:55:29.363365 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:55:29.363326    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:55:29.494246 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:55:29.494206    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:55:29.635436 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:55:29.635341    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:55:29.753401 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:55:29.753362    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:55:30.188216 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:55:30.188179    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:55:36.864349 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:55:36.864313    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:55:36.976639 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:55:36.976608    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:55:37.321872 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:55:37.321836    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:55:37.531531 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:55:37.531472    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:55:58.489098 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:55:58.489044    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:55:58.604776 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:55:58.604743    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:56:02.364691 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:02.364648    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:56:03.468262 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:03.468223    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj"]
Jul 29 14:56:03.539579 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:03.539537    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:56:10.800479 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:10.800452    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdw8l_b93f6faa-c8bf-43e9-94e0-b3dcd97b7296/ovn-acl-logging/0.log"
Jul 29 14:56:10.809981 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:10.809957    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdw8l_b93f6faa-c8bf-43e9-94e0-b3dcd97b7296/ovn-acl-logging/0.log"
Jul 29 14:56:18.487188 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:18.487150    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj" podUID="29c7c69b-9cbf-406e-ae51-744f2b27c5fa" containerName="main" containerID="cri-o://1ccb94da087da5e3c52d12327ec200f3a86c831636c0b066e6c118065a0c0dee" gracePeriod=45
Jul 29 14:56:18.723553 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:18.723522    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj"
Jul 29 14:56:18.745324 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:18.745283    2559 generic.go:356] "Generic (PLEG): container finished" podID="29c7c69b-9cbf-406e-ae51-744f2b27c5fa" containerID="1ccb94da087da5e3c52d12327ec200f3a86c831636c0b066e6c118065a0c0dee" exitCode=0
Jul 29 14:56:18.745479 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:18.745324    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj"
Jul 29 14:56:18.745479 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:18.745375    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj" event={"ID":"29c7c69b-9cbf-406e-ae51-744f2b27c5fa","Type":"ContainerDied","Data":"1ccb94da087da5e3c52d12327ec200f3a86c831636c0b066e6c118065a0c0dee"}
Jul 29 14:56:18.745479 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:18.745427    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj" event={"ID":"29c7c69b-9cbf-406e-ae51-744f2b27c5fa","Type":"ContainerDied","Data":"e68a9eab4625e6044ad83b2d4ea4b56469d2c896dccf2156bca89815fffe8943"}
Jul 29 14:56:18.745479 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:18.745450    2559 scope.go:117] "RemoveContainer" containerID="1ccb94da087da5e3c52d12327ec200f3a86c831636c0b066e6c118065a0c0dee"
Jul 29 14:56:18.756007 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:18.755990    2559 scope.go:117] "RemoveContainer" containerID="1ccb94da087da5e3c52d12327ec200f3a86c831636c0b066e6c118065a0c0dee"
Jul 29 14:56:18.756415 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:56:18.756391    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ccb94da087da5e3c52d12327ec200f3a86c831636c0b066e6c118065a0c0dee\": container with ID starting with 1ccb94da087da5e3c52d12327ec200f3a86c831636c0b066e6c118065a0c0dee not found: ID does not exist" containerID="1ccb94da087da5e3c52d12327ec200f3a86c831636c0b066e6c118065a0c0dee"
Jul 29 14:56:18.756489 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:18.756429    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ccb94da087da5e3c52d12327ec200f3a86c831636c0b066e6c118065a0c0dee"} err="failed to get container status \"1ccb94da087da5e3c52d12327ec200f3a86c831636c0b066e6c118065a0c0dee\": rpc error: code = NotFound desc = could not find container \"1ccb94da087da5e3c52d12327ec200f3a86c831636c0b066e6c118065a0c0dee\": container with ID starting with 1ccb94da087da5e3c52d12327ec200f3a86c831636c0b066e6c118065a0c0dee not found: ID does not exist"
Jul 29 14:56:18.786737 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:18.786713    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-home\") pod \"29c7c69b-9cbf-406e-ae51-744f2b27c5fa\" (UID: \"29c7c69b-9cbf-406e-ae51-744f2b27c5fa\") "
Jul 29 14:56:18.786852 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:18.786764    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-tmp-dir\") pod \"29c7c69b-9cbf-406e-ae51-744f2b27c5fa\" (UID: \"29c7c69b-9cbf-406e-ae51-744f2b27c5fa\") "
Jul 29 14:56:18.786852 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:18.786791    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7xppb\" (UniqueName: \"kubernetes.io/projected/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-kube-api-access-7xppb\") pod \"29c7c69b-9cbf-406e-ae51-744f2b27c5fa\" (UID: \"29c7c69b-9cbf-406e-ae51-744f2b27c5fa\") "
Jul 29 14:56:18.786852 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:18.786830    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-tls-certs\") pod \"29c7c69b-9cbf-406e-ae51-744f2b27c5fa\" (UID: \"29c7c69b-9cbf-406e-ae51-744f2b27c5fa\") "
Jul 29 14:56:18.786852 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:18.786851    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-dshm\") pod \"29c7c69b-9cbf-406e-ae51-744f2b27c5fa\" (UID: \"29c7c69b-9cbf-406e-ae51-744f2b27c5fa\") "
Jul 29 14:56:18.787080 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:18.786893    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-model-cache\") pod \"29c7c69b-9cbf-406e-ae51-744f2b27c5fa\" (UID: \"29c7c69b-9cbf-406e-ae51-744f2b27c5fa\") "
Jul 29 14:56:18.787080 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:18.787030    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "29c7c69b-9cbf-406e-ae51-744f2b27c5fa" (UID: "29c7c69b-9cbf-406e-ae51-744f2b27c5fa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:56:18.787251 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:18.787222    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-model-cache" (OuterVolumeSpecName: "model-cache") pod "29c7c69b-9cbf-406e-ae51-744f2b27c5fa" (UID: "29c7c69b-9cbf-406e-ae51-744f2b27c5fa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:56:18.787334 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:18.787253    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-home" (OuterVolumeSpecName: "home") pod "29c7c69b-9cbf-406e-ae51-744f2b27c5fa" (UID: "29c7c69b-9cbf-406e-ae51-744f2b27c5fa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:56:18.787334 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:18.787273    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-tmp-dir\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:56:18.789256 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:18.789235    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "29c7c69b-9cbf-406e-ae51-744f2b27c5fa" (UID: "29c7c69b-9cbf-406e-ae51-744f2b27c5fa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 14:56:18.789340 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:18.789323    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-dshm" (OuterVolumeSpecName: "dshm") pod "29c7c69b-9cbf-406e-ae51-744f2b27c5fa" (UID: "29c7c69b-9cbf-406e-ae51-744f2b27c5fa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:56:18.789425 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:18.789404    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-kube-api-access-7xppb" (OuterVolumeSpecName: "kube-api-access-7xppb") pod "29c7c69b-9cbf-406e-ae51-744f2b27c5fa" (UID: "29c7c69b-9cbf-406e-ae51-744f2b27c5fa"). InnerVolumeSpecName "kube-api-access-7xppb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:56:18.888462 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:18.888431    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-home\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:56:18.888462 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:18.888458    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xppb\" (UniqueName: \"kubernetes.io/projected/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-kube-api-access-7xppb\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:56:18.888462 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:18.888468    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-tls-certs\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:56:18.888677 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:18.888476    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-dshm\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:56:18.888677 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:18.888486    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29c7c69b-9cbf-406e-ae51-744f2b27c5fa-model-cache\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:56:19.069711 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:19.069676    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj"]
Jul 29 14:56:19.080325 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:19.079878    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-19afc1/ipp-route-19afc1-kserve-7d7b8f5795-q42gj"]
Jul 29 14:56:20.457680 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:20.457643    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:56:20.635963 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:20.635930    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29c7c69b-9cbf-406e-ae51-744f2b27c5fa" path="/var/lib/kubelet/pods/29c7c69b-9cbf-406e-ae51-744f2b27c5fa/volumes"
Jul 29 14:56:40.077032 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:40.076942    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:56:41.444914 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:41.444863    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:56:48.138401 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:48.138349    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:56:48.218087 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:48.218037    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:56:48.489295 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:48.489205    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"]
Jul 29 14:56:48.489500 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:48.489475    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv" podUID="18d0724a-eb28-4815-a8a4-48d958dda388" containerName="istio-proxy" containerID="cri-o://5f11f2db8ea90f209025188ab98a7177fd30d46511ebcd688a54a463bf84d74b" gracePeriod=30
Jul 29 14:56:48.516440 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:48.516398    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:56:48.794439 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:48.794401    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:56:53.736299 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.736275    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:56:53.789584 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.789499    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18d0724a-eb28-4815-a8a4-48d958dda388-istio-token\") pod \"18d0724a-eb28-4815-a8a4-48d958dda388\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") "
Jul 29 14:56:53.789584 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.789551    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18d0724a-eb28-4815-a8a4-48d958dda388-istiod-ca-cert\") pod \"18d0724a-eb28-4815-a8a4-48d958dda388\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") "
Jul 29 14:56:53.789584 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.789575    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18d0724a-eb28-4815-a8a4-48d958dda388-istio-podinfo\") pod \"18d0724a-eb28-4815-a8a4-48d958dda388\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") "
Jul 29 14:56:53.789848 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.789674    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18d0724a-eb28-4815-a8a4-48d958dda388-workload-socket\") pod \"18d0724a-eb28-4815-a8a4-48d958dda388\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") "
Jul 29 14:56:53.789848 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.789725    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18d0724a-eb28-4815-a8a4-48d958dda388-istio-data\") pod \"18d0724a-eb28-4815-a8a4-48d958dda388\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") "
Jul 29 14:56:53.789848 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.789754    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wz595\" (UniqueName: \"kubernetes.io/projected/18d0724a-eb28-4815-a8a4-48d958dda388-kube-api-access-wz595\") pod \"18d0724a-eb28-4815-a8a4-48d958dda388\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") "
Jul 29 14:56:53.789848 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.789781    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18d0724a-eb28-4815-a8a4-48d958dda388-workload-certs\") pod \"18d0724a-eb28-4815-a8a4-48d958dda388\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") "
Jul 29 14:56:53.789848 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.789814    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18d0724a-eb28-4815-a8a4-48d958dda388-credential-socket\") pod \"18d0724a-eb28-4815-a8a4-48d958dda388\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") "
Jul 29 14:56:53.790112 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.789912    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18d0724a-eb28-4815-a8a4-48d958dda388-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "18d0724a-eb28-4815-a8a4-48d958dda388" (UID: "18d0724a-eb28-4815-a8a4-48d958dda388"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:56:53.790112 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.789927    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18d0724a-eb28-4815-a8a4-48d958dda388-istio-envoy\") pod \"18d0724a-eb28-4815-a8a4-48d958dda388\" (UID: \"18d0724a-eb28-4815-a8a4-48d958dda388\") "
Jul 29 14:56:53.790112 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.790010    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18d0724a-eb28-4815-a8a4-48d958dda388-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "18d0724a-eb28-4815-a8a4-48d958dda388" (UID: "18d0724a-eb28-4815-a8a4-48d958dda388"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 14:56:53.790268 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.790175    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18d0724a-eb28-4815-a8a4-48d958dda388-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "18d0724a-eb28-4815-a8a4-48d958dda388" (UID: "18d0724a-eb28-4815-a8a4-48d958dda388"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:56:53.790268 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.790192    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18d0724a-eb28-4815-a8a4-48d958dda388-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "18d0724a-eb28-4815-a8a4-48d958dda388" (UID: "18d0724a-eb28-4815-a8a4-48d958dda388"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:56:53.790357 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.790283    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18d0724a-eb28-4815-a8a4-48d958dda388-istiod-ca-cert\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:56:53.790357 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.790300    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18d0724a-eb28-4815-a8a4-48d958dda388-workload-socket\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:56:53.790357 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.790313    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18d0724a-eb28-4815-a8a4-48d958dda388-workload-certs\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:56:53.790357 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.790327    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18d0724a-eb28-4815-a8a4-48d958dda388-credential-socket\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:56:53.790825 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.790800    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18d0724a-eb28-4815-a8a4-48d958dda388-istio-data" (OuterVolumeSpecName: "istio-data") pod "18d0724a-eb28-4815-a8a4-48d958dda388" (UID: "18d0724a-eb28-4815-a8a4-48d958dda388"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:56:53.791773 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.791748    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/18d0724a-eb28-4815-a8a4-48d958dda388-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "18d0724a-eb28-4815-a8a4-48d958dda388" (UID: "18d0724a-eb28-4815-a8a4-48d958dda388"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 14:56:53.792041 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.792019    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18d0724a-eb28-4815-a8a4-48d958dda388-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "18d0724a-eb28-4815-a8a4-48d958dda388" (UID: "18d0724a-eb28-4815-a8a4-48d958dda388"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:56:53.792282 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.792259    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18d0724a-eb28-4815-a8a4-48d958dda388-kube-api-access-wz595" (OuterVolumeSpecName: "kube-api-access-wz595") pod "18d0724a-eb28-4815-a8a4-48d958dda388" (UID: "18d0724a-eb28-4815-a8a4-48d958dda388"). InnerVolumeSpecName "kube-api-access-wz595". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:56:53.792350 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.792262    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18d0724a-eb28-4815-a8a4-48d958dda388-istio-token" (OuterVolumeSpecName: "istio-token") pod "18d0724a-eb28-4815-a8a4-48d958dda388" (UID: "18d0724a-eb28-4815-a8a4-48d958dda388"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:56:53.891448 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.891412    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18d0724a-eb28-4815-a8a4-48d958dda388-istio-envoy\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:56:53.891448 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.891440    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18d0724a-eb28-4815-a8a4-48d958dda388-istio-token\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:56:53.891448 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.891449    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18d0724a-eb28-4815-a8a4-48d958dda388-istio-podinfo\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:56:53.891448 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.891457    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18d0724a-eb28-4815-a8a4-48d958dda388-istio-data\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:56:53.891703 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.891466    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wz595\" (UniqueName: \"kubernetes.io/projected/18d0724a-eb28-4815-a8a4-48d958dda388-kube-api-access-wz595\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:56:53.906148 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.906114    2559 generic.go:356] "Generic (PLEG): container finished" podID="18d0724a-eb28-4815-a8a4-48d958dda388" containerID="5f11f2db8ea90f209025188ab98a7177fd30d46511ebcd688a54a463bf84d74b" exitCode=0
Jul 29 14:56:53.906267 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.906163    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv" event={"ID":"18d0724a-eb28-4815-a8a4-48d958dda388","Type":"ContainerDied","Data":"5f11f2db8ea90f209025188ab98a7177fd30d46511ebcd688a54a463bf84d74b"}
Jul 29 14:56:53.906267 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.906175    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"
Jul 29 14:56:53.906267 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.906218    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv" event={"ID":"18d0724a-eb28-4815-a8a4-48d958dda388","Type":"ContainerDied","Data":"859b3cf8d1dd10c7730fd5b00ad1e770ec58711a3823f12da11fed0112876095"}
Jul 29 14:56:53.906267 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.906247    2559 scope.go:117] "RemoveContainer" containerID="5f11f2db8ea90f209025188ab98a7177fd30d46511ebcd688a54a463bf84d74b"
Jul 29 14:56:53.915210 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.915192    2559 scope.go:117] "RemoveContainer" containerID="2d530ef2f60ddcb6a940ecd2b56aa1009b543bcb9bb0921a8e8b3a3244a206b1"
Jul 29 14:56:53.922713 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.922694    2559 scope.go:117] "RemoveContainer" containerID="5f11f2db8ea90f209025188ab98a7177fd30d46511ebcd688a54a463bf84d74b"
Jul 29 14:56:53.922962 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:56:53.922941    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f11f2db8ea90f209025188ab98a7177fd30d46511ebcd688a54a463bf84d74b\": container with ID starting with 5f11f2db8ea90f209025188ab98a7177fd30d46511ebcd688a54a463bf84d74b not found: ID does not exist" containerID="5f11f2db8ea90f209025188ab98a7177fd30d46511ebcd688a54a463bf84d74b"
Jul 29 14:56:53.923036 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.922973    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f11f2db8ea90f209025188ab98a7177fd30d46511ebcd688a54a463bf84d74b"} err="failed to get container status \"5f11f2db8ea90f209025188ab98a7177fd30d46511ebcd688a54a463bf84d74b\": rpc error: code = NotFound desc = could not find container \"5f11f2db8ea90f209025188ab98a7177fd30d46511ebcd688a54a463bf84d74b\": container with ID starting with 5f11f2db8ea90f209025188ab98a7177fd30d46511ebcd688a54a463bf84d74b not found: ID does not exist"
Jul 29 14:56:53.923036 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.922998    2559 scope.go:117] "RemoveContainer" containerID="2d530ef2f60ddcb6a940ecd2b56aa1009b543bcb9bb0921a8e8b3a3244a206b1"
Jul 29 14:56:53.923247 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:56:53.923231    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d530ef2f60ddcb6a940ecd2b56aa1009b543bcb9bb0921a8e8b3a3244a206b1\": container with ID starting with 2d530ef2f60ddcb6a940ecd2b56aa1009b543bcb9bb0921a8e8b3a3244a206b1 not found: ID does not exist" containerID="2d530ef2f60ddcb6a940ecd2b56aa1009b543bcb9bb0921a8e8b3a3244a206b1"
Jul 29 14:56:53.923305 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.923254    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d530ef2f60ddcb6a940ecd2b56aa1009b543bcb9bb0921a8e8b3a3244a206b1"} err="failed to get container status \"2d530ef2f60ddcb6a940ecd2b56aa1009b543bcb9bb0921a8e8b3a3244a206b1\": rpc error: code = NotFound desc = could not find container \"2d530ef2f60ddcb6a940ecd2b56aa1009b543bcb9bb0921a8e8b3a3244a206b1\": container with ID starting with 2d530ef2f60ddcb6a940ecd2b56aa1009b543bcb9bb0921a8e8b3a3244a206b1 not found: ID does not exist"
Jul 29 14:56:53.930883 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.930860    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"]
Jul 29 14:56:53.937423 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:53.937403    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-19afc1-openshift-default-67dd8ffddd-m9tcv"]
Jul 29 14:56:54.636166 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:54.636131    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18d0724a-eb28-4815-a8a4-48d958dda388" path="/var/lib/kubelet/pods/18d0724a-eb28-4815-a8a4-48d958dda388/volumes"
Jul 29 14:56:54.793732 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:54.793701    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:56:59.629760 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:59.629725    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:56:59.708048 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:59.708009    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:56:59.792710 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:59.792674    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:56:59.897320 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:56:59.897219    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:57:00.333475 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:00.333431    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:57:06.558723 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:06.558680    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:57:06.630274 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:06.630238    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:57:06.845927 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:06.845849    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"]
Jul 29 14:57:06.846099 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:06.846075    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq" podUID="7eb92074-59a2-4c68-8722-51c2e97a2d8e" containerName="istio-proxy" containerID="cri-o://6b5909b289f67880e6be7616889ebefc20deb1c870c89e9dea096969fd734bdf" gracePeriod=30
Jul 29 14:57:06.873558 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:06.873522    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:57:07.116381 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:07.116292    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:57:11.989971 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:11.989936    2559 generic.go:356] "Generic (PLEG): container finished" podID="7eb92074-59a2-4c68-8722-51c2e97a2d8e" containerID="6b5909b289f67880e6be7616889ebefc20deb1c870c89e9dea096969fd734bdf" exitCode=0
Jul 29 14:57:11.990446 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:11.990006    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq" event={"ID":"7eb92074-59a2-4c68-8722-51c2e97a2d8e","Type":"ContainerDied","Data":"6b5909b289f67880e6be7616889ebefc20deb1c870c89e9dea096969fd734bdf"}
Jul 29 14:57:11.990446 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:11.990137    2559 scope.go:117] "RemoveContainer" containerID="835c811f0c0a081f405b0c1cb19391e7b59b955eb6ae282d71e58a0cbb82faaf"
Jul 29 14:57:12.104282 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:12.104255    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:57:12.257095 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:12.257049    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7eb92074-59a2-4c68-8722-51c2e97a2d8e-istio-envoy\") pod \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") "
Jul 29 14:57:12.257277 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:12.257101    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7eb92074-59a2-4c68-8722-51c2e97a2d8e-credential-socket\") pod \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") "
Jul 29 14:57:12.257277 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:12.257125    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7eb92074-59a2-4c68-8722-51c2e97a2d8e-istio-podinfo\") pod \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") "
Jul 29 14:57:12.257277 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:12.257150    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7eb92074-59a2-4c68-8722-51c2e97a2d8e-workload-certs\") pod \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") "
Jul 29 14:57:12.257277 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:12.257173    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7eb92074-59a2-4c68-8722-51c2e97a2d8e-istio-token\") pod \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") "
Jul 29 14:57:12.257277 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:12.257229    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7eb92074-59a2-4c68-8722-51c2e97a2d8e-istiod-ca-cert\") pod \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") "
Jul 29 14:57:12.257277 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:12.257267    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7eb92074-59a2-4c68-8722-51c2e97a2d8e-workload-socket\") pod \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") "
Jul 29 14:57:12.257594 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:12.257298    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qsz2n\" (UniqueName: \"kubernetes.io/projected/7eb92074-59a2-4c68-8722-51c2e97a2d8e-kube-api-access-qsz2n\") pod \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") "
Jul 29 14:57:12.257594 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:12.257321    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7eb92074-59a2-4c68-8722-51c2e97a2d8e-istio-data\") pod \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\" (UID: \"7eb92074-59a2-4c68-8722-51c2e97a2d8e\") "
Jul 29 14:57:12.257594 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:12.257469    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7eb92074-59a2-4c68-8722-51c2e97a2d8e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7eb92074-59a2-4c68-8722-51c2e97a2d8e" (UID: "7eb92074-59a2-4c68-8722-51c2e97a2d8e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:57:12.257594 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:12.257518    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7eb92074-59a2-4c68-8722-51c2e97a2d8e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7eb92074-59a2-4c68-8722-51c2e97a2d8e" (UID: "7eb92074-59a2-4c68-8722-51c2e97a2d8e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:57:12.257812 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:12.257717    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7eb92074-59a2-4c68-8722-51c2e97a2d8e-workload-certs\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:57:12.257812 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:12.257739    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7eb92074-59a2-4c68-8722-51c2e97a2d8e-credential-socket\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:57:12.257812 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:12.257773    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7eb92074-59a2-4c68-8722-51c2e97a2d8e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7eb92074-59a2-4c68-8722-51c2e97a2d8e" (UID: "7eb92074-59a2-4c68-8722-51c2e97a2d8e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 14:57:12.257955 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:12.257927    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7eb92074-59a2-4c68-8722-51c2e97a2d8e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7eb92074-59a2-4c68-8722-51c2e97a2d8e" (UID: "7eb92074-59a2-4c68-8722-51c2e97a2d8e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:57:12.258177 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:12.258162    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7eb92074-59a2-4c68-8722-51c2e97a2d8e-istio-data" (OuterVolumeSpecName: "istio-data") pod "7eb92074-59a2-4c68-8722-51c2e97a2d8e" (UID: "7eb92074-59a2-4c68-8722-51c2e97a2d8e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:57:12.259783 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:12.259750    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7eb92074-59a2-4c68-8722-51c2e97a2d8e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7eb92074-59a2-4c68-8722-51c2e97a2d8e" (UID: "7eb92074-59a2-4c68-8722-51c2e97a2d8e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 14:57:12.259783 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:12.259753    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7eb92074-59a2-4c68-8722-51c2e97a2d8e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7eb92074-59a2-4c68-8722-51c2e97a2d8e" (UID: "7eb92074-59a2-4c68-8722-51c2e97a2d8e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:57:12.259948 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:12.259856    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7eb92074-59a2-4c68-8722-51c2e97a2d8e-istio-token" (OuterVolumeSpecName: "istio-token") pod "7eb92074-59a2-4c68-8722-51c2e97a2d8e" (UID: "7eb92074-59a2-4c68-8722-51c2e97a2d8e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:57:12.259948 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:12.259863    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7eb92074-59a2-4c68-8722-51c2e97a2d8e-kube-api-access-qsz2n" (OuterVolumeSpecName: "kube-api-access-qsz2n") pod "7eb92074-59a2-4c68-8722-51c2e97a2d8e" (UID: "7eb92074-59a2-4c68-8722-51c2e97a2d8e"). InnerVolumeSpecName "kube-api-access-qsz2n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:57:12.358476 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:12.358447    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qsz2n\" (UniqueName: \"kubernetes.io/projected/7eb92074-59a2-4c68-8722-51c2e97a2d8e-kube-api-access-qsz2n\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:57:12.358476 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:12.358471    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7eb92074-59a2-4c68-8722-51c2e97a2d8e-istio-data\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:57:12.358476 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:12.358480    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7eb92074-59a2-4c68-8722-51c2e97a2d8e-istio-envoy\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:57:12.358673 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:12.358488    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7eb92074-59a2-4c68-8722-51c2e97a2d8e-istio-podinfo\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:57:12.358673 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:12.358495    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7eb92074-59a2-4c68-8722-51c2e97a2d8e-istio-token\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:57:12.358673 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:12.358504    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7eb92074-59a2-4c68-8722-51c2e97a2d8e-istiod-ca-cert\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:57:12.358673 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:12.358511    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7eb92074-59a2-4c68-8722-51c2e97a2d8e-workload-socket\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:57:12.995291 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:12.995262    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"
Jul 29 14:57:12.995715 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:12.995266    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq" event={"ID":"7eb92074-59a2-4c68-8722-51c2e97a2d8e","Type":"ContainerDied","Data":"723696965e7f7545689781d69fb043c8ddeb035de01bb8752611d9dd5c944f4a"}
Jul 29 14:57:12.995715 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:12.995399    2559 scope.go:117] "RemoveContainer" containerID="6b5909b289f67880e6be7616889ebefc20deb1c870c89e9dea096969fd734bdf"
Jul 29 14:57:13.016403 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:13.016369    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"]
Jul 29 14:57:13.021981 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:13.021957    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-726aca-openshift-default-5c4f499b-8tmkq"]
Jul 29 14:57:14.636626 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:14.636593    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7eb92074-59a2-4c68-8722-51c2e97a2d8e" path="/var/lib/kubelet/pods/7eb92074-59a2-4c68-8722-51c2e97a2d8e/volumes"
Jul 29 14:57:17.862894 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:17.862860    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x"]
Jul 29 14:57:17.863314 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:17.863099    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x" podUID="11db5568-30e2-4d34-9a60-2b7d4e5670e3" containerName="maas-api" containerID="cri-o://7b12eec7d782bcb3c46eb062c59c80ede193a0d2a197db5badebf10f5691338b" gracePeriod=30
Jul 29 14:57:17.895887 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:17.895857    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:57:17.967929 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:17.967879    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:57:18.023748 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:18.023709    2559 generic.go:356] "Generic (PLEG): container finished" podID="11db5568-30e2-4d34-9a60-2b7d4e5670e3" containerID="7b12eec7d782bcb3c46eb062c59c80ede193a0d2a197db5badebf10f5691338b" exitCode=0
Jul 29 14:57:18.023936 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:18.023788    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x" event={"ID":"11db5568-30e2-4d34-9a60-2b7d4e5670e3","Type":"ContainerDied","Data":"7b12eec7d782bcb3c46eb062c59c80ede193a0d2a197db5badebf10f5691338b"}
Jul 29 14:57:18.030522 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:18.030494    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:57:18.097305 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:18.097247    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:57:18.108609 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:18.108573    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x"
Jul 29 14:57:18.211379 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:18.211289    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbts8\" (UniqueName: \"kubernetes.io/projected/11db5568-30e2-4d34-9a60-2b7d4e5670e3-kube-api-access-mbts8\") pod \"11db5568-30e2-4d34-9a60-2b7d4e5670e3\" (UID: \"11db5568-30e2-4d34-9a60-2b7d4e5670e3\") "
Jul 29 14:57:18.211379 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:18.211328    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11db5568-30e2-4d34-9a60-2b7d4e5670e3-openshift-service-ca\") pod \"11db5568-30e2-4d34-9a60-2b7d4e5670e3\" (UID: \"11db5568-30e2-4d34-9a60-2b7d4e5670e3\") "
Jul 29 14:57:18.211568 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:18.211430    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11db5568-30e2-4d34-9a60-2b7d4e5670e3-maas-api-tls\") pod \"11db5568-30e2-4d34-9a60-2b7d4e5670e3\" (UID: \"11db5568-30e2-4d34-9a60-2b7d4e5670e3\") "
Jul 29 14:57:18.211782 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:18.211752    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11db5568-30e2-4d34-9a60-2b7d4e5670e3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "11db5568-30e2-4d34-9a60-2b7d4e5670e3" (UID: "11db5568-30e2-4d34-9a60-2b7d4e5670e3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 14:57:18.213499 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:18.213467    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11db5568-30e2-4d34-9a60-2b7d4e5670e3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "11db5568-30e2-4d34-9a60-2b7d4e5670e3" (UID: "11db5568-30e2-4d34-9a60-2b7d4e5670e3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 14:57:18.213499 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:18.213492    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11db5568-30e2-4d34-9a60-2b7d4e5670e3-kube-api-access-mbts8" (OuterVolumeSpecName: "kube-api-access-mbts8") pod "11db5568-30e2-4d34-9a60-2b7d4e5670e3" (UID: "11db5568-30e2-4d34-9a60-2b7d4e5670e3"). InnerVolumeSpecName "kube-api-access-mbts8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:57:18.312447 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:18.312412    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbts8\" (UniqueName: \"kubernetes.io/projected/11db5568-30e2-4d34-9a60-2b7d4e5670e3-kube-api-access-mbts8\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:57:18.312447 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:18.312442    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11db5568-30e2-4d34-9a60-2b7d4e5670e3-openshift-service-ca\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:57:18.312447 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:18.312452    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11db5568-30e2-4d34-9a60-2b7d4e5670e3-maas-api-tls\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:57:18.498285 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:18.498243    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:57:19.029419 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:19.029391    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x"
Jul 29 14:57:19.029974 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:19.029390    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x" event={"ID":"11db5568-30e2-4d34-9a60-2b7d4e5670e3","Type":"ContainerDied","Data":"abfbc3993cdde70ea808568ae8f50ded8ff6063ebba35d3f2904bee06d256e2d"}
Jul 29 14:57:19.029974 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:19.029539    2559 scope.go:117] "RemoveContainer" containerID="7b12eec7d782bcb3c46eb062c59c80ede193a0d2a197db5badebf10f5691338b"
Jul 29 14:57:19.046558 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:19.046535    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x"]
Jul 29 14:57:19.050407 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:19.050385    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x"]
Jul 29 14:57:20.637809 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:20.637777    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11db5568-30e2-4d34-9a60-2b7d4e5670e3" path="/var/lib/kubelet/pods/11db5568-30e2-4d34-9a60-2b7d4e5670e3/volumes"
Jul 29 14:57:21.065475 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:21.065433    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-61bc0a-7d8548fb4-cpk5x" podUID="11db5568-30e2-4d34-9a60-2b7d4e5670e3" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.51:8443/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 29 14:57:21.682099 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:21.682046    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:57:22.479344 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:22.479299    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:57:22.581004 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:22.580970    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:57:24.710490 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:24.710451    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:57:24.779432 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:24.779394    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:57:25.010099 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:25.010072    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:57:25.274933 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:25.274853    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:57:35.889280 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:35.889243    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:57:35.975934 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:35.975899    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:57:36.048653 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:36.048615    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:57:36.125210 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:36.125172    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:57:36.517570 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:36.517530    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:57:42.407268 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:42.407230    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:57:42.479997 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:42.479950    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:57:42.672034 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:42.671949    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"]
Jul 29 14:57:42.672259 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:42.672232    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8" podUID="ee8a4c85-69ec-44a7-88a8-8f3e3984c33c" containerName="istio-proxy" containerID="cri-o://0001c77f222d4e4acd31d8bc636d2fa9e83731793e4cb0fc6eab3ec4c0364bde" gracePeriod=30
Jul 29 14:57:42.699080 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:42.699034    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:57:42.969346 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:42.969252    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:57:46.254230 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:46.254196    2559 patch_prober.go:28] interesting pod/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 29 14:57:46.254615 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:46.254252    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8" podUID="ee8a4c85-69ec-44a7-88a8-8f3e3984c33c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 14:57:47.914289 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:47.914264    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:57:47.957589 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:47.957518    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-istiod-ca-cert\") pod \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") "
Jul 29 14:57:47.957589 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:47.957565    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-istio-token\") pod \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") "
Jul 29 14:57:47.957589 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:47.957587    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-istio-podinfo\") pod \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") "
Jul 29 14:57:47.957803 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:47.957655    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tghcv\" (UniqueName: \"kubernetes.io/projected/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-kube-api-access-tghcv\") pod \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") "
Jul 29 14:57:47.957803 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:47.957679    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-workload-certs\") pod \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") "
Jul 29 14:57:47.957908 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:47.957843    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-workload-socket\") pod \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") "
Jul 29 14:57:47.957908 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:47.957896    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-credential-socket\") pod \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") "
Jul 29 14:57:47.958012 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:47.957922    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-istio-envoy\") pod \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") "
Jul 29 14:57:47.958012 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:47.957959    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-istio-data\") pod \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\" (UID: \"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c\") "
Jul 29 14:57:47.958138 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:47.957918    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ee8a4c85-69ec-44a7-88a8-8f3e3984c33c" (UID: "ee8a4c85-69ec-44a7-88a8-8f3e3984c33c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 14:57:47.958138 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:47.958009    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ee8a4c85-69ec-44a7-88a8-8f3e3984c33c" (UID: "ee8a4c85-69ec-44a7-88a8-8f3e3984c33c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:57:47.958230 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:47.958179    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ee8a4c85-69ec-44a7-88a8-8f3e3984c33c" (UID: "ee8a4c85-69ec-44a7-88a8-8f3e3984c33c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:57:47.958230 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:47.958209    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ee8a4c85-69ec-44a7-88a8-8f3e3984c33c" (UID: "ee8a4c85-69ec-44a7-88a8-8f3e3984c33c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:57:47.958394 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:47.958338    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-workload-certs\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:57:47.958394 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:47.958362    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-workload-socket\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:57:47.958394 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:47.958376    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-credential-socket\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:57:47.958394 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:47.958390    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-istiod-ca-cert\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:57:47.959001 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:47.958971    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-istio-data" (OuterVolumeSpecName: "istio-data") pod "ee8a4c85-69ec-44a7-88a8-8f3e3984c33c" (UID: "ee8a4c85-69ec-44a7-88a8-8f3e3984c33c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:57:47.959855 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:47.959829    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ee8a4c85-69ec-44a7-88a8-8f3e3984c33c" (UID: "ee8a4c85-69ec-44a7-88a8-8f3e3984c33c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 14:57:47.959945 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:47.959876    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-istio-token" (OuterVolumeSpecName: "istio-token") pod "ee8a4c85-69ec-44a7-88a8-8f3e3984c33c" (UID: "ee8a4c85-69ec-44a7-88a8-8f3e3984c33c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:57:47.960013 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:47.959991    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-kube-api-access-tghcv" (OuterVolumeSpecName: "kube-api-access-tghcv") pod "ee8a4c85-69ec-44a7-88a8-8f3e3984c33c" (UID: "ee8a4c85-69ec-44a7-88a8-8f3e3984c33c"). InnerVolumeSpecName "kube-api-access-tghcv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 14:57:47.960381 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:47.960365    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ee8a4c85-69ec-44a7-88a8-8f3e3984c33c" (UID: "ee8a4c85-69ec-44a7-88a8-8f3e3984c33c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 14:57:48.059281 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:48.059254    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-istio-token\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:57:48.059281 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:48.059278    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-istio-podinfo\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:57:48.059281 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:48.059287    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tghcv\" (UniqueName: \"kubernetes.io/projected/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-kube-api-access-tghcv\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:57:48.059475 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:48.059297    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-istio-envoy\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:57:48.059475 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:48.059305    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c-istio-data\") on node \"ip-10-0-131-187.ec2.internal\" DevicePath \"\""
Jul 29 14:57:48.163978 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:48.163948    2559 generic.go:356] "Generic (PLEG): container finished" podID="ee8a4c85-69ec-44a7-88a8-8f3e3984c33c" containerID="0001c77f222d4e4acd31d8bc636d2fa9e83731793e4cb0fc6eab3ec4c0364bde" exitCode=0
Jul 29 14:57:48.164158 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:48.163997    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"
Jul 29 14:57:48.164158 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:48.164039    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8" event={"ID":"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c","Type":"ContainerDied","Data":"0001c77f222d4e4acd31d8bc636d2fa9e83731793e4cb0fc6eab3ec4c0364bde"}
Jul 29 14:57:48.164158 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:48.164119    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8" event={"ID":"ee8a4c85-69ec-44a7-88a8-8f3e3984c33c","Type":"ContainerDied","Data":"8035dc4b8bc5e5be20569a92d346bd739e4d3b41cebb8b4cfcc07610cdc7d4c5"}
Jul 29 14:57:48.164158 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:48.164144    2559 scope.go:117] "RemoveContainer" containerID="0001c77f222d4e4acd31d8bc636d2fa9e83731793e4cb0fc6eab3ec4c0364bde"
Jul 29 14:57:48.173256 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:48.173240    2559 scope.go:117] "RemoveContainer" containerID="bb1e48a49013d02a22f2c917a7dfce3478fb3ba6ab6148983d767ac45b840bcb"
Jul 29 14:57:48.180445 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:48.180430    2559 scope.go:117] "RemoveContainer" containerID="0001c77f222d4e4acd31d8bc636d2fa9e83731793e4cb0fc6eab3ec4c0364bde"
Jul 29 14:57:48.180674 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:57:48.180656    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0001c77f222d4e4acd31d8bc636d2fa9e83731793e4cb0fc6eab3ec4c0364bde\": container with ID starting with 0001c77f222d4e4acd31d8bc636d2fa9e83731793e4cb0fc6eab3ec4c0364bde not found: ID does not exist" containerID="0001c77f222d4e4acd31d8bc636d2fa9e83731793e4cb0fc6eab3ec4c0364bde"
Jul 29 14:57:48.180718 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:48.180682    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0001c77f222d4e4acd31d8bc636d2fa9e83731793e4cb0fc6eab3ec4c0364bde"} err="failed to get container status \"0001c77f222d4e4acd31d8bc636d2fa9e83731793e4cb0fc6eab3ec4c0364bde\": rpc error: code = NotFound desc = could not find container \"0001c77f222d4e4acd31d8bc636d2fa9e83731793e4cb0fc6eab3ec4c0364bde\": container with ID starting with 0001c77f222d4e4acd31d8bc636d2fa9e83731793e4cb0fc6eab3ec4c0364bde not found: ID does not exist"
Jul 29 14:57:48.180718 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:48.180696    2559 scope.go:117] "RemoveContainer" containerID="bb1e48a49013d02a22f2c917a7dfce3478fb3ba6ab6148983d767ac45b840bcb"
Jul 29 14:57:48.180885 ip-10-0-131-187 kubenswrapper[2559]: E0729 14:57:48.180870    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb1e48a49013d02a22f2c917a7dfce3478fb3ba6ab6148983d767ac45b840bcb\": container with ID starting with bb1e48a49013d02a22f2c917a7dfce3478fb3ba6ab6148983d767ac45b840bcb not found: ID does not exist" containerID="bb1e48a49013d02a22f2c917a7dfce3478fb3ba6ab6148983d767ac45b840bcb"
Jul 29 14:57:48.180924 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:48.180890    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb1e48a49013d02a22f2c917a7dfce3478fb3ba6ab6148983d767ac45b840bcb"} err="failed to get container status \"bb1e48a49013d02a22f2c917a7dfce3478fb3ba6ab6148983d767ac45b840bcb\": rpc error: code = NotFound desc = could not find container \"bb1e48a49013d02a22f2c917a7dfce3478fb3ba6ab6148983d767ac45b840bcb\": container with ID starting with bb1e48a49013d02a22f2c917a7dfce3478fb3ba6ab6148983d767ac45b840bcb not found: ID does not exist"
Jul 29 14:57:48.186779 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:48.186760    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"]
Jul 29 14:57:48.192748 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:48.192726    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-532c20-openshift-default-d868d5fd8-nksf8"]
Jul 29 14:57:48.636656 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:48.636628    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee8a4c85-69ec-44a7-88a8-8f3e3984c33c" path="/var/lib/kubelet/pods/ee8a4c85-69ec-44a7-88a8-8f3e3984c33c/volumes"
Jul 29 14:57:49.398625 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:49.398599    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-pqrns_1ff31e21-bbe1-41c8-801f-e02d452b7718/server/0.log"
Jul 29 14:57:49.798614 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:49.798588    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49_9b5129c9-2f62-466a-87a5-6076b6c6fa30/copy-manifests/0.log"
Jul 29 14:57:49.807715 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:49.807690    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49_9b5129c9-2f62-466a-87a5-6076b6c6fa30/manager/0.log"
Jul 29 14:57:50.306852 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:50.306817    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-968d69cc5-m82rk_360e93a9-d4c3-463f-9f5a-859e2d222e96/maas-api/0.log"
Jul 29 14:57:53.346445 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:53.346413    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl_27d584d0-3cea-4000-910c-5d089ec8360e/istio-proxy/0.log"
Jul 29 14:57:53.570676 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:53.570630    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b4bcbd496-4w54r_d28ff993-6a2b-42e8-bced-e9c5d34e0ff8/kube-auth-proxy/0.log"
Jul 29 14:57:54.512031 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:54.511976    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-l24gw_33d14efa-fc8e-473a-8252-18f6308f0514/main/0.log"
Jul 29 14:57:54.961649 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:54.961568    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-r2qs7_9df8bffa-4c17-4e30-8770-38ca42f4e2c3/main/0.log"
Jul 29 14:57:55.076589 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:57:55.076563    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-hs5l7_6fd71469-d266-469f-b7f6-972bec639c9d/main/0.log"
Jul 29 14:58:00.649382 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:00.649340    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:58:00.716840 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:00.716806    2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log" limit=1024
Jul 29 14:58:03.576864 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:03.576781    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-84fpq_b0c6da59-c35d-4d20-8346-3dd5afc3a8f3/global-pull-secret-syncer/0.log"
Jul 29 14:58:03.675980 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:03.675945    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-7xpvn_62eed460-c3d4-4d6e-a7ff-cf1290a96c53/konnectivity-agent/0.log"
Jul 29 14:58:03.807368 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:03.807335    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-187.ec2.internal_4dad8c770ea44aa6fdaf6d1c2a53cc88/haproxy/0.log"
Jul 29 14:58:12.544101 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:12.544053    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hk7bj_42fb3745-8927-4c55-a75e-8838f8fcd33a/node-exporter/0.log"
Jul 29 14:58:12.563837 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:12.563814    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hk7bj_42fb3745-8927-4c55-a75e-8838f8fcd33a/kube-rbac-proxy/0.log"
Jul 29 14:58:12.594808 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:12.594782    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hk7bj_42fb3745-8927-4c55-a75e-8838f8fcd33a/init-textfile/0.log"
Jul 29 14:58:13.984805 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:13.984768    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tpxzz/perf-node-gather-daemonset-lppb2"]
Jul 29 14:58:13.985240 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:13.985094    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7eb92074-59a2-4c68-8722-51c2e97a2d8e" containerName="istio-proxy"
Jul 29 14:58:13.985240 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:13.985105    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7eb92074-59a2-4c68-8722-51c2e97a2d8e" containerName="istio-proxy"
Jul 29 14:58:13.985240 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:13.985117    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11db5568-30e2-4d34-9a60-2b7d4e5670e3" containerName="maas-api"
Jul 29 14:58:13.985240 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:13.985122    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="11db5568-30e2-4d34-9a60-2b7d4e5670e3" containerName="maas-api"
Jul 29 14:58:13.985240 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:13.985129    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee8a4c85-69ec-44a7-88a8-8f3e3984c33c" containerName="istio-proxy"
Jul 29 14:58:13.985240 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:13.985135    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee8a4c85-69ec-44a7-88a8-8f3e3984c33c" containerName="istio-proxy"
Jul 29 14:58:13.985240 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:13.985148    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18d0724a-eb28-4815-a8a4-48d958dda388" containerName="istio-proxy"
Jul 29 14:58:13.985240 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:13.985154    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="18d0724a-eb28-4815-a8a4-48d958dda388" containerName="istio-proxy"
Jul 29 14:58:13.985240 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:13.985162    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29c7c69b-9cbf-406e-ae51-744f2b27c5fa" containerName="main"
Jul 29 14:58:13.985240 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:13.985167    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="29c7c69b-9cbf-406e-ae51-744f2b27c5fa" containerName="main"
Jul 29 14:58:13.985240 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:13.985215    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7eb92074-59a2-4c68-8722-51c2e97a2d8e" containerName="istio-proxy"
Jul 29 14:58:13.985240 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:13.985222    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="29c7c69b-9cbf-406e-ae51-744f2b27c5fa" containerName="main"
Jul 29 14:58:13.985240 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:13.985229    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee8a4c85-69ec-44a7-88a8-8f3e3984c33c" containerName="istio-proxy"
Jul 29 14:58:13.985240 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:13.985235    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="18d0724a-eb28-4815-a8a4-48d958dda388" containerName="istio-proxy"
Jul 29 14:58:13.985240 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:13.985240    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="18d0724a-eb28-4815-a8a4-48d958dda388" containerName="istio-proxy"
Jul 29 14:58:13.985240 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:13.985247    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="11db5568-30e2-4d34-9a60-2b7d4e5670e3" containerName="maas-api"
Jul 29 14:58:13.989492 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:13.989470    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tpxzz/perf-node-gather-daemonset-lppb2"
Jul 29 14:58:13.994752 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:13.994722    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tpxzz\"/\"openshift-service-ca.crt\""
Jul 29 14:58:13.994884 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:13.994727    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-tpxzz\"/\"default-dockercfg-2v4xz\""
Jul 29 14:58:13.996291 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:13.996268    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tpxzz\"/\"kube-root-ca.crt\""
Jul 29 14:58:13.998145 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:13.998123    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tpxzz/perf-node-gather-daemonset-lppb2"]
Jul 29 14:58:14.080872 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:14.080833    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7lns\" (UniqueName: \"kubernetes.io/projected/27791736-52d4-482d-b87b-593e92b96b42-kube-api-access-t7lns\") pod \"perf-node-gather-daemonset-lppb2\" (UID: \"27791736-52d4-482d-b87b-593e92b96b42\") " pod="openshift-must-gather-tpxzz/perf-node-gather-daemonset-lppb2"
Jul 29 14:58:14.081092 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:14.080892    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/27791736-52d4-482d-b87b-593e92b96b42-sys\") pod \"perf-node-gather-daemonset-lppb2\" (UID: \"27791736-52d4-482d-b87b-593e92b96b42\") " pod="openshift-must-gather-tpxzz/perf-node-gather-daemonset-lppb2"
Jul 29 14:58:14.081092 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:14.080934    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/27791736-52d4-482d-b87b-593e92b96b42-podres\") pod \"perf-node-gather-daemonset-lppb2\" (UID: \"27791736-52d4-482d-b87b-593e92b96b42\") " pod="openshift-must-gather-tpxzz/perf-node-gather-daemonset-lppb2"
Jul 29 14:58:14.081092 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:14.080971    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/27791736-52d4-482d-b87b-593e92b96b42-proc\") pod \"perf-node-gather-daemonset-lppb2\" (UID: \"27791736-52d4-482d-b87b-593e92b96b42\") " pod="openshift-must-gather-tpxzz/perf-node-gather-daemonset-lppb2"
Jul 29 14:58:14.081092 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:14.080991    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/27791736-52d4-482d-b87b-593e92b96b42-lib-modules\") pod \"perf-node-gather-daemonset-lppb2\" (UID: \"27791736-52d4-482d-b87b-593e92b96b42\") " pod="openshift-must-gather-tpxzz/perf-node-gather-daemonset-lppb2"
Jul 29 14:58:14.182169 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:14.182128    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/27791736-52d4-482d-b87b-593e92b96b42-podres\") pod \"perf-node-gather-daemonset-lppb2\" (UID: \"27791736-52d4-482d-b87b-593e92b96b42\") " pod="openshift-must-gather-tpxzz/perf-node-gather-daemonset-lppb2"
Jul 29 14:58:14.182330 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:14.182188    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/27791736-52d4-482d-b87b-593e92b96b42-proc\") pod \"perf-node-gather-daemonset-lppb2\" (UID: \"27791736-52d4-482d-b87b-593e92b96b42\") " pod="openshift-must-gather-tpxzz/perf-node-gather-daemonset-lppb2"
Jul 29 14:58:14.182330 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:14.182210    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/27791736-52d4-482d-b87b-593e92b96b42-lib-modules\") pod \"perf-node-gather-daemonset-lppb2\" (UID: \"27791736-52d4-482d-b87b-593e92b96b42\") " pod="openshift-must-gather-tpxzz/perf-node-gather-daemonset-lppb2"
Jul 29 14:58:14.182330 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:14.182280    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/27791736-52d4-482d-b87b-593e92b96b42-podres\") pod \"perf-node-gather-daemonset-lppb2\" (UID: \"27791736-52d4-482d-b87b-593e92b96b42\") " pod="openshift-must-gather-tpxzz/perf-node-gather-daemonset-lppb2"
Jul 29 14:58:14.182330 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:14.182314    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/27791736-52d4-482d-b87b-593e92b96b42-proc\") pod \"perf-node-gather-daemonset-lppb2\" (UID: \"27791736-52d4-482d-b87b-593e92b96b42\") " pod="openshift-must-gather-tpxzz/perf-node-gather-daemonset-lppb2"
Jul 29 14:58:14.182521 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:14.182341    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7lns\" (UniqueName: \"kubernetes.io/projected/27791736-52d4-482d-b87b-593e92b96b42-kube-api-access-t7lns\") pod \"perf-node-gather-daemonset-lppb2\" (UID: \"27791736-52d4-482d-b87b-593e92b96b42\") " pod="openshift-must-gather-tpxzz/perf-node-gather-daemonset-lppb2"
Jul 29 14:58:14.182521 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:14.182378    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/27791736-52d4-482d-b87b-593e92b96b42-sys\") pod \"perf-node-gather-daemonset-lppb2\" (UID: \"27791736-52d4-482d-b87b-593e92b96b42\") " pod="openshift-must-gather-tpxzz/perf-node-gather-daemonset-lppb2"
Jul 29 14:58:14.182521 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:14.182340    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/27791736-52d4-482d-b87b-593e92b96b42-lib-modules\") pod \"perf-node-gather-daemonset-lppb2\" (UID: \"27791736-52d4-482d-b87b-593e92b96b42\") " pod="openshift-must-gather-tpxzz/perf-node-gather-daemonset-lppb2"
Jul 29 14:58:14.182521 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:14.182497    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/27791736-52d4-482d-b87b-593e92b96b42-sys\") pod \"perf-node-gather-daemonset-lppb2\" (UID: \"27791736-52d4-482d-b87b-593e92b96b42\") " pod="openshift-must-gather-tpxzz/perf-node-gather-daemonset-lppb2"
Jul 29 14:58:14.190910 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:14.190882    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7lns\" (UniqueName: \"kubernetes.io/projected/27791736-52d4-482d-b87b-593e92b96b42-kube-api-access-t7lns\") pod \"perf-node-gather-daemonset-lppb2\" (UID: \"27791736-52d4-482d-b87b-593e92b96b42\") " pod="openshift-must-gather-tpxzz/perf-node-gather-daemonset-lppb2"
Jul 29 14:58:14.298896 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:14.298853    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tpxzz/perf-node-gather-daemonset-lppb2"
Jul 29 14:58:14.432249 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:14.432222    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tpxzz/perf-node-gather-daemonset-lppb2"]
Jul 29 14:58:14.435026 ip-10-0-131-187 kubenswrapper[2559]: W0729 14:58:14.434998    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod27791736_52d4_482d_b87b_593e92b96b42.slice/crio-da744e0840a2ebd42cc70d54b5a82ccc00db6a3386002749aeb52ffc6dc91932 WatchSource:0}: Error finding container da744e0840a2ebd42cc70d54b5a82ccc00db6a3386002749aeb52ffc6dc91932: Status 404 returned error can't find the container with id da744e0840a2ebd42cc70d54b5a82ccc00db6a3386002749aeb52ffc6dc91932
Jul 29 14:58:14.557781 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:14.557709    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-zmjrn_88728d56-a8af-49d4-a52c-674e9ac76cd9/networking-console-plugin/0.log"
Jul 29 14:58:15.289358 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:15.289321    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tpxzz/perf-node-gather-daemonset-lppb2" event={"ID":"27791736-52d4-482d-b87b-593e92b96b42","Type":"ContainerStarted","Data":"8c653a57dc9125f8cdbc4c665547bbd43f7d0f9d952b956ac6c00bafe86e73c6"}
Jul 29 14:58:15.289867 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:15.289380    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tpxzz/perf-node-gather-daemonset-lppb2" event={"ID":"27791736-52d4-482d-b87b-593e92b96b42","Type":"ContainerStarted","Data":"da744e0840a2ebd42cc70d54b5a82ccc00db6a3386002749aeb52ffc6dc91932"}
Jul 29 14:58:15.289867 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:15.289411    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-tpxzz/perf-node-gather-daemonset-lppb2"
Jul 29 14:58:15.307130 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:15.307082    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tpxzz/perf-node-gather-daemonset-lppb2" podStartSLOduration=2.3070508849999998 podStartE2EDuration="2.307050885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 14:58:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:58:15.305952945 +0000 UTC m=+3425.196624295" watchObservedRunningTime="2026-07-29 14:58:15.307050885 +0000 UTC m=+3425.197722220"
Jul 29 14:58:17.376177 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:17.376142    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-msl82_0b17154a-2770-4e62-8cde-5114d45a8dc2/dns/0.log"
Jul 29 14:58:17.396519 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:17.396486    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-msl82_0b17154a-2770-4e62-8cde-5114d45a8dc2/kube-rbac-proxy/0.log"
Jul 29 14:58:17.476810 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:17.476783    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-d9ncc_22c32d1e-2781-499f-a092-2898013fb37e/dns-node-resolver/0.log"
Jul 29 14:58:18.107152 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:18.107111    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nl5mz_2317c90b-86b0-4ce3-a455-a973befb1b11/node-ca/0.log"
Jul 29 14:58:19.156004 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:19.155970    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl_27d584d0-3cea-4000-910c-5d089ec8360e/istio-proxy/0.log"
Jul 29 14:58:19.324688 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:19.324658    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b4bcbd496-4w54r_d28ff993-6a2b-42e8-bced-e9c5d34e0ff8/kube-auth-proxy/0.log"
Jul 29 14:58:20.385544 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:20.385513    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b76j4_697b63cb-b6c9-4178-892b-822d19e03056/serve-healthcheck-canary/0.log"
Jul 29 14:58:21.085036 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:21.085003    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7s84h_8a8fd4f5-53e9-484a-b530-30e018cb3e51/kube-rbac-proxy/0.log"
Jul 29 14:58:21.108732 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:21.108704    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7s84h_8a8fd4f5-53e9-484a-b530-30e018cb3e51/exporter/0.log"
Jul 29 14:58:21.135002 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:21.134964    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7s84h_8a8fd4f5-53e9-484a-b530-30e018cb3e51/extractor/0.log"
Jul 29 14:58:21.302778 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:21.302750    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-tpxzz/perf-node-gather-daemonset-lppb2"
Jul 29 14:58:24.129017 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:24.128984    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-pqrns_1ff31e21-bbe1-41c8-801f-e02d452b7718/server/0.log"
Jul 29 14:58:24.448350 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:24.448259    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49_9b5129c9-2f62-466a-87a5-6076b6c6fa30/manager/0.log"
Jul 29 14:58:24.474884 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:24.474856    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5ff7cbc5f5-sjw49_9b5129c9-2f62-466a-87a5-6076b6c6fa30/copy-manifests/0.log"
Jul 29 14:58:25.114807 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:25.114773    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl_27d584d0-3cea-4000-910c-5d089ec8360e/istio-proxy/0.log"
Jul 29 14:58:25.276022 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:25.275991    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b4bcbd496-4w54r_d28ff993-6a2b-42e8-bced-e9c5d34e0ff8/kube-auth-proxy/0.log"
Jul 29 14:58:25.927980 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:25.927936    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-f9f5fc96c-qvnsk_6a5c29f8-366e-4e4e-ad77-e406d4213703/manager/0.log"
Jul 29 14:58:26.818438 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:26.818409    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-qzlkl_27d584d0-3cea-4000-910c-5d089ec8360e/istio-proxy/0.log"
Jul 29 14:58:26.981685 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:26.981654    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b4bcbd496-4w54r_d28ff993-6a2b-42e8-bced-e9c5d34e0ff8/kube-auth-proxy/0.log"
Jul 29 14:58:30.851102 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:30.851045    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-jd9td_0c24a7a4-774d-4b1b-a581-c583863a2c8d/migrator/0.log"
Jul 29 14:58:30.871233 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:30.871206    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-jd9td_0c24a7a4-774d-4b1b-a581-c583863a2c8d/graceful-termination/0.log"
Jul 29 14:58:31.228171 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:31.228079    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-5nxst_49f8acf6-08de-4400-adfd-fa32d9a066b5/kube-storage-version-migrator-operator/1.log"
Jul 29 14:58:31.229874 ip-10-0-131-187 kubenswrapper[2559]: I0729 14:58:31.229852    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-5nxst_49f8acf6-08de-4400-adfd-fa32d9a066b5/kube-storage-version-migrator-operator/0.log"